Task 15526866

Name	hadcm3n_3mx8_1940_40_008260726_1
Workunit	8415850
Created	9 Jan 2013, 7:32:41 UTC
Sent	9 Jan 2013, 7:32:43 UTC
Report deadline	10 Apr 2013, 14:59:54 UTC
Received	28 Feb 2013, 5:16:56 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1147086
Run time	25 days 0 hours 50 min 23 sec
CPU time	21 days 23 hours 1 min 52 sec
Validate state	Invalid
Credit	9,331.20
Device peak FLOPS	2.24 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.10.60</core_client_version> <![CDATA[ <message> - exit code 193 (0xc1) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Atmos Hold Restart file rename failed on atmos_restart.hold Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 20:55:34 (8332): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:55:36 (8332): No heartbeat from core client for 30 sec - exiting 20:55:37 (8332): No heartbeat from core client for 30 sec - exiting 20:55:39 (8332): No heartbeat from core client for 30 sec - exiting 20:55:40 (8332): No heartbeat from core client for 30 sec - exiting 20:55:41 (8332): No heartbeat from core client for 30 sec - exiting 20:55:42 (8332): No heartbeat from core client for 30 sec - exiting 20:55:43 (8332): No heartbeat from core client for 30 sec - exiting 20:55:44 (8332): No heartbeat from core client for 30 sec - exiting 20:55:45 (8332): No heartbeat from core client for 30 sec - exiting 20:55:47 (8332): No heartbeat from core client for 30 sec - exiting 20:55:48 (8332): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 16:21:35 (44368): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:22:14 (44368): No heartbeat from core client for 30 sec - exiting 16:22:17 (44368): No heartbeat from core client for 30 sec - exiting 16:35:36 (35616): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:35:45 (35616): No heartbeat from core client for 30 sec - exiting 16:35:47 (35616): No heartbeat from core client for 30 sec - exiting 16:35:48 (35616): No heartbeat from core client for 30 sec - exiting 16:35:49 (35616): No heartbeat from core client for 30 sec - exiting 16:35:50 (35616): No heartbeat from core client for 30 sec - exiting 16:35:51 (35616): No heartbeat from core client for 30 sec - exiting 16:35:52 (35616): No heartbeat from core client for 30 sec - exiting 16:35:53 (35616): No heartbeat from core client for 30 sec - exiting 16:35:54 (35616): No heartbeat from core client for 30 sec - exiting 16:35:55 (35616): No heartbeat from core client for 30 sec - exiting 16:35:56 (35616): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1296, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6260, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=17260, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 07:34:51 (7656): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:34:59 (7656): No heartbeat from core client for 30 sec - exiting 07:35:02 (7656): No heartbeat from core client for 30 sec - exiting 07:35:03 (7656): No heartbeat from core client for 30 sec - exiting 07:35:04 (7656): No heartbeat from core client for 30 sec - exiting 07:35:05 (7656): No heartbeat from core client for 30 sec - exiting 07:35:06 (7656): No heartbeat from core client for 30 sec - exiting 07:35:07 (7656): No heartbeat from core client for 30 sec - exiting 07:35:08 (7656): No heartbeat from core client for 30 sec - exiting 07:35:09 (7656): No heartbeat from core client for 30 sec - exiting 07:35:10 (7656): No heartbeat from core client for 30 sec - exiting 07:35:11 (7656): No heartbeat from core client for 30 sec - exiting 07:35:12 (7656): No heartbeat from core client for 30 sec - exiting 07:35:13 (7656): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x772B3AB3 read attempt to address 0x00000000 Engaging BOINC Windows Runtime Debugger... Cannot serialize file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_3mx8_1940_40_008260726/dataout/shmem_restart.day Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
28 Feb 2013 04:16:35	1147086	15526866	hadcm3n_3mx8_1940_40_008260726_1	777,600	1,897,298	2.4399
26 Feb 2013 14:55:41	1147086	15526866	hadcm3n_3mx8_1940_40_008260726_1	751,680	1,833,301	2.4389
24 Feb 2013 14:15:43	1147086	15526866	hadcm3n_3mx8_1940_40_008260726_1	725,760	1,771,044	2.4403
23 Feb 2013 02:10:43	1147086	15526866	hadcm3n_3mx8_1940_40_008260726_1	699,840	1,708,011	2.4406
22 Feb 2013 07:31:43	1147086	15526866	hadcm3n_3mx8_1940_40_008260726_1	673,920	1,644,986	2.4409
20 Feb 2013 12:28:47	1147086	15526866	hadcm3n_3mx8_1940_40_008260726_1	648,000	1,581,848	2.4411
18 Feb 2013 09:58:34	1147086	15526866	hadcm3n_3mx8_1940_40_008260726_1	622,080	1,518,365	2.4408
17 Feb 2013 00:30:19	1147086	15526866	hadcm3n_3mx8_1940_40_008260726_1	596,160	1,454,773	2.4402
15 Feb 2013 13:20:11	1147086	15526866	hadcm3n_3mx8_1940_40_008260726_1	570,240	1,390,884	2.4391
14 Feb 2013 06:30:14	1147086	15526866	hadcm3n_3mx8_1940_40_008260726_1	544,320	1,326,529	2.4370
12 Feb 2013 02:53:18	1147086	15526866	hadcm3n_3mx8_1940_40_008260726_1	518,400	1,263,320	2.4370
10 Feb 2013 08:31:52	1147086	15526866	hadcm3n_3mx8_1940_40_008260726_1	492,480	1,200,532	2.4377
09 Feb 2013 00:59:52	1147086	15526866	hadcm3n_3mx8_1940_40_008260726_1	466,560	1,137,603	2.4383
07 Feb 2013 23:33:20	1147086	15526866	hadcm3n_3mx8_1940_40_008260726_1	440,640	1,075,117	2.4399
06 Feb 2013 20:11:22	1147086	15526866	hadcm3n_3mx8_1940_40_008260726_1	414,720	1,011,580	2.4392
05 Feb 2013 20:13:29	1147086	15526866	hadcm3n_3mx8_1940_40_008260726_1	388,800	947,725	2.4376
03 Feb 2013 23:10:29	1147086	15526866	hadcm3n_3mx8_1940_40_008260726_1	362,880	884,164	2.4365
02 Feb 2013 20:27:48	1147086	15526866	hadcm3n_3mx8_1940_40_008260726_1	336,960	818,756	2.4298
01 Feb 2013 14:30:44	1147086	15526866	hadcm3n_3mx8_1940_40_008260726_1	311,040	754,730	2.4265
29 Jan 2013 15:50:38	1147086	15526866	hadcm3n_3mx8_1940_40_008260726_1	285,120	691,183	2.4242