Task 13394268

Name	hadcm3n_t59t_1940_40_007446527_3
Workunit	7644030
Created	17 Sep 2011, 17:17:35 UTC
Sent	17 Sep 2011, 17:33:35 UTC
Report deadline	18 Dec 2011, 1:00:46 UTC
Received	2 Oct 2011, 14:38:02 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1170225
Run time	9 days 0 hours 19 min 40 sec
CPU time	8 days 2 hours 37 min 32 sec
Validate state	Invalid
Credit	3,110.40
Device peak FLOPS	2.31 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.12.34</core_client_version> <![CDATA[ <message> - exit code 193 (0xc1) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3916, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 16:52:19 (2424): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:52:20 (2424): No heartbeat from core client for 30 sec - exiting 16:53:06 (8976): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 17:42:36 (4520): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 19:55:22 (4804): No heartbeat from core client for 30 sec - exiting 19:55:23 (4804): No heartbeat from core client for 30 sec - exiting 19:55:24 (4804): No heartbeat from core client for 30 sec - exiting 19:55:25 (4804): No heartbeat from core client for 30 sec - exiting 19:55:26 (4804): No heartbeat from core client for 30 sec - exiting 19:55:27 (4804): No heartbeat from core client for 30 sec - exiting 19:55:28 (4804): No heartbeat from core client for 30 sec - exiting 19:55:29 (4804): No heartbeat from core client for 30 sec - exiting 19:55:30 (4804): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5980, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4212, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 17:51:08 (4108): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 17:04:42 (3228): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:12:13 (4696): No heartbeat from core client for 30 sec - exiting 18:12:14 (4696): No heartbeat from core client for 30 sec - exiting 18:12:16 (4696): No heartbeat from core client for 30 sec - exiting 18:12:17 (4696): No heartbeat from core client for 30 sec - exiting 18:12:18 (4696): No heartbeat from core client for 30 sec - exiting 18:12:19 (4696): No heartbeat from core client for 30 sec - exiting 18:12:20 (4696): No heartbeat from core client for 30 sec - exiting 18:12:21 (4696): No heartbeat from core client for 30 sec - exiting 18:12:22 (4696): No heartbeat from core client for 30 sec - exiting 18:12:23 (4696): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5516, iMonCtr=1 Model crash detected, will try to restart... 19:25:05 (1976): No heartbeat from core client for 30 sec - exiting 19:25:06 (1976): No heartbeat from core client for 30 sec - exiting 19:25:07 (1976): No heartbeat from core client for 30 sec - exiting 19:25:08 (1976): No heartbeat from core client for 30 sec - exiting 19:25:09 (1976): No heartbeat from core client for 30 sec - exiting 19:25:11 (1976): No heartbeat from core client for 30 sec - exiting 19:25:12 (1976): No heartbeat from core client for 30 sec - exiting 19:25:13 (1976): No heartbeat from core client for 30 sec - exiting 19:25:14 (1976): No heartbeat from core client for 30 sec - exiting 19:25:15 (1976): No heartbeat from core client for 30 sec - exiting 19:25:16 (1976): No heartbeat from core client for 30 sec - exiting 19:25:17 (1976): No heartbeat from core client for 30 sec - exiting 19:25:18 (1976): No heartbeat from core client for 30 sec - exiting 19:25:19 (1976): No heartbeat from core client for 30 sec - exiting 19:25:20 (1976): No heartbeat from core client for 30 sec - exiting 19:25:21 (1976): No heartbeat from core client for 30 sec - exiting 19:25:23 (1976): No heartbeat from core client for 30 sec - exiting 19:25:24 (1976): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77B87353 read attempt to address 0xFFFFFFF8 Engaging BOINC Windows Runtime Debugger... Cannot serialize file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_t59t_1940_40_007446527/dataout/shmem_restart.day Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
02 Oct 2011 14:35:58	1170225	13394268	hadcm3n_t59t_1940_40_007446527_3	259,200	700,634	2.7031
01 Oct 2011 02:57:50	1170225	13394268	hadcm3n_t59t_1940_40_007446527_3	233,280	641,321	2.7491
30 Sep 2011 11:13:27	1170225	13394268	hadcm3n_t59t_1940_40_007446527_3	207,360	589,479	2.8428
29 Sep 2011 11:58:34	1170225	13394268	hadcm3n_t59t_1940_40_007446527_3	181,440	531,326	2.9284
27 Sep 2011 12:35:37	1170225	13394268	hadcm3n_t59t_1940_40_007446527_3	155,520	473,951	3.0475
26 Sep 2011 17:33:57	1170225	13394268	hadcm3n_t59t_1940_40_007446527_3	129,600	417,400	3.2207
25 Sep 2011 03:37:16	1170225	13394268	hadcm3n_t59t_1940_40_007446527_3	103,680	361,136	3.4832
23 Sep 2011 21:18:27	1170225	13394268	hadcm3n_t59t_1940_40_007446527_3	77,760	297,965	3.8319
21 Sep 2011 13:40:17	1170225	13394268	hadcm3n_t59t_1940_40_007446527_3	51,840	179,064	3.4542
19 Sep 2011 00:57:49	1170225	13394268	hadcm3n_t59t_1940_40_007446527_3	25,920	62,689	2.4186