Task 15417555

Name	hadcm3n_zllx_1880_40_008242249_0
Workunit	8397373
Created	29 Oct 2012, 18:44:34 UTC
Sent	29 Oct 2012, 18:44:45 UTC
Report deadline	29 Jan 2013, 2:11:56 UTC
Received	18 Nov 2012, 11:09:01 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1072815
Run time	5 days 17 hours 33 min 50 sec
CPU time	5 days 14 hours 58 min 16 sec
Validate state	Invalid
Credit	3,110.40
Device peak FLOPS	2.65 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.28</core_client_version> <![CDATA[ <message> - exit code 193 (0xc1) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3628, iMonCtr=1 Model crash detected, will try to restart... 11:25:49 (5512): No heartbeat from core client for 30 sec - exiting 11:25:50 (5512): No heartbeat from core client for 30 sec - exiting 11:25:51 (5512): No heartbeat from core client for 30 sec - exiting 11:25:52 (5512): No heartbeat from core client for 30 sec - exiting 11:25:53 (5512): No heartbeat from core client for 30 sec - exiting 11:25:54 (5512): No heartbeat from core client for 30 sec - exiting 11:25:56 (5512): No heartbeat from core client for 30 sec - exiting 11:25:57 (5512): No heartbeat from core client for 30 sec - exiting 11:25:58 (5512): No heartbeat from core client for 30 sec - exiting 11:25:59 (5512): No heartbeat from core client for 30 sec - exiting 11:26:00 (5512): No heartbeat from core client for 30 sec - exiting 11:26:01 (5512): No heartbeat from core client for 30 sec - exiting 11:26:02 (5512): No heartbeat from core client for 30 sec - exiting 11:26:03 (5512): No heartbeat from core client for 30 sec - exiting 11:26:04 (5512): No heartbeat from core client for 30 sec - exiting 11:26:05 (5512): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4592, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5156, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 13:11:01 (4688): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 18:45:31 (6000): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 19:51:26 (5752): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 12:11:41 (5992): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4380, iMonCtr=1 Model crash detected, will try to restart... 18:51:58 (4372): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:49:25 (4392): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 18:49:00 (6124): No heartbeat from core client for 30 sec - exiting 18:49:02 (6124): No heartbeat from core client for 30 sec - exiting 18:49:03 (6124): No heartbeat from core client for 30 sec - exiting 18:49:04 (6124): No heartbeat from core client for 30 sec - exiting 18:49:05 (6124): No heartbeat from core client for 30 sec - exiting 18:49:06 (6124): No heartbeat from core client for 30 sec - exiting 18:49:07 (6124): No heartbeat from core client for 30 sec - exiting 18:49:08 (6124): No heartbeat from core client for 30 sec - exiting 18:49:09 (6124): No heartbeat from core client for 30 sec - exiting 18:49:10 (6124): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:49:56 (4668): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 11:43:39 (1920): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:44:17 (5152): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 12:45:44 (1052): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:46:25 (6564): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 16:40:41 (4676): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 19:39:38 (3228): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 22:38:26 (5280): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 03:07:43 (1932): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 11:33:18 (5668): No heartbeat from core client for 30 sec - exiting 11:33:19 (5668): No heartbeat from core client for 30 sec - exiting 11:33:20 (5668): No heartbeat from core client for 30 sec - exiting 11:33:21 (5668): No heartbeat from core client for 30 sec - exiting 11:33:22 (5668): No heartbeat from core client for 30 sec - exiting 11:33:23 (5668): No heartbeat from core client for 30 sec - exiting 11:33:24 (5668): No heartbeat from core client for 30 sec - exiting 11:33:26 (5668): No heartbeat from core client for 30 sec - exiting 11:33:27 (5668): No heartbeat from core client for 30 sec - exiting 11:33:28 (5668): No heartbeat from core client for 30 sec - exiting 11:33:29 (5668): No heartbeat from core client for 30 sec - exiting 11:33:30 (5668): No heartbeat from core client for 30 sec - exiting 11:33:31 (5668): No heartbeat from core client for 30 sec - exiting 11:33:32 (5668): No heartbeat from core client for 30 sec - exiting 11:33:33 (5668): No heartbeat from core client for 30 sec - exiting 11:33:34 (5668): No heartbeat from core client for 30 sec - exiting 11:33:35 (5668): No heartbeat from core client for 30 sec - exiting 11:33:36 (5668): No heartbeat from core client for 30 sec - exiting 11:33:38 (5668): No heartbeat from core client for 30 sec - exiting 11:33:39 (5668): No heartbeat from core client for 30 sec - exiting 11:33:40 (5668): No heartbeat from core client for 30 sec - exiting 11:33:41 (5668): No heartbeat from core client for 30 sec - exiting 11:33:42 (5668): No heartbeat from core client for 30 sec - exiting 11:33:43 (5668): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:33:44 (5668): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 18:18:16 (4900): No heartbeat from core client for 30 sec - exiting 18:18:17 (4900): No heartbeat from core client for 30 sec - exiting 18:18:18 (4900): No heartbeat from core client for 30 sec - exiting 18:18:19 (4900): No heartbeat from core client for 30 sec - exiting 18:18:20 (4900): No heartbeat from core client for 30 sec - exiting 18:18:21 (4900): No heartbeat from core client for 30 sec - exiting 18:18:22 (4900): No heartbeat from core client for 30 sec - exiting 18:18:23 (4900): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:18:25 (4900): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=648, iMonCtr=1 Model crash detected, will try to restart... 17:37:23 (6180): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 18:07:51 (6044): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 19:34:06 (2388): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 12:37:35 (5668): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77B93AB3 read attempt to address 0x4081B029 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77543AB3 read attempt to address 0x4081B029 Engaging BOINC Windows Runtime Debugger... Cannot serialize file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_zllx_1880_40_008242249/dataout/shmem_restart.day Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
17 Nov 2012 14:41:08	1072815	15417555	hadcm3n_zllx_1880_40_008242249_0	259,200	446,616	1.7231
12 Nov 2012 23:17:05	1072815	15417555	hadcm3n_zllx_1880_40_008242249_0	233,280	401,661	1.7218
10 Nov 2012 19:44:19	1072815	15417555	hadcm3n_zllx_1880_40_008242249_0	207,360	356,377	1.7186
08 Nov 2012 20:58:03	1072815	15417555	hadcm3n_zllx_1880_40_008242249_0	181,440	311,832	1.7187
06 Nov 2012 19:54:30	1072815	15417555	hadcm3n_zllx_1880_40_008242249_0	155,520	267,013	1.7169
03 Nov 2012 21:08:19	1072815	15417555	hadcm3n_zllx_1880_40_008242249_0	129,600	222,536	1.7171
03 Nov 2012 00:08:35	1072815	15417555	hadcm3n_zllx_1880_40_008242249_0	103,680	177,432	1.7113
02 Nov 2012 12:35:28	1072815	15417555	hadcm3n_zllx_1880_40_008242249_0	77,760	133,225	1.7133
01 Nov 2012 14:32:29	1072815	15417555	hadcm3n_zllx_1880_40_008242249_0	51,840	88,911	1.7151
30 Oct 2012 17:47:44	1072815	15417555	hadcm3n_zllx_1880_40_008242249_0	25,920	44,389	1.7125