Task 15357673

Name	hadcm3n_zadv_1880_40_008200165_3
Workunit	8355289
Created	9 Oct 2012, 3:19:48 UTC
Sent	9 Oct 2012, 3:19:51 UTC
Report deadline	8 Jan 2013, 10:47:02 UTC
Received	3 Nov 2012, 23:18:08 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1169472
Run time	8 days 8 hours 14 min 34 sec
CPU time	8 days 7 hours 11 min 29 sec
Validate state	Invalid
Credit	6,220.80
Device peak FLOPS	2.64 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.28</core_client_version> <![CDATA[ <message> - exit code 193 (0xc1) </message> <stderr_txt> Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3592, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3436, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3592, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3364, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3364, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 19:52:34 (3712): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 17:24:40 (4168): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:13:00 (7312): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:57:57 (5080): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4184, iMonCtr=1 Model crash detected, will try to restart... 19:16:58 (3572): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:34:01 (5048): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:23:34 (7544): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:13:10 (3760): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:09:23 (3296): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:07:57 (3384): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 19:19:08 (4880): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3460, iMonCtr=1 Model crash detected, will try to restart... 11:06:04 (7684): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 19:33:34 (3504): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77953AB3 read attempt to address 0x00000000 Engaging BOINC Windows Runtime Debugger... Cannot serialize file D:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_zadv_1880_40_008200165/dataout/shmem_restart.day Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
03 Nov 2012 23:19:11	1169472	15357673	hadcm3n_zadv_1880_40_008200165_3	518,400	717,085	1.3833
02 Nov 2012 18:37:07	1169472	15357673	hadcm3n_zadv_1880_40_008200165_3	492,480	680,912	1.3826
02 Nov 2012 07:14:43	1169472	15357673	hadcm3n_zadv_1880_40_008200165_3	466,560	644,277	1.3809
01 Nov 2012 17:55:31	1169472	15357673	hadcm3n_zadv_1880_40_008200165_3	440,640	607,899	1.3796
31 Oct 2012 00:22:51	1169472	15357673	hadcm3n_zadv_1880_40_008200165_3	414,720	571,722	1.3786
29 Oct 2012 14:12:10	1169472	15357673	hadcm3n_zadv_1880_40_008200165_3	388,800	535,975	1.3785
28 Oct 2012 16:14:45	1169472	15357673	hadcm3n_zadv_1880_40_008200165_3	362,880	499,381	1.3762
27 Oct 2012 17:42:18	1169472	15357673	hadcm3n_zadv_1880_40_008200165_3	336,960	463,612	1.3759
26 Oct 2012 18:24:37	1169472	15357673	hadcm3n_zadv_1880_40_008200165_3	311,040	426,633	1.3716
24 Oct 2012 21:28:30	1169472	15357673	hadcm3n_zadv_1880_40_008200165_3	285,120	391,197	1.3720
24 Oct 2012 11:37:09	1169472	15357673	hadcm3n_zadv_1880_40_008200165_3	259,200	355,857	1.3729
23 Oct 2012 18:06:49	1169472	15357673	hadcm3n_zadv_1880_40_008200165_3	233,280	321,102	1.3765
23 Oct 2012 08:28:10	1169472	15357673	hadcm3n_zadv_1880_40_008200165_3	207,360	286,392	1.3811
22 Oct 2012 21:50:25	1169472	15357673	hadcm3n_zadv_1880_40_008200165_3	181,440	251,902	1.3883
21 Oct 2012 18:26:30	1169472	15357673	hadcm3n_zadv_1880_40_008200165_3	155,520	215,808	1.3877
19 Oct 2012 23:35:05	1169472	15357673	hadcm3n_zadv_1880_40_008200165_3	129,600	179,572	1.3856
19 Oct 2012 12:11:58	1169472	15357673	hadcm3n_zadv_1880_40_008200165_3	103,680	144,713	1.3958
18 Oct 2012 01:00:48	1169472	15357673	hadcm3n_zadv_1880_40_008200165_3	77,760	108,771	1.3988
14 Oct 2012 23:19:10	1169472	15357673	hadcm3n_zadv_1880_40_008200165_3	51,840	72,211	1.3930
13 Oct 2012 22:59:47	1169472	15357673	hadcm3n_zadv_1880_40_008200165_3	25,920	36,592	1.4117