Task 16069827

Name	hadcm3n_o3aw_1940_40_008381271_2
Workunit	8532130
Created	18 Oct 2013, 11:34:12 UTC
Sent	18 Oct 2013, 11:34:49 UTC
Report deadline	17 Jan 2014, 19:02:00 UTC
Received	1 Jul 2014, 8:21:25 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1185926
Run time	12 days 12 hours 37 min 45 sec
CPU time	12 days 10 hours 3 min 22 sec
Validate state	Invalid
Credit	9,331.20
Device peak FLOPS	2.45 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.64</core_client_version> <![CDATA[ <message> (unknown error) - exit code 193 (0xc1) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 13:59:45 (3508): No heartbeat from core client for 30 sec - exiting 13:59:46 (3508): No heartbeat from core client for 30 sec - exiting 13:59:47 (3508): No heartbeat from core client for 30 sec - exiting 13:59:48 (3508): No heartbeat from core client for 30 sec - exiting 13:59:49 (3508): No heartbeat from core client for 30 sec - exiting 13:59:50 (3508): No heartbeat from core client for 30 sec - exiting 13:59:51 (3508): No heartbeat from core client for 30 sec - exiting 13:59:52 (3508): No heartbeat from core client for 30 sec - exiting 13:59:53 (3508): No heartbeat from core client for 30 sec - exiting 13:59:54 (3508): No heartbeat from core client for 30 sec - exiting 13:59:55 (3508): No heartbeat from core client for 30 sec - exiting 13:59:56 (3508): No heartbeat from core client for 30 sec - exiting 13:59:57 (3508): No heartbeat from core client for 30 sec - exiting 13:59:58 (3508): No heartbeat from core client for 30 sec - exiting 13:59:59 (3508): No heartbeat from core client for 30 sec - exiting 14:00:00 (3508): No heartbeat from core client for 30 sec - exiting 14:00:01 (3508): No heartbeat from core client for 30 sec - exiting 14:00:02 (3508): No heartbeat from core client for 30 sec - exiting 14:00:03 (3508): No heartbeat from core client for 30 sec - exiting 14:00:04 (3508): No heartbeat from core client for 30 sec - exiting 14:00:05 (3508): No heartbeat from core client for 30 sec - exiting 14:00:06 (3508): No heartbeat from core client for 30 sec - exiting 14:00:07 (3508): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:07:30 (5088): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 19:23:36 (5000): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 11:54:03 (5024): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 18:51:30 (3156): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 16:47:23 (8900): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7536, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 17:56:45 (6008): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 18:45:18 (6312): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5460, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... 22:46:52 (3716): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:04:02 (4952): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 21:27:34 (2084): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 16:04:55 (6112): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 17:09:06 (4412): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:13:21 (4164): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 21:54:50 (5820): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x778D3AC3 read attempt to address 0x42C7C110 Engaging BOINC Windows Runtime Debugger... Cannot serialize file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_o3aw_1940_40_008381271/dataout/shmem_restart.day Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
01 Jul 2014 08:26:16	1185926	16069827	hadcm3n_o3aw_1940_40_008381271_2	777,600	1,072,999	1.3799
15 Jun 2014 20:12:14	1185926	16069827	hadcm3n_o3aw_1940_40_008381271_2	751,680	988,763	1.3154
25 May 2014 13:32:50	1185926	16069827	hadcm3n_o3aw_1940_40_008381271_2	725,760	906,379	1.2489
22 May 2014 22:23:23	1185926	16069827	hadcm3n_o3aw_1940_40_008381271_2	699,840	823,972	1.1774
18 May 2014 15:31:43	1185926	16069827	hadcm3n_o3aw_1940_40_008381271_2	673,920	769,889	1.1424
27 Apr 2014 18:30:51	1185926	16069827	hadcm3n_o3aw_1940_40_008381271_2	648,000	741,614	1.1445
26 Apr 2014 18:50:39	1185926	16069827	hadcm3n_o3aw_1940_40_008381271_2	622,080	716,833	1.1523
30 Mar 2014 19:28:10	1185926	16069827	hadcm3n_o3aw_1940_40_008381271_2	596,160	687,141	1.1526
24 Mar 2014 20:54:31	1185926	16069827	hadcm3n_o3aw_1940_40_008381271_2	570,240	646,135	1.1331
18 Mar 2014 21:30:05	1185926	16069827	hadcm3n_o3aw_1940_40_008381271_2	544,320	620,729	1.1404
15 Mar 2014 11:46:17	1185926	16069827	hadcm3n_o3aw_1940_40_008381271_2	518,400	596,545	1.1507
11 Mar 2014 20:24:09	1185926	16069827	hadcm3n_o3aw_1940_40_008381271_2	492,480	571,523	1.1605
01 Mar 2014 17:40:22	1185926	16069827	hadcm3n_o3aw_1940_40_008381271_2	466,560	545,591	1.1694
23 Feb 2014 18:26:37	1185926	16069827	hadcm3n_o3aw_1940_40_008381271_2	440,640	518,710	1.1772
15 Feb 2014 21:21:54	1185926	16069827	hadcm3n_o3aw_1940_40_008381271_2	414,720	481,463	1.1609
12 Feb 2014 14:17:39	1185926	16069827	hadcm3n_o3aw_1940_40_008381271_2	388,800	447,260	1.1504
10 Feb 2014 20:08:37	1185926	16069827	hadcm3n_o3aw_1940_40_008381271_2	362,880	423,246	1.1664
09 Feb 2014 14:52:11	1185926	16069827	hadcm3n_o3aw_1940_40_008381271_2	336,960	398,306	1.1821
06 Feb 2014 21:48:18	1185926	16069827	hadcm3n_o3aw_1940_40_008381271_2	311,040	372,311	1.1970
05 Feb 2014 17:16:41	1185926	16069827	hadcm3n_o3aw_1940_40_008381271_2	285,120	346,207	1.2143