Task 15769129

Name	hadcm3n_4h1q_1940_40_008305331_3
Workunit	8456466
Created	9 May 2013, 16:31:56 UTC
Sent	9 May 2013, 16:32:13 UTC
Report deadline	8 Aug 2013, 23:59:24 UTC
Received	24 Jun 2013, 1:10:22 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	-1073741819 (0xC0000005) STATUS_ACCESS_VIOLATION
Computer ID	1093006
Run time	41 days 8 hours 39 min 42 sec
CPU time	39 days 1 hours 42 min 26 sec
Validate state	Invalid
Credit	12,441.60
Device peak FLOPS	1.57 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.64</core_client_version> <![CDATA[ <message> (unknown error) - exit code -1073741819 (0xc0000005) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8928, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... 12:33:44 (4472): No heartbeat from core client for 30 sec - exiting 12:33:45 (4472): No heartbeat from core client for 30 sec - exiting 12:33:46 (4472): No heartbeat from core client for 30 sec - exiting 12:33:47 (4472): No heartbeat from core client for 30 sec - exiting 12:33:48 (4472): No heartbeat from core client for 30 sec - exiting 12:33:49 (4472): No heartbeat from core client for 30 sec - exiting 12:33:50 (4472): No heartbeat from core client for 30 sec - exiting 12:33:51 (4472): No heartbeat from core client for 30 sec - exiting 12:33:52 (4472): No heartbeat from core client for 30 sec - exiting 12:33:53 (4472): No heartbeat from core client for 30 sec - exiting 12:33:54 (4472): No heartbeat from core client for 30 sec - exiting 12:33:56 (4472): No heartbeat from core client for 30 sec - exiting 12:33:57 (4472): No heartbeat from core client for 30 sec - exiting 12:33:58 (4472): No heartbeat from core client for 30 sec - exiting 12:33:59 (4472): No heartbeat from core client for 30 sec - exiting 12:34:00 (4472): No heartbeat from core client for 30 sec - exiting 12:34:01 (4472): No heartbeat from core client for 30 sec - exiting 12:34:02 (4472): No heartbeat from core client for 30 sec - exiting 12:34:03 (4472): No heartbeat from core client for 30 sec - exiting 12:34:04 (4472): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:34:05 (4472): No heartbeat from core client for 30 sec - exiting 12:34:06 (4472): No heartbeat from core client for 30 sec - exiting 12:34:08 (4472): No heartbeat from core client for 30 sec - exiting 12:34:09 (4472): No heartbeat from core client for 30 sec - exiting 12:34:10 (4472): No heartbeat from core client for 30 sec - exiting 12:34:11 (4472): No heartbeat from core client for 30 sec - exiting 12:34:12 (4472): No heartbeat from core client for 30 sec - exiting 12:34:13 (4472): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6808, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5384, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5384, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6332, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3352, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 03:45:04 (3492): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x771A579C read attempt to address 0x40E7F9D7 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x771A579C read attempt to address 0x40E7F9D7 Engaging BOINC Windows Runtime Debugger... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
23 Jun 2013 14:39:16	1093006	15769129	hadcm3n_4h1q_1940_40_008305331_3	1,036,800	3,374,898	3.2551
22 Jun 2013 18:01:46	1093006	15769129	hadcm3n_4h1q_1940_40_008305331_3	1,010,880	3,310,444	3.2748
21 Jun 2013 22:43:51	1093006	15769129	hadcm3n_4h1q_1940_40_008305331_3	984,960	3,242,500	3.2920
20 Jun 2013 17:50:00	1093006	15769129	hadcm3n_4h1q_1940_40_008305331_3	959,040	3,148,984	3.2835
18 Jun 2013 22:56:26	1093006	15769129	hadcm3n_4h1q_1940_40_008305331_3	933,120	3,061,382	3.2808
18 Jun 2013 11:11:37	1093006	15769129	hadcm3n_4h1q_1940_40_008305331_3	907,200	3,019,275	3.3281
17 Jun 2013 22:23:20	1093006	15769129	hadcm3n_4h1q_1940_40_008305331_3	881,280	2,973,769	3.3744
17 Jun 2013 08:50:58	1093006	15769129	hadcm3n_4h1q_1940_40_008305331_3	855,360	2,931,858	3.4276
16 Jun 2013 20:48:37	1093006	15769129	hadcm3n_4h1q_1940_40_008305331_3	829,440	2,889,911	3.4842
16 Jun 2013 07:50:23	1093006	15769129	hadcm3n_4h1q_1940_40_008305331_3	803,520	2,844,185	3.5397
15 Jun 2013 16:33:27	1093006	15769129	hadcm3n_4h1q_1940_40_008305331_3	777,600	2,798,236	3.5986
14 Jun 2013 23:40:34	1093006	15769129	hadcm3n_4h1q_1940_40_008305331_3	751,680	2,738,431	3.6431
13 Jun 2013 13:25:40	1093006	15769129	hadcm3n_4h1q_1940_40_008305331_3	725,760	2,656,779	3.6607
12 Jun 2013 08:05:43	1093006	15769129	hadcm3n_4h1q_1940_40_008305331_3	699,840	2,562,206	3.6611
11 Jun 2013 06:17:14	1093006	15769129	hadcm3n_4h1q_1940_40_008305331_3	673,920	2,483,942	3.6858
09 Jun 2013 01:26:49	1093006	15769129	hadcm3n_4h1q_1940_40_008305331_3	648,000	2,389,445	3.6874
07 Jun 2013 18:43:59	1093006	15769129	hadcm3n_4h1q_1940_40_008305331_3	622,080	2,286,987	3.6764
06 Jun 2013 16:39:51	1093006	15769129	hadcm3n_4h1q_1940_40_008305331_3	596,160	2,196,745	3.6848
05 Jun 2013 14:27:50	1093006	15769129	hadcm3n_4h1q_1940_40_008305331_3	570,240	2,104,445	3.6905
04 Jun 2013 11:51:15	1093006	15769129	hadcm3n_4h1q_1940_40_008305331_3	544,320	2,010,795	3.6941