Task 15765107

Name	hadcm3n_zbmd_1920_40_008340088_1
Workunit	8490949
Created	7 May 2013, 17:18:15 UTC
Sent	7 May 2013, 17:19:14 UTC
Report deadline	7 Aug 2013, 0:46:25 UTC
Received	2 Jun 2013, 11:59:03 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	-1073741819 (0xC0000005) STATUS_ACCESS_VIOLATION
Computer ID	1242385
Run time	22 days 21 hours 43 min 23 sec
CPU time	22 days 9 hours 38 min 36 sec
Validate state	Invalid
Credit	12,441.60
Device peak FLOPS	2.52 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.64</core_client_version> <![CDATA[ <message> (unknown error) - exit code -1073741819 (0xc0000005) </message> <stderr_txt> 13:05:06 (2856): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 08:00:40 (1180): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 09:02:08 (3568): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 13:02:00 (6972): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:02:01 (6972): No heartbeat from core client for 30 sec - exiting 13:02:02 (6972): No heartbeat from core client for 30 sec - exiting 13:02:03 (6972): No heartbeat from core client for 30 sec - exiting 13:02:04 (6972): No heartbeat from core client for 30 sec - exiting 13:02:05 (6972): No heartbeat from core client for 30 sec - exiting 13:02:06 (6972): No heartbeat from core client for 30 sec - exiting 13:02:07 (6972): No heartbeat from core client for 30 sec - exiting 13:02:08 (6972): No heartbeat from core client for 30 sec - exiting 13:02:09 (6972): No heartbeat from core client for 30 sec - exiting 13:02:10 (6972): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... 14:04:13 (7696): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 08:02:58 (6196): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 14:02:42 (7024): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:02:43 (7024): No heartbeat from core client for 30 sec - exiting 14:02:44 (7024): No heartbeat from core client for 30 sec - exiting 14:02:45 (7024): No heartbeat from core client for 30 sec - exiting 14:02:46 (7024): No heartbeat from core client for 30 sec - exiting 14:02:47 (7024): No heartbeat from core client for 30 sec - exiting 14:02:48 (7024): No heartbeat from core client for 30 sec - exiting 14:02:49 (7024): No heartbeat from core client for 30 sec - exiting 14:02:50 (7024): No heartbeat from core client for 30 sec - exiting 14:02:51 (7024): No heartbeat from core client for 30 sec - exiting 14:02:52 (7024): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 08:00:42 (3572): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 09:01:01 (7960): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:01:02 (7960): No heartbeat from core client for 30 sec - exiting 09:01:03 (7960): No heartbeat from core client for 30 sec - exiting 09:01:04 (7960): No heartbeat from core client for 30 sec - exiting 09:01:05 (7960): No heartbeat from core client for 30 sec - exiting 09:01:06 (7960): No heartbeat from core client for 30 sec - exiting 09:01:07 (7960): No heartbeat from core client for 30 sec - exiting 09:01:08 (7960): No heartbeat from core client for 30 sec - exiting 09:01:09 (7960): No heartbeat from core client for 30 sec - exiting 09:01:10 (7960): No heartbeat from core client for 30 sec - exiting 09:01:11 (7960): No heartbeat from core client for 30 sec - exiting 09:02:46 (2384): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 09:22:19 (5836): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:47:40 (5660): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:03:14 (6512): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 14:01:06 (7728): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5428, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 08:40:56 (6024): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5896, iMonCtr=1 Model crash detected, will try to restart... 07:27:01 (6632): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:00:21 (6480): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 11:14:04 (3920): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6960, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... 07:57:13 (1164): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Atmos Hold Restart file rename failed on atmos_restart.hold Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 14:08:30 (6356): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6028, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6028, iMonCtr=1 Model crash detected, will try to restart... 09:00:47 (7080): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 13:07:13 (3460): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 14:02:26 (9196): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:02:27 (9196): No heartbeat from core client for 30 sec - exiting 14:02:28 (9196): No heartbeat from core client for 30 sec - exiting 14:02:29 (9196): No heartbeat from core client for 30 sec - exiting 14:02:30 (9196): No heartbeat from core client for 30 sec - exiting 14:02:31 (9196): No heartbeat from core client for 30 sec - exiting 14:02:32 (9196): No heartbeat from core client for 30 sec - exiting 14:02:33 (9196): No heartbeat from core client for 30 sec - exiting 14:02:34 (9196): No heartbeat from core client for 30 sec - exiting 14:02:35 (9196): No heartbeat from core client for 30 sec - exiting 14:02:36 (9196): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5432, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5432, iMonCtr=1 Model crash detected, will try to restart... 08:00:44 (6680): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 08:52:01 (1232): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:52:02 (1232): No heartbeat from core client for 30 sec - exiting 08:52:03 (1232): No heartbeat from core client for 30 sec - exiting 08:52:04 (1232): No heartbeat from core client for 30 sec - exiting 08:52:05 (1232): No heartbeat from core client for 30 sec - exiting 08:52:06 (1232): No heartbeat from core client for 30 sec - exiting 08:52:07 (1232): No heartbeat from core client for 30 sec - exiting 08:52:08 (1232): No heartbeat from core client for 30 sec - exiting 08:52:09 (1232): No heartbeat from core client for 30 sec - exiting 08:52:10 (1232): No heartbeat from core client for 30 sec - exiting 08:52:11 (1232): No heartbeat from core client for 30 sec - exiting Atmos Hold Restart file rename failed on atmos_restart.hold 09:02:01 (6484): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:02:02 (6484): No heartbeat from core client for 30 sec - exiting 09:02:03 (6484): No heartbeat from core client for 30 sec - exiting 09:02:04 (6484): No heartbeat from core client for 30 sec - exiting 09:02:05 (6484): No heartbeat from core client for 30 sec - exiting 09:02:06 (6484): No heartbeat from core client for 30 sec - exiting 09:02:07 (6484): No heartbeat from core client for 30 sec - exiting 09:02:08 (6484): No heartbeat from core client for 30 sec - exiting 09:02:09 (6484): No heartbeat from core client for 30 sec - exiting 09:02:10 (6484): No heartbeat from core client for 30 sec - exiting 09:02:11 (6484): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... 10:30:09 (3020): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:30:10 (3020): No heartbeat from core client for 30 sec - exiting 10:30:11 (3020): No heartbeat from core client for 30 sec - exiting 10:30:12 (3020): No heartbeat from core client for 30 sec - exiting 10:30:13 (3020): No heartbeat from core client for 30 sec - exiting 10:30:14 (3020): No heartbeat from core client for 30 sec - exiting 10:30:15 (3020): No heartbeat from core client for 30 sec - exiting 10:30:16 (3020): No heartbeat from core client for 30 sec - exiting 10:30:17 (3020): No heartbeat from core client for 30 sec - exiting 10:30:18 (3020): No heartbeat from core client for 30 sec - exiting 10:30:19 (3020): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... 13:00:47 (1872): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1196, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6444, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6444, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... 08:07:15 (8128): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 09:13:11 (2108): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x776E3AB3 read attempt to address 0x00000000 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x776E3AB3 read attempt to address 0x00000000 Engaging BOINC Windows Runtime Debugger... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
01 Jun 2013 03:29:50	1242385	15765107	hadcm3n_zbmd_1920_40_008340088_1	1,036,800	1,934,827	1.8662
31 May 2013 10:16:50	1242385	15765107	hadcm3n_zbmd_1920_40_008340088_1	1,010,880	1,880,010	1.8598
30 May 2013 20:29:19	1242385	15765107	hadcm3n_zbmd_1920_40_008340088_1	984,960	1,831,110	1.8591
30 May 2013 04:52:43	1242385	15765107	hadcm3n_zbmd_1920_40_008340088_1	959,040	1,780,916	1.8570
29 May 2013 11:03:47	1242385	15765107	hadcm3n_zbmd_1920_40_008340088_1	933,120	1,727,277	1.8511
28 May 2013 22:44:06	1242385	15765107	hadcm3n_zbmd_1920_40_008340088_1	907,200	1,681,311	1.8533
28 May 2013 05:52:39	1242385	15765107	hadcm3n_zbmd_1920_40_008340088_1	881,280	1,630,581	1.8502
27 May 2013 16:42:04	1242385	15765107	hadcm3n_zbmd_1920_40_008340088_1	855,360	1,583,784	1.8516
27 May 2013 03:27:03	1242385	15765107	hadcm3n_zbmd_1920_40_008340088_1	829,440	1,537,083	1.8532
26 May 2013 14:26:49	1242385	15765107	hadcm3n_zbmd_1920_40_008340088_1	803,520	1,491,231	1.8559
26 May 2013 01:43:06	1242385	15765107	hadcm3n_zbmd_1920_40_008340088_1	777,600	1,446,198	1.8598
25 May 2013 13:03:32	1242385	15765107	hadcm3n_zbmd_1920_40_008340088_1	751,680	1,401,181	1.8641
25 May 2013 00:29:12	1242385	15765107	hadcm3n_zbmd_1920_40_008340088_1	725,760	1,356,442	1.8690
24 May 2013 06:07:00	1242385	15765107	hadcm3n_zbmd_1920_40_008340088_1	699,840	1,308,895	1.8703
23 May 2013 13:14:37	1242385	15765107	hadcm3n_zbmd_1920_40_008340088_1	673,920	1,259,169	1.8684
22 May 2013 23:19:43	1242385	15765107	hadcm3n_zbmd_1920_40_008340088_1	648,000	1,213,612	1.8729
22 May 2013 09:53:45	1242385	15765107	hadcm3n_zbmd_1920_40_008340088_1	622,080	1,165,952	1.8743
21 May 2013 20:32:30	1242385	15765107	hadcm3n_zbmd_1920_40_008340088_1	596,160	1,118,775	1.8766
21 May 2013 07:02:22	1242385	15765107	hadcm3n_zbmd_1920_40_008340088_1	570,240	1,070,887	1.8780
20 May 2013 17:21:23	1242385	15765107	hadcm3n_zbmd_1920_40_008340088_1	544,320	1,022,708	1.8789