Task 15773286

Name	hadcm3n_3h9u_1980_40_008363984_0
Workunit	8514843
Created	10 May 2013, 18:50:11 UTC
Sent	11 May 2013, 11:06:42 UTC
Report deadline	10 Aug 2013, 18:33:53 UTC
Received	17 Jun 2013, 15:22:05 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	-1073741819 (0xC0000005) STATUS_ACCESS_VIOLATION
Computer ID	1245161
Run time	9 days 1 hours 36 min 31 sec
CPU time	8 days 23 hours 33 min 57 sec
Validate state	Invalid
Credit	9,331.20
Device peak FLOPS	3.53 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.64</core_client_version> <![CDATA[ <message> (unknown error) - exit code -1073741819 (0xc0000005) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5176, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3976, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3992, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4716, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2468, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... 10:02:19 (6328): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 19:39:48 (3140): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 17:02:21 (2788): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 10:00:31 (5348): No heartbeat from core client for 30 sec - exiting 10:00:32 (5348): No heartbeat from core client for 30 sec - exiting 10:00:33 (5348): No heartbeat from core client for 30 sec - exiting 10:00:34 (5348): No heartbeat from core client for 30 sec - exiting 10:00:35 (5348): No heartbeat from core client for 30 sec - exiting 10:00:36 (5348): No heartbeat from core client for 30 sec - exiting 10:00:37 (5348): No heartbeat from core client for 30 sec - exiting 10:00:38 (5348): No heartbeat from core client for 30 sec - exiting 10:00:39 (5348): No heartbeat from core client for 30 sec - exiting 10:00:40 (5348): No heartbeat from core client for 30 sec - exiting 10:00:41 (5348): No heartbeat from core client for 30 sec - exiting 10:00:42 (5348): No heartbeat from core client for 30 sec - exiting 10:00:43 (5348): No heartbeat from core client for 30 sec - exiting 10:00:44 (5348): No heartbeat from core client for 30 sec - exiting 10:00:45 (5348): No heartbeat from core client for 30 sec - exiting 10:00:46 (5348): No heartbeat from core client for 30 sec - exiting 10:00:47 (5348): No heartbeat from core client for 30 sec - exiting 10:00:48 (5348): No heartbeat from core client for 30 sec - exiting 10:00:49 (5348): No heartbeat from core client for 30 sec - exiting 10:00:50 (5348): No heartbeat from core client for 30 sec - exiting 10:00:51 (5348): No heartbeat from core client for 30 sec - exiting 10:00:52 (5348): No heartbeat from core client for 30 sec - exiting 10:00:53 (5348): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:00:54 (5348): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... 18:26:23 (4444): No heartbeat from core client for 30 sec - exiting 18:26:24 (4444): No heartbeat from core client for 30 sec - exiting 18:26:25 (4444): No heartbeat from core client for 30 sec - exiting 18:26:26 (4444): No heartbeat from core client for 30 sec - exiting 18:26:27 (4444): No heartbeat from core client for 30 sec - exiting 18:26:28 (4444): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 09:49:29 (3216): No heartbeat from core client for 30 sec - exiting 09:49:30 (3216): No heartbeat from core client for 30 sec - exiting 09:49:31 (3216): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 16:39:27 (4860): No heartbeat from core client for 30 sec - exiting 16:39:28 (4860): No heartbeat from core client for 30 sec - exiting 16:39:29 (4860): No heartbeat from core client for 30 sec - exiting 16:39:30 (4860): No heartbeat from core client for 30 sec - exiting 16:39:31 (4860): No heartbeat from core client for 30 sec - exiting 16:39:32 (4860): No heartbeat from core client for 30 sec - exiting 16:39:33 (4860): No heartbeat from core client for 30 sec - exiting 16:39:34 (4860): No heartbeat from core client for 30 sec - exiting 16:39:35 (4860): No heartbeat from core client for 30 sec - exiting 16:39:36 (4860): No heartbeat from core client for 30 sec - exiting 16:39:37 (4860): No heartbeat from core client for 30 sec - exiting 16:39:38 (4860): No heartbeat from core client for 30 sec - exiting 16:39:39 (4860): No heartbeat from core client for 30 sec - exiting 16:39:40 (4860): No heartbeat from core client for 30 sec - exiting 16:39:41 (4860): No heartbeat from core client for 30 sec - exiting 16:39:42 (4860): No heartbeat from core client for 30 sec - exiting 16:39:43 (4860): No heartbeat from core client for 30 sec - exiting 16:39:44 (4860): No heartbeat from core client for 30 sec - exiting 16:39:45 (4860): No heartbeat from core client for 30 sec - exiting 16:39:46 (4860): No heartbeat from core client for 30 sec - exiting 16:39:47 (4860): No heartbeat from core client for 30 sec - exiting 16:39:48 (4860): No heartbeat from core client for 30 sec - exiting 16:39:49 (4860): No heartbeat from core client for 30 sec - exiting 16:39:50 (4860): No heartbeat from core client for 30 sec - exiting 16:39:51 (4860): No heartbeat from core client for 30 sec - exiting 16:39:52 (4860): No heartbeat from core client for 30 sec - exiting 16:39:53 (4860): No heartbeat from core client for 30 sec - exiting 16:39:54 (4860): No heartbeat from core client for 30 sec - exiting 16:39:55 (4860): No heartbeat from core client for 30 sec - exiting 16:39:56 (4860): No heartbeat from core client for 30 sec - exiting 16:39:57 (4860): No heartbeat from core client for 30 sec - exiting 16:39:58 (4860): No heartbeat from core client for 30 sec - exiting 16:39:59 (4860): No heartbeat from core client for 30 sec - exiting 16:40:00 (4860): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 16:58:15 (856): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:40:05 (4912): No heartbeat from core client for 30 sec - exiting 09:40:06 (4912): No heartbeat from core client for 30 sec - exiting 09:40:07 (4912): No heartbeat from core client for 30 sec - exiting 09:40:08 (4912): No heartbeat from core client for 30 sec - exiting 09:40:09 (4912): No heartbeat from core client for 30 sec - exiting 09:40:10 (4912): No heartbeat from core client for 30 sec - exiting 09:40:11 (4912): No heartbeat from core client for 30 sec - exiting 09:40:12 (4912): No heartbeat from core client for 30 sec - exiting 09:40:13 (4912): No heartbeat from core client for 30 sec - exiting 09:40:14 (4912): No heartbeat from core client for 30 sec - exiting 09:40:15 (4912): No heartbeat from core client for 30 sec - exiting 09:40:16 (4912): No heartbeat from core client for 30 sec - exiting 09:40:17 (4912): No heartbeat from core client for 30 sec - exiting 09:40:18 (4912): No heartbeat from core client for 30 sec - exiting 09:40:19 (4912): No heartbeat from core client for 30 sec - exiting 09:40:20 (4912): No heartbeat from core client for 30 sec - exiting 09:40:21 (4912): No heartbeat from core client for 30 sec - exiting 09:40:22 (4912): No heartbeat from core client for 30 sec - exiting 09:40:23 (4912): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 09:20:26 (1508): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 17:01:19 (6440): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 10:03:29 (5808): No heartbeat from core client for 30 sec - exiting 10:03:30 (5808): No heartbeat from core client for 30 sec - exiting 10:03:31 (5808): No heartbeat from core client for 30 sec - exiting 10:03:32 (5808): No heartbeat from core client for 30 sec - exiting 10:03:33 (5808): No heartbeat from core client for 30 sec - exiting 10:03:34 (5808): No heartbeat from core client for 30 sec - exiting 10:03:35 (5808): No heartbeat from core client for 30 sec - exiting 10:03:36 (5808): No heartbeat from core client for 30 sec - exiting 10:03:37 (5808): No heartbeat from core client for 30 sec - exiting 10:03:38 (5808): No heartbeat from core client for 30 sec - exiting 10:03:39 (5808): No heartbeat from core client for 30 sec - exiting 10:03:40 (5808): No heartbeat from core client for 30 sec - exiting 10:03:41 (5808): No heartbeat from core client for 30 sec - exiting 10:03:42 (5808): No heartbeat from core client for 30 sec - exiting 10:03:43 (5808): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 19:27:05 (5912): No heartbeat from core client for 30 sec - exiting 19:27:06 (5912): No heartbeat from core client for 30 sec - exiting 19:27:07 (5912): No heartbeat from core client for 30 sec - exiting 19:27:08 (5912): No heartbeat from core client for 30 sec - exiting 19:27:09 (5912): No heartbeat from core client for 30 sec - exiting 19:27:10 (5912): No heartbeat from core client for 30 sec - exiting 19:27:11 (5912): No heartbeat from core client for 30 sec - exiting 19:27:12 (5912): No heartbeat from core client for 30 sec - exiting 19:27:13 (5912): No heartbeat from core client for 30 sec - exiting 19:27:14 (5912): No heartbeat from core client for 30 sec - exiting 19:27:15 (5912): No heartbeat from core client for 30 sec - exiting 19:27:16 (5912): No heartbeat from core client for 30 sec - exiting 19:27:17 (5912): No heartbeat from core client for 30 sec - exiting 19:27:18 (5912): No heartbeat from core client for 30 sec - exiting 19:27:19 (5912): No heartbeat from core client for 30 sec - exiting 19:27:20 (5912): No heartbeat from core client for 30 sec - exiting 19:27:21 (5912): No heartbeat from core client for 30 sec - exiting 19:27:22 (5912): No heartbeat from core client for 30 sec - exiting 19:27:23 (5912): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 09:26:55 (4872): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 17:02:07 (2980): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1012, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6112, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... 09:12:06 (6064): No heartbeat from core client for 30 sec - exiting 09:12:07 (6064): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 10:11:07 (3404): No heartbeat from core client for 30 sec - exiting 10:11:08 (3404): No heartbeat from core client for 30 sec - exiting 10:11:09 (3404): No heartbeat from core client for 30 sec - exiting 10:11:10 (3404): No heartbeat from core client for 30 sec - exiting 10:11:11 (3404): No heartbeat from core client for 30 sec - exiting 10:11:12 (3404): No heartbeat from core client for 30 sec - exiting 10:11:13 (3404): No heartbeat from core client for 30 sec - exiting 10:11:14 (3404): No heartbeat from core client for 30 sec - exiting 10:11:15 (3404): No heartbeat from core client for 30 sec - exiting 10:11:16 (3404): No heartbeat from core client for 30 sec - exiting 10:11:17 (3404): No heartbeat from core client for 30 sec - exiting 10:11:18 (3404): No heartbeat from core client for 30 sec - exiting 10:11:19 (3404): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77687373 read attempt to address 0xFFFFFFF8 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77C47373 read attempt to address 0xFFFFFFF8 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77907373 read attempt to address 0xFFFFFFF8 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77003AB3 read attempt to address 0x00000000 Engaging BOINC Windows Runtime Debugger... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
15 Jun 2013 08:29:33	1245161	15773286	hadcm3n_3h9u_1980_40_008363984_0	777,600	761,574	0.9794
14 Jun 2013 08:42:31	1245161	15773286	hadcm3n_3h9u_1980_40_008363984_0	751,680	736,510	0.9798
12 Jun 2013 09:01:10	1245161	15773286	hadcm3n_3h9u_1980_40_008363984_0	725,760	711,705	0.9806
11 Jun 2013 10:03:48	1245161	15773286	hadcm3n_3h9u_1980_40_008363984_0	699,840	686,604	0.9811
10 Jun 2013 11:18:50	1245161	15773286	hadcm3n_3h9u_1980_40_008363984_0	673,920	661,544	0.9816
08 Jun 2013 17:17:37	1245161	15773286	hadcm3n_3h9u_1980_40_008363984_0	648,000	636,431	0.9821
07 Jun 2013 17:22:42	1245161	15773286	hadcm3n_3h9u_1980_40_008363984_0	622,080	611,618	0.9832
07 Jun 2013 07:34:59	1245161	15773286	hadcm3n_3h9u_1980_40_008363984_0	596,160	587,108	0.9848
06 Jun 2013 09:26:36	1245161	15773286	hadcm3n_3h9u_1980_40_008363984_0	570,240	562,640	0.9867
05 Jun 2013 10:00:55	1245161	15773286	hadcm3n_3h9u_1980_40_008363984_0	544,320	537,642	0.9877
04 Jun 2013 15:02:40	1245161	15773286	hadcm3n_3h9u_1980_40_008363984_0	518,400	512,803	0.9892
03 Jun 2013 11:25:44	1245161	15773286	hadcm3n_3h9u_1980_40_008363984_0	492,480	488,289	0.9915
01 Jun 2013 08:39:07	1245161	15773286	hadcm3n_3h9u_1980_40_008363984_0	466,560	462,192	0.9906
31 May 2013 08:31:09	1245161	15773286	hadcm3n_3h9u_1980_40_008363984_0	440,640	435,609	0.9886
30 May 2013 08:44:09	1245161	15773286	hadcm3n_3h9u_1980_40_008363984_0	414,720	411,069	0.9912
29 May 2013 09:53:18	1245161	15773286	hadcm3n_3h9u_1980_40_008363984_0	388,800	385,865	0.9925
28 May 2013 08:08:31	1245161	15773286	hadcm3n_3h9u_1980_40_008363984_0	362,880	359,862	0.9917
28 May 2013 01:05:50	1245161	15773286	hadcm3n_3h9u_1980_40_008363984_0	336,960	334,973	0.9941
27 May 2013 19:06:13	1245161	15773286	hadcm3n_3h9u_1980_40_008363984_0	311,040	310,446	0.9981
25 May 2013 17:46:54	1245161	15773286	hadcm3n_3h9u_1980_40_008363984_0	285,120	284,319	0.9972