Task 15635544

Name	hadcm3n_3jm0_1980_40_008320697_0
Workunit	8471832
Created	24 Feb 2013, 15:24:16 UTC
Sent	24 Feb 2013, 15:32:01 UTC
Report deadline	26 May 2013, 22:59:12 UTC
Received	20 Apr 2013, 7:57:57 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	-1073741819 (0xC0000005) STATUS_ACCESS_VIOLATION
Computer ID	1164908
Run time	10 days 18 hours 8 min 12 sec
CPU time	10 days 10 hours 29 min 19 sec
Validate state	Invalid
Credit	6,220.80
Device peak FLOPS	2.59 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.28</core_client_version> <![CDATA[ <message> - exit code -1073741819 (0xc0000005) </message> <stderr_txt> 10:30:25 (4548): No heartbeat from core client for 30 sec - exiting 10:30:26 (4548): No heartbeat from core client for 30 sec - exiting 10:30:27 (4548): No heartbeat from core client for 30 sec - exiting 10:30:28 (4548): No heartbeat from core client for 30 sec - exiting 10:30:29 (4548): No heartbeat from core client for 30 sec - exiting 10:30:30 (4548): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:45:14 (6296): No heartbeat from core client for 30 sec - exiting 10:45:15 (6296): No heartbeat from core client for 30 sec - exiting 10:45:16 (6296): No heartbeat from core client for 30 sec - exiting 10:45:17 (6296): No heartbeat from core client for 30 sec - exiting 10:45:18 (6296): No heartbeat from core client for 30 sec - exiting 10:45:19 (6296): No heartbeat from core client for 30 sec - exiting 10:45:20 (6296): No heartbeat from core client for 30 sec - exiting 10:45:21 (6296): No heartbeat from core client for 30 sec - exiting 10:45:22 (6296): No heartbeat from core client for 30 sec - exiting 10:45:23 (6296): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:28:24 (4716): No heartbeat from core client for 30 sec - exiting 14:28:26 (4716): No heartbeat from core client for 30 sec - exiting 14:28:27 (4716): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:20:59 (5780): No heartbeat from core client for 30 sec - exiting 11:21:00 (5780): No heartbeat from core client for 30 sec - exiting 11:21:01 (5780): No heartbeat from core client for 30 sec - exiting 11:21:02 (5780): No heartbeat from core client for 30 sec - exiting 11:21:03 (5780): No heartbeat from core client for 30 sec - exiting 11:21:04 (5780): No heartbeat from core client for 30 sec - exiting 11:21:05 (5780): No heartbeat from core client for 30 sec - exiting 11:21:06 (5780): No heartbeat from core client for 30 sec - exiting 11:21:07 (5780): No heartbeat from core client for 30 sec - exiting 11:21:08 (5780): No heartbeat from core client for 30 sec - exiting 11:21:09 (5780): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:23:34 (4332): No heartbeat from core client for 30 sec - exiting 11:23:35 (4332): No heartbeat from core client for 30 sec - exiting 11:23:36 (4332): No heartbeat from core client for 30 sec - exiting 11:23:37 (4332): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:07:46 (3308): No heartbeat from core client for 30 sec - exiting 12:07:47 (3308): No heartbeat from core client for 30 sec - exiting 12:07:48 (3308): No heartbeat from core client for 30 sec - exiting 12:07:49 (3308): No heartbeat from core client for 30 sec - exiting 12:07:50 (3308): No heartbeat from core client for 30 sec - exiting 12:07:51 (3308): No heartbeat from core client for 30 sec - exiting 12:07:52 (3308): No heartbeat from core client for 30 sec - exiting 12:07:53 (3308): No heartbeat from core client for 30 sec - exiting 12:07:54 (3308): No heartbeat from core client for 30 sec - exiting 12:07:55 (3308): No heartbeat from core client for 30 sec - exiting 12:07:56 (3308): No heartbeat from core client for 30 sec - exiting 12:07:57 (3308): No heartbeat from core client for 30 sec - exiting 12:07:58 (3308): No heartbeat from core client for 30 sec - exiting 12:07:59 (3308): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:13:48 (1632): No heartbeat from core client for 30 sec - exiting 11:13:50 (1632): No heartbeat from core client for 30 sec - exiting 11:13:51 (1632): No heartbeat from core client for 30 sec - exiting 11:13:52 (1632): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:15:35 (6072): No heartbeat from core client for 30 sec - exiting 11:15:36 (6072): No heartbeat from core client for 30 sec - exiting 11:15:37 (6072): No heartbeat from core client for 30 sec - exiting 11:15:38 (6072): No heartbeat from core client for 30 sec - exiting 11:15:39 (6072): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:41:12 (7916): No heartbeat from core client for 30 sec - exiting 14:41:13 (7916): No heartbeat from core client for 30 sec - exiting 14:41:14 (7916): No heartbeat from core client for 30 sec - exiting 14:41:15 (7916): No heartbeat from core client for 30 sec - exiting 14:41:16 (7916): No heartbeat from core client for 30 sec - exiting 14:41:17 (7916): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7936, iMonCtr=1 Model crash detected, will try to restart... 15:37:51 (1836): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:25:39 (5692): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:25:40 (5692): No heartbeat from core client for 30 sec - exiting 10:20:21 (5520): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:20:23 (5520): No heartbeat from core client for 30 sec - exiting 12:53:47 (6460): No heartbeat from core client for 30 sec - exiting 12:53:48 (6460): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:15:09 (1204): No heartbeat from core client for 30 sec - exiting 14:15:10 (1204): No heartbeat from core client for 30 sec - exiting 14:15:11 (1204): No heartbeat from core client for 30 sec - exiting 14:15:12 (1204): No heartbeat from core client for 30 sec - exiting 14:15:13 (1204): No heartbeat from core client for 30 sec - exiting 14:15:14 (1204): No heartbeat from core client for 30 sec - exiting 14:15:15 (1204): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:13:29 (5948): No heartbeat from core client for 30 sec - exiting 12:13:30 (5948): No heartbeat from core client for 30 sec - exiting 12:13:31 (5948): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8548, iMonCtr=1 Model crash detected, will try to restart... 16:43:12 (5800): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:38:42 (7436): No heartbeat from core client for 30 sec - exiting 14:38:43 (7436): No heartbeat from core client for 30 sec - exiting 14:38:44 (7436): No heartbeat from core client for 30 sec - exiting 14:38:45 (7436): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:04:03 (6128): No heartbeat from core client for 30 sec - exiting 16:04:04 (6128): No heartbeat from core client for 30 sec - exiting 16:04:05 (6128): No heartbeat from core client for 30 sec - exiting 16:04:06 (6128): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:04:07 (6128): No heartbeat from core client for 30 sec - exiting 10:19:04 (5624): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:17:24 (6476): No heartbeat from core client for 30 sec - exiting 14:17:26 (6476): No heartbeat from core client for 30 sec - exiting 14:17:27 (6476): No heartbeat from core client for 30 sec - exiting 14:17:28 (6476): No heartbeat from core client for 30 sec - exiting 14:17:29 (6476): No heartbeat from core client for 30 sec - exiting 14:17:30 (6476): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:53:39 (3832): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 10:41:45 (5648): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:16:29 (6140): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:15:45 (6100): No heartbeat from core client for 30 sec - exiting 10:15:47 (6100): No heartbeat from core client for 30 sec - exiting 10:15:48 (6100): No heartbeat from core client for 30 sec - exiting 10:15:49 (6100): No heartbeat from core client for 30 sec - exiting 10:15:50 (6100): No heartbeat from core client for 30 sec - exiting 10:15:51 (6100): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:06:38 (4632): No heartbeat from core client for 30 sec - exiting 10:06:39 (4632): No heartbeat from core client for 30 sec - exiting 10:06:40 (4632): No heartbeat from core client for 30 sec - exiting 10:06:41 (4632): No heartbeat from core client for 30 sec - exiting 10:06:42 (4632): No heartbeat from core client for 30 sec - exiting 10:06:43 (4632): No heartbeat from core client for 30 sec - exiting 10:06:44 (4632): No heartbeat from core client for 30 sec - exiting 10:06:45 (4632): No heartbeat from core client for 30 sec - exiting 10:06:46 (4632): No heartbeat from core client for 30 sec - exiting 10:06:47 (4632): No heartbeat from core client for 30 sec - exiting 10:06:48 (4632): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:00:18 (6080): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:42:02 (6756): No heartbeat from core client for 30 sec - exiting 11:42:03 (6756): No heartbeat from core client for 30 sec - exiting 11:42:04 (6756): No heartbeat from core client for 30 sec - exiting 11:42:05 (6756): No heartbeat from core client for 30 sec - exiting 11:42:06 (6756): No heartbeat from core client for 30 sec - exiting 11:42:07 (6756): No heartbeat from core client for 30 sec - exiting 11:42:08 (6756): No heartbeat from core client for 30 sec - exiting 11:42:09 (6756): No heartbeat from core client for 30 sec - exiting 11:42:10 (6756): No heartbeat from core client for 30 sec - exiting 11:42:11 (6756): No heartbeat from core client for 30 sec - exiting 11:42:12 (6756): No heartbeat from core client for 30 sec - exiting 11:42:13 (6756): No heartbeat from core client for 30 sec - exiting 11:42:14 (6756): No heartbeat from core client for 30 sec - exiting 11:42:15 (6756): No heartbeat from core client for 30 sec - exiting 11:42:16 (6756): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:57:31 (4608): No heartbeat from core client for 30 sec - exiting 10:57:32 (4608): No heartbeat from core client for 30 sec - exiting 10:57:33 (4608): No heartbeat from core client for 30 sec - exiting 10:57:34 (4608): No heartbeat from core client for 30 sec - exiting 10:57:35 (4608): No heartbeat from core client for 30 sec - exiting 10:57:36 (4608): No heartbeat from core client for 30 sec - exiting 10:57:37 (4608): No heartbeat from core client for 30 sec - exiting 10:57:38 (4608): No heartbeat from core client for 30 sec - exiting 10:57:39 (4608): No heartbeat from core client for 30 sec - exiting 10:57:40 (4608): No heartbeat from core client for 30 sec - exiting 10:57:41 (4608): No heartbeat from core client for 30 sec - exiting 10:57:42 (4608): No heartbeat from core client for 30 sec - exiting 10:57:43 (4608): No heartbeat from core client for 30 sec - exiting 10:57:44 (4608): No heartbeat from core client for 30 sec - exiting 10:57:45 (4608): No heartbeat from core client for 30 sec - exiting 10:57:46 (4608): No heartbeat from core client for 30 sec - exiting 10:57:47 (4608): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:30:55 (6128): No heartbeat from core client for 30 sec - exiting 10:30:56 (6128): No heartbeat from core client for 30 sec - exiting 10:30:57 (6128): No heartbeat from core client for 30 sec - exiting 10:30:58 (6128): No heartbeat from core client for 30 sec - exiting 10:30:59 (6128): No heartbeat from core client for 30 sec - exiting 10:31:00 (6128): No heartbeat from core client for 30 sec - exiting 10:31:01 (6128): No heartbeat from core client for 30 sec - exiting 10:31:02 (6128): No heartbeat from core client for 30 sec - exiting 10:31:03 (6128): No heartbeat from core client for 30 sec - exiting 10:31:04 (6128): No heartbeat from core client for 30 sec - exiting 10:31:05 (6128): No heartbeat from core client for 30 sec - exiting 10:31:06 (6128): No heartbeat from core client for 30 sec - exiting 10:31:07 (6128): No heartbeat from core client for 30 sec - exiting 10:31:08 (6128): No heartbeat from core client for 30 sec - exiting 10:31:09 (6128): No heartbeat from core client for 30 sec - exiting 10:31:10 (6128): No heartbeat from core client for 30 sec - exiting 10:31:11 (6128): No heartbeat from core client for 30 sec - exiting 10:31:12 (6128): No heartbeat from core client for 30 sec - exiting 10:31:13 (6128): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:52:53 (8460): No heartbeat from core client for 30 sec - exiting 18:52:54 (8460): No heartbeat from core client for 30 sec - exiting 18:52:55 (8460): No heartbeat from core client for 30 sec - exiting 18:52:56 (8460): No heartbeat from core client for 30 sec - exiting 18:52:57 (8460): No heartbeat from core client for 30 sec - exiting 18:52:58 (8460): No heartbeat from core client for 30 sec - exiting 18:52:59 (8460): No heartbeat from core client for 30 sec - exiting 18:53:00 (8460): No heartbeat from core client for 30 sec - exiting 18:53:01 (8460): No heartbeat from core client for 30 sec - exiting 18:53:02 (8460): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:12:26 (6060): No heartbeat from core client for 30 sec - exiting 11:12:28 (6060): No heartbeat from core client for 30 sec - exiting 11:12:29 (6060): No heartbeat from core client for 30 sec - exiting 11:12:30 (6060): No heartbeat from core client for 30 sec - exiting 11:12:31 (6060): No heartbeat from core client for 30 sec - exiting 11:12:32 (6060): No heartbeat from core client for 30 sec - exiting 11:12:33 (6060): No heartbeat from core client for 30 sec - exiting 11:12:34 (6060): No heartbeat from core client for 30 sec - exiting 11:12:35 (6060): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:43:54 (6180): No heartbeat from core client for 30 sec - exiting 13:43:55 (6180): No heartbeat from core client for 30 sec - exiting 13:43:56 (6180): No heartbeat from core client for 30 sec - exiting 13:43:57 (6180): No heartbeat from core client for 30 sec - exiting 13:43:58 (6180): No heartbeat from core client for 30 sec - exiting 13:43:59 (6180): No heartbeat from core client for 30 sec - exiting 13:44:00 (6180): No heartbeat from core client for 30 sec - exiting 13:44:01 (6180): No heartbeat from core client for 30 sec - exiting 13:44:02 (6180): No heartbeat from core client for 30 sec - exiting 13:44:03 (6180): No heartbeat from core client for 30 sec - exiting 13:44:04 (6180): No heartbeat from core client for 30 sec - exiting 13:44:05 (6180): No heartbeat from core client for 30 sec - exiting 13:44:06 (6180): No heartbeat from core client for 30 sec - exiting 13:44:07 (6180): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:09:12 (5372): No heartbeat from core client for 30 sec - exiting 15:09:13 (5372): No heartbeat from core client for 30 sec - exiting 15:09:14 (5372): No heartbeat from core client for 30 sec - exiting 15:09:15 (5372): No heartbeat from core client for 30 sec - exiting 15:09:16 (5372): No heartbeat from core client for 30 sec - exiting 15:09:17 (5372): No heartbeat from core client for 30 sec - exiting 15:09:18 (5372): No heartbeat from core client for 30 sec - exiting 15:09:19 (5372): No heartbeat from core client for 30 sec - exiting 15:09:20 (5372): No heartbeat from core client for 30 sec - exiting 15:09:21 (5372): No heartbeat from core client for 30 sec - exiting 15:09:22 (5372): No heartbeat from core client for 30 sec - exiting 15:09:23 (5372): No heartbeat from core client for 30 sec - exiting 15:09:24 (5372): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:26:28 (3936): No heartbeat from core client for 30 sec - exiting 15:26:30 (3936): No heartbeat from core client for 30 sec - exiting 15:26:31 (3936): No heartbeat from core client for 30 sec - exiting 15:26:32 (3936): No heartbeat from core client for 30 sec - exiting 15:26:33 (3936): No heartbeat from core client for 30 sec - exiting 15:26:34 (3936): No heartbeat from core client for 30 sec - exiting 15:26:35 (3936): No heartbeat from core client for 30 sec - exiting 15:26:36 (3936): No heartbeat from core client for 30 sec - exiting 15:26:37 (3936): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CSuspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6068, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... 15:21:10 (8056): No heartbeat from core client for 30 sec - exiting 15:21:11 (8056): No heartbeat from core client for 30 sec - exiting 15:21:12 (8056): No heartbeat from core client for 30 sec - exiting 15:21:13 (8056): No heartbeat from core client for 30 sec - exiting 15:21:14 (8056): No heartbeat from core client for 30 sec - exiting 15:21:15 (8056): No heartbeat from core client for 30 sec - exiting 15:21:16 (8056): No heartbeat from core client for 30 sec - exiting 15:21:17 (8056): No heartbeat from core client for 30 sec - exiting 15:21:18 (8056): No heartbeat from core client for 30 sec - exiting 15:21:19 (8056): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:24:33 (1276): No heartbeat from core client for 30 sec - exiting 15:24:34 (1276): No heartbeat from core client for 30 sec - exiting 15:24:35 (1276): No heartbeat from core client for 30 sec - exiting 15:24:36 (1276): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77973AB3 read attempt to address 0x40D0C3DA Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77703AB3 read attempt to address 0x40D0C3DA Engaging BOINC Windows Runtime Debugger... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
19 Apr 2013 17:52:03	1164908	15635544	hadcm3n_3jm0_1980_40_008320697_0	518,400	890,677	1.7181
14 Apr 2013 10:47:15	1164908	15635544	hadcm3n_3jm0_1980_40_008320697_0	492,480	845,744	1.7173
13 Apr 2013 09:18:36	1164908	15635544	hadcm3n_3jm0_1980_40_008320697_0	466,560	801,443	1.7178
07 Apr 2013 13:55:13	1164908	15635544	hadcm3n_3jm0_1980_40_008320697_0	440,640	758,525	1.7214
06 Apr 2013 14:20:26	1164908	15635544	hadcm3n_3jm0_1980_40_008320697_0	414,720	716,454	1.7276
05 Apr 2013 13:26:43	1164908	15635544	hadcm3n_3jm0_1980_40_008320697_0	388,800	673,734	1.7329
04 Apr 2013 11:31:41	1164908	15635544	hadcm3n_3jm0_1980_40_008320697_0	362,880	628,991	1.7333
02 Apr 2013 17:34:27	1164908	15635544	hadcm3n_3jm0_1980_40_008320697_0	336,960	581,037	1.7244
01 Apr 2013 08:59:29	1164908	15635544	hadcm3n_3jm0_1980_40_008320697_0	311,040	532,437	1.7118
30 Mar 2013 18:56:40	1164908	15635544	hadcm3n_3jm0_1980_40_008320697_0	285,120	484,568	1.6995
29 Mar 2013 17:11:08	1164908	15635544	hadcm3n_3jm0_1980_40_008320697_0	259,200	436,145	1.6827
28 Mar 2013 15:00:45	1164908	15635544	hadcm3n_3jm0_1980_40_008320697_0	233,280	389,750	1.6707
23 Mar 2013 16:10:50	1164908	15635544	hadcm3n_3jm0_1980_40_008320697_0	207,360	349,114	1.6836
22 Mar 2013 16:20:38	1164908	15635544	hadcm3n_3jm0_1980_40_008320697_0	181,440	308,030	1.6977
16 Mar 2013 19:48:50	1164908	15635544	hadcm3n_3jm0_1980_40_008320697_0	155,520	265,263	1.7057
16 Mar 2013 09:23:22	1164908	15635544	hadcm3n_3jm0_1980_40_008320697_0	129,600	222,000	1.7130
10 Mar 2013 10:16:45	1164908	15635544	hadcm3n_3jm0_1980_40_008320697_0	103,680	176,359	1.7010
08 Mar 2013 16:45:58	1164908	15635544	hadcm3n_3jm0_1980_40_008320697_0	77,760	132,224	1.7004
03 Mar 2013 11:51:32	1164908	15635544	hadcm3n_3jm0_1980_40_008320697_0	51,840	88,581	1.7087
02 Mar 2013 09:43:44	1164908	15635544	hadcm3n_3jm0_1980_40_008320697_0	25,920	44,180	1.7045