Task 16083484

Name	hadcm3n_u54w_2020_40_008410721_1
Workunit	8561577
Created	21 Nov 2013, 15:21:14 UTC
Sent	21 Nov 2013, 15:21:22 UTC
Report deadline	20 Feb 2014, 22:48:33 UTC
Received	2 Jan 2014, 20:47:09 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	-1073741819 (0xC0000005) STATUS_ACCESS_VIOLATION
Computer ID	1294966
Run time	17 days 10 hours 39 min 21 sec
CPU time	16 days 12 hours 7 min 9 sec
Validate state	Invalid
Credit	12,441.60
Device peak FLOPS	2.95 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.2.33</core_client_version> <![CDATA[ <message> (unknown error) - exit code -1073741819 (0xc0000005) </message> <stderr_txt> 08:31:08 (4256): No heartbeat from core client for 30 sec - exiting 08:31:09 (4256): No heartbeat from core client for 30 sec - exiting 08:31:10 (4256): No heartbeat from core client for 30 sec - exiting 08:31:11 (4256): No heartbeat from core client for 30 sec - exiting 08:31:12 (4256): No heartbeat from core client for 30 sec - exiting 08:31:13 (4256): No heartbeat from core client for 30 sec - exiting 08:31:14 (4256): No heartbeat from core client for 30 sec - exiting 08:31:15 (4256): No heartbeat from core client for 30 sec - exiting 08:31:16 (4256): No heartbeat from core client for 30 sec - exiting 08:31:17 (4256): No heartbeat from core client for 30 sec - exiting 08:31:18 (4256): No heartbeat from core client for 30 sec - exiting 08:31:20 (4256): No heartbeat from core client for 30 sec - exiting 08:31:21 (4256): No heartbeat from core client for 30 sec - exiting 08:31:22 (4256): No heartbeat from core client for 30 sec - exiting 08:31:23 (4256): No heartbeat from core client for 30 sec - exiting 08:31:24 (4256): No heartbeat from core client for 30 sec - exiting 08:31:25 (4256): No heartbeat from core client for 30 sec - exiting 08:31:26 (4256): No heartbeat from core client for 30 sec - exiting 08:31:27 (4256): No heartbeat from core client for 30 sec - exiting 08:31:28 (4256): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 08:34:23 (5000): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3352, iMonCtr=1 Model crash detected, will try to 08:08:06 (4600): No heartbeat from core client for 30 sec - exiting 08:08:07 (4600): No heartbeat from core client for 30 sec - exiting 08:08:08 (4600): No heartbeat from core client for 30 sec - exiting 08:08:09 (4600): No heartbeat from core client for 30 sec - exiting 08:08:10 (4600): No heartbeat from core client for 30 sec - exiting 08:08:11 (4600): No heartbeat from core client for 30 sec - exiting 08:08:12 (4600): No heartbeat from core client for 30 sec - exiting 08:08:13 (4600): No heartbeat from core client for 30 sec - exiting 08:08:15 (4600): No heartbeat from core client for 30 sec - exiting 08:08:16 (4600): No heartbeat from core client for 30 sec - exiting 08:08:17 (4600): No heartbeat from core client for 30 sec - exiting 08:08:18 (4600): No heartbeat from core client for 30 sec - exiting 08:08:19 (4600): No heartbeat from core client for 30 sec - exiting 08:08:20 (4600): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:51:39 (1288): No heartbeat from core client for 30 sec - exiting 11:51:42 (1288): No heartbeat from core client for 30 sec - exiting 11:51:43 (1288): No heartbeat from core client for 30 sec - exiting 11:51:44 (1288): No heartbeat from core client for 30 sec - exiting 11:51:45 (1288): No heartbeat from core client for 30 sec - exiting 11:51:46 (1288): No heartbeat from core client for 30 sec - exiting 11:51:47 (1288): No heartbeat from core client for 30 sec - exiting 11:51:48 (1288): No heartbeat from core client for 30 sec - exiting 11:51:49 (1288): No heartbeat from core client for 30 sec - exiting 11:51:50 (1288): No heartbeat from core client for 30 sec - exiting 11:51:51 (1288): No heartbeat from core client for 30 sec - exiting 11:51:52 (1288): No heartbeat from core client for 30 sec - exiting 11:51:54 (1288): No heartbeat from core client for 30 sec - exiting 11:51:55 (1288): No heartbeat from core client for 30 sec - exiting 11:51:56 (1288): No heartbeat from core client for 30 sec - exiting 11:51:57 (1288): No heartbeat from core client for 30 sec - exiting 11:51:58 (1288): No heartbeat from core client for 30 sec - exiting 11:51:59 (1288): No heartbeat from core client for 30 sec - exiting 11:52:00 (1288): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 09:37:36 (4640): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 09:43:43 (4672): No heartbeat from core client for 30 sec - exiting 09:43:44 (4672): No heartbeat from core client for 30 sec - exiting 09:43:45 (4672): No heartbeat from core client for 30 sec - exiting 09:43:46 (4672): No heartbeat from core client for 30 sec - exiting 09:43:47 (4672): No heartbeat from core client for 30 sec - exiting 09:43:48 (4672): No heartbeat from core client for 30 sec - exiting 09:43:50 (4672): No heartbeat from core client for 30 sec - exiting 09:43:51 (4672): No heartbeat from core client for 30 sec - exiting 09:43:52 (4672): No heartbeat from core client for 30 sec - exiting 09:43:53 (4672): No heartbeat from core client for 30 sec - exiting 09:43:54 (4672): No heartbeat from core client for 30 sec - exiting 09:43:55 (4672): No heartbeat from core client for 30 sec - exiting 09:43:56 (4672): No heartbeat from core client for 30 sec - exiting 09:43:57 (4672): No heartbeat from core client for 30 sec - exiting 09:43:58 (4672): No heartbeat from core client for 30 sec - exiting 09:43:59 (4672): No heartbeat from core client for 30 sec - exiting 09:44:01 (4672): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3004, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 09:12:31 (4468): No heartbeat from core client for 30 sec - exiting 09:12:32 (4468): No heartbeat from core client for 30 sec - exiting 09:12:33 (4468): No heartbeat from core client for 30 sec - exiting 09:12:35 (4468): No heartbeat from core client for 30 sec - exiting 09:12:36 (4468): No heartbeat from core client for 30 sec - exiting 09:12:37 (4468): No heartbeat from core client for 30 sec - exiting 09:12:38 (4468): No heartbeat from core client for 30 sec - exiting 09:12:39 (4468): No heartbeat from core client for 30 sec - exiting 09:12:40 (4468): No heartbeat from core client for 30 sec - exiting 09:12:41 (4468): No heartbeat from core client for 30 sec - exiting 09:12:42 (4468): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3248, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=952, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 08:31:26 (4468): No heartbeat from core client for 30 sec - exiting 08:31:27 (4468): No heartbeat from core client for 30 sec - exiting 08:31:28 (4468): No heartbeat from core client for 30 sec - exiting 08:31:29 (4468): No heartbeat from core client for 30 sec - exiting 08:31:30 (4468): No heartbeat from core client for 30 sec - exiting 08:31:31 (4468): No heartbeat from core client for 30 sec - exiting 08:31:32 (4468): No heartbeat from core client for 30 sec - exiting 08:31:33 (4468): No heartbeat from core client for 30 sec - exiting 08:31:34 (4468): No heartbeat from core client for 30 sec - exiting 08:31:36 (4468): No heartbeat from core client for 30 sec - exiting 08:31:37 (4468): No heartbeat from core client for 30 sec - exiting 08:31:38 (4468): No heartbeat from core client for 30 sec - exiting 08:31:39 (4468): No heartbeat from core client for 30 sec - exiting 08:31:40 (4468): No heartbeat from core client for 30 sec - exiting 08:31:41 (4468): No heartbeat from core client for 30 sec - exiting 08:31:42 (4468): No heartbeat from core client for 30 sec - exiting 08:31:43 (4468): No heartbeat from core client for 30 sec - exiting 08:31:44 (4468): No heartbeat from core client for 30 sec - exiting 08:31:45 (4468): No heartbeat from core client for 30 sec - exiting 08:31:46 (4468): No heartbeat from core client for 30 sec - exiting 08:31:48 (4468): No heartbeat from core client for 30 sec - exiting 08:31:49 (4468): No heartbeat from core client for 30 sec - exiting 08:31:50 (4468): No heartbeat from core client for 30 sec - exiting 08:31:51 (4468): No heartbeat from core client for 30 sec - exiting 08:31:52 (4468): No heartbeat from core client for 30 sec - exiting 08:31:53 (4468): No heartbeat from core client for 30 sec - exiting 08:31:54 (4468): No heartbeat from core client for 30 sec - exiting 08:31:55 (4468): No heartbeat from core client for 30 sec - exiting 08:31:56 (4468): No heartbeat from core client for 30 sec - exiting 08:31:57 (4468): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 10:37:32 (4852): No heartbeat from core client for 30 sec - exiting 10:37:33 (4852): No heartbeat from core client for 30 sec - exiting 10:37:34 (4852): No heartbeat from core client for 30 sec - exiting 10:37:35 (4852): No heartbeat from core client for 30 sec - exiting 10:37:37 (4852): No heartbeat from core client for 30 sec - exiting 10:37:38 (4852): No heartbeat from core client for 30 sec - exiting 10:37:39 (4852): No heartbeat from core client for 30 sec - exiting 10:37:40 (4852): No heartbeat from core client for 30 sec - exiting 10:37:41 (4852): No heartbeat from core client for 30 sec - exiting 10:37:42 (4852): No heartbeat from core client for 30 sec - exiting 10:37:43 (4852): No heartbeat from core client for 30 sec - exiting 10:37:44 (4852): No heartbeat from core client for 30 sec - exiting 10:37:45 (4852): No heartbeat from core client for 30 sec - exiting 10:37:46 (4852): No heartbeat from core client for 30 sec - exiting 10:37:47 (4852): No heartbeat from core client for 30 sec - exiting 10:37:49 (4852): No heartbeat from core client for 30 sec - exiting 10:37:50 (4852): No heartbeat from core client for 30 sec - exiting 10:37:51 (4852): No heartbeat from core client for 30 sec - exiting 10:37:52 (4852): No heartbeat from core client for 30 sec - exiting 10:37:53 (4852): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3892, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3860, iMonCtr=1 Model crash detected, will try to restart... 22:08:11 (6576): Can't acquire lockfile (32) - waiting 35s 22:08:36 (5400): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... C17:40:19 (4796): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 15:06:58 (4264): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:06:59 (4264): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 12:37:36 (956): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3016, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 11:19:36 (4464): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77593AC3 read attempt to address 0x403B3309 Engaging BOINC Windows Runtime Debugger... Signal 11 received, exiting... Called boinc_finish ERROR: Invalid parameter detected in function (null). File: (null) Line: 0 ERROR: Expression: (null) </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
01 Jan 2014 17:45:43	1294966	16083484	hadcm3n_u54w_2020_40_008410721_1	1,036,800	1,426,048	1.3754
31 Dec 2013 12:05:33	1294966	16083484	hadcm3n_u54w_2020_40_008410721_1	1,010,880	1,389,553	1.3746
29 Dec 2013 21:14:13	1294966	16083484	hadcm3n_u54w_2020_40_008410721_1	984,960	1,353,291	1.3740
28 Dec 2013 21:04:59	1294966	16083484	hadcm3n_u54w_2020_40_008410721_1	959,040	1,316,693	1.3729
27 Dec 2013 10:36:45	1294966	16083484	hadcm3n_u54w_2020_40_008410721_1	933,120	1,280,619	1.3724
25 Dec 2013 18:04:35	1294966	16083484	hadcm3n_u54w_2020_40_008410721_1	907,200	1,244,029	1.3713
24 Dec 2013 14:15:06	1294966	16083484	hadcm3n_u54w_2020_40_008410721_1	881,280	1,206,883	1.3695
23 Dec 2013 13:22:16	1294966	16083484	hadcm3n_u54w_2020_40_008410721_1	855,360	1,170,276	1.3682
21 Dec 2013 19:01:06	1294966	16083484	hadcm3n_u54w_2020_40_008410721_1	829,440	1,133,889	1.3671
20 Dec 2013 14:20:43	1294966	16083484	hadcm3n_u54w_2020_40_008410721_1	803,520	1,097,439	1.3658
19 Dec 2013 09:25:27	1294966	16083484	hadcm3n_u54w_2020_40_008410721_1	777,600	1,060,203	1.3634
18 Dec 2013 12:13:07	1294966	16083484	hadcm3n_u54w_2020_40_008410721_1	751,680	1,023,741	1.3619
16 Dec 2013 16:59:59	1294966	16083484	hadcm3n_u54w_2020_40_008410721_1	725,760	989,984	1.3641
15 Dec 2013 20:05:20	1294966	16083484	hadcm3n_u54w_2020_40_008410721_1	699,840	956,183	1.3663
15 Dec 2013 10:26:50	1294966	16083484	hadcm3n_u54w_2020_40_008410721_1	673,920	922,385	1.3687
14 Dec 2013 12:42:17	1294966	16083484	hadcm3n_u54w_2020_40_008410721_1	648,000	887,178	1.3691
13 Dec 2013 14:35:05	1294966	16083484	hadcm3n_u54w_2020_40_008410721_1	622,080	852,882	1.3710
12 Dec 2013 16:47:10	1294966	16083484	hadcm3n_u54w_2020_40_008410721_1	596,160	819,284	1.3743
11 Dec 2013 19:13:20	1294966	16083484	hadcm3n_u54w_2020_40_008410721_1	570,240	785,818	1.3780
11 Dec 2013 09:48:34	1294966	16083484	hadcm3n_u54w_2020_40_008410721_1	544,320	752,225	1.3820