Task 15427076

Name	hadcm3n_yhkv_1980_40_008244106_1
Workunit	8399230
Created	5 Nov 2012, 13:47:05 UTC
Sent	5 Nov 2012, 13:47:07 UTC
Report deadline	4 Feb 2013, 21:14:18 UTC
Received	24 Jan 2013, 14:36:00 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	984012
Run time	24 days 9 hours 54 min 2 sec
CPU time	21 days 5 hours 1 min
Validate state	Invalid
Credit	9,331.20
Device peak FLOPS	2.12 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.6.28</core_client_version> <![CDATA[ <message> - exit code 193 (0xc1) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3404, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... 13:01:03 (4280): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:01:04 (4280): No heartbeat from core client for 30 sec - exiting 13:03:26 (4828): No heartbeat from core client for 30 sec - exiting 13:04:01 (4828): No heartbeat from core client for 30 sec - exiting 13:04:02 (4828): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:51:13 (6140): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:51:15 (6140): No heartbeat from core client for 30 sec - exiting 14:07:15 (2708): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:07:25 (2708): No heartbeat from core client for 30 sec - exiting 14:07:26 (2708): No heartbeat from core client for 30 sec - exiting 14:07:27 (2708): No heartbeat from core client for 30 sec - exiting 14:11:19 (5944): No heartbeat from core client for 30 sec - exiting 14:11:20 (5944): No heartbeat from core client for 30 sec - exiting 14:11:21 (5944): No heartbeat from core client for 30 sec - exiting 14:11:22 (5944): No heartbeat from core client for 30 sec - exiting 14:11:23 (5944): No heartbeat from core client for 30 sec - exiting 14:11:24 (5944): No heartbeat from core client for 30 sec - exiting 14:11:25 (5944): No heartbeat from core client for 30 sec - exiting 14:11:26 (5944): No heartbeat from core client for 30 sec - exiting 14:11:27 (5944): No heartbeat from core client for 30 sec - exiting 14:11:28 (5944): No heartbeat from core client for 30 sec - exiting 14:11:29 (5944): No heartbeat from core client for 30 sec - exiting 14:11:30 (5944): No heartbeat from core client for 30 sec - exiting 14:11:31 (5944): No heartbeat from core client for 30 sec - exiting 14:11:32 (5944): No heartbeat from core client for 30 sec - exiting 14:11:33 (5944): No heartbeat from core client for 30 sec - exiting 14:11:34 (5944): No heartbeat from core client for 30 sec - exiting 14:11:35 (5944): No heartbeat from core client for 30 sec - exiting 14:11:36 (5944): No heartbeat from core client for 30 sec - exiting 14:11:37 (5944): No heartbeat from core client for 30 sec - exiting 14:11:38 (5944): No heartbeat from core client for 30 sec - exiting 14:11:39 (5944): No heartbeat from core client for 30 sec - exiting 14:11:40 (5944): No heartbeat from core client for 30 sec - exiting 14:11:41 (5944): No heartbeat from core client for 30 sec - exiting 14:11:42 (5944): No heartbeat from core client for 30 sec - exiting 14:11:43 (5944): No heartbeat from core client for 30 sec - exiting 14:11:44 (5944): No heartbeat from core client for 30 sec - exiting 14:11:45 (5944): No heartbeat from core client for 30 sec - exiting 14:11:46 (5944): No heartbeat from core client for 30 sec - exiting 14:11:47 (5944): No heartbeat from core client for 30 sec - exiting 14:11:48 (5944): No heartbeat from core client for 30 sec - exiting 14:11:49 (5944): No heartbeat from core client for 30 sec - exiting 14:11:50 (5944): No heartbeat from core client for 30 sec - exiting 14:11:51 (5944): No heartbeat from core client for 30 sec - exiting 14:11:52 (5944): No heartbeat from core client for 30 sec - exiting 14:11:53 (5944): No heartbeat from core client for 30 sec - exiting 14:11:54 (5944): No heartbeat from core client for 30 sec - exiting 14:11:55 (5944): No heartbeat from core client for 30 sec - exiting 14:11:56 (5944): No heartbeat from core client for 30 sec - exiting 14:11:57 (5944): No heartbeat from core client for 30 sec - exiting 14:11:58 (5944): No heartbeat from core client for 30 sec - exiting 14:11:59 (5944): No heartbeat from core client for 30 sec - exiting 14:12:00 (5944): No heartbeat from core client for 30 sec - exiting 14:12:01 (5944): No heartbeat from core client for 30 sec - exiting 14:12:02 (5944): No heartbeat from core client for 30 sec - exiting 14:12:03 (5944): No heartbeat from core client for 30 sec - exiting 14:12:04 (5944): No heartbeat from core client for 30 sec - exiting 14:12:05 (5944): No heartbeat from core client for 30 sec - exiting 14:12:06 (5944): No heartbeat from core client for 30 sec - exiting 14:12:07 (5944): No heartbeat from core client for 30 sec - exiting 14:12:08 (5944): No heartbeat from core client for 30 sec - exiting 14:12:09 (5944): No heartbeat from core client for 30 sec - exiting 14:12:10 (5944): No heartbeat from core client for 30 sec - exiting 14:12:11 (5944): No heartbeat from core client for 30 sec - exiting 14:12:12 (5944): No heartbeat from core client for 30 sec - exiting 14:12:13 (5944): No heartbeat from core client for 30 sec - exiting 14:12:14 (5944): No heartbeat from core client for 30 sec - exiting 14:12:15 (5944): No heartbeat from core client for 30 sec - exiting 14:12:16 (5944): No heartbeat from core client for 30 sec - exiting 14:12:17 (5944): No heartbeat from core client for 30 sec - exiting 14:12:18 (5944): No heartbeat from core client for 30 sec - exiting 14:12:19 (5944): No heartbeat from core client for 30 sec - exiting 14:12:20 (5944): No heartbeat from core client for 30 sec - exiting 14:12:21 (5944): No heartbeat from core client for 30 sec - exiting 14:12:22 (5944): No heartbeat from core client for 30 sec - exiting 14:12:23 (5944): No heartbeat from core client for 30 sec - exiting 14:12:24 (5944): No heartbeat from core client for 30 sec - exiting 14:12:25 (5944): No heartbeat from core client for 30 sec - exiting 14:12:26 (5944): No heartbeat from core client for 30 sec - exiting 14:12:27 (5944): No heartbeat from core client for 30 sec - exiting 14:12:28 (5944): No heartbeat from core client for 30 sec - exiting 14:12:29 (5944): No heartbeat from core client for 30 sec - exiting 14:12:30 (5944): No heartbeat from core client for 30 sec - exiting 14:12:31 (5944): No heartbeat from core client for 30 sec - exiting 14:12:32 (5944): No heartbeat from core client for 30 sec - exiting 14:12:33 (5944): No heartbeat from core client for 30 sec - exiting 14:12:34 (5944): No heartbeat from core client for 30 sec - exiting 14:12:35 (5944): No heartbeat from core client for 30 sec - exiting 14:12:36 (5944): No heartbeat from core client for 30 sec - exiting 14:12:37 (5944): No heartbeat from core client for 30 sec - exiting 14:12:38 (5944): No heartbeat from core client for 30 sec - exiting 14:12:39 (5944): No heartbeat from core client for 30 sec - exiting 14:12:40 (5944): No heartbeat from core client for 30 sec - exiting 14:12:41 (5944): No heartbeat from core client for 30 sec - exiting 14:12:42 (5944): No heartbeat from core client for 30 sec - exiting 14:12:43 (5944): No heartbeat from core client for 30 sec - exiting 14:12:44 (5944): No heartbeat from core client for 30 sec - exiting 14:12:45 (5944): No heartbeat from core client for 30 sec - exiting 14:12:46 (5944): No heartbeat from core client for 30 sec - exiting 14:12:47 (5944): No heartbeat from core client for 30 sec - exiting 14:12:48 (5944): No heartbeat from core client for 30 sec - exiting 14:12:49 (5944): No heartbeat from core client for 30 sec - exiting 14:12:50 (5944): No heartbeat from core client for 30 sec - exiting 14:12:51 (5944): No heartbeat from core client for 30 sec - exiting 14:12:52 (5944): No heartbeat from core client for 30 sec - exiting 14:12:53 (5944): No heartbeat from core client for 30 sec - exiting 14:12:54 (5944): No heartbeat from core client for 30 sec - exiting 14:12:55 (5944): No heartbeat from core client for 30 sec - exiting 14:12:56 (5944): No heartbeat from core client for 30 sec - exiting 14:12:57 (5944): No heartbeat from core client for 30 sec - exiting 14:12:58 (5944): No heartbeat from core client for 30 sec - exiting 14:12:59 (5944): No heartbeat from core client for 30 sec - exiting 14:13:00 (5944): No heartbeat from core client for 30 sec - exiting 14:13:01 (5944): No heartbeat from core client for 30 sec - exiting 14:13:02 (5944): No heartbeat from core client for 30 sec - exiting 14:13:03 (5944): No heartbeat from core client for 30 sec - exiting 14:13:04 (5944): No heartbeat from core client for 30 sec - exiting 14:13:05 (5944): No heartbeat from core client for 30 sec - exiting 14:13:06 (5944): No heartbeat from core client for 30 sec - exiting 14:13:07 (5944): No heartbeat from core client for 30 sec - exiting 14:13:10 (5944): No heartbeat from core client for 30 sec - exiting 14:13:11 (5944): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:13:12 (5944): No heartbeat from core client for 30 sec - exiting 14:13:13 (5944): No heartbeat from core client for 30 sec - exiting 14:13:14 (5944): No heartbeat from core client for 30 sec - exiting 14:13:15 (5944): No heartbeat from core client for 30 sec - exiting 14:13:16 (5944): No heartbeat from core client for 30 sec - exiting 14:13:20 (5944): No heartbeat from core client for 30 sec - exiting 14:14:41 (5788): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:14:53 (5788): No heartbeat from core client for 30 sec - exiting 14:14:54 (5788): No heartbeat from core client for 30 sec - exiting 14:14:55 (5788): No heartbeat from core client for 30 sec - exiting 14:14:56 (5788): No heartbeat from core client for 30 sec - exiting 14:14:57 (5788): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CCPDN Monitor - Quit request from BOINC... CCPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5104, iMonCtr=1 Model crash detected, will try to restart... 15:34:39 (276): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:34:40 (276): No heartbeat from core client for 30 sec - exiting 15:34:41 (276): No heartbeat from core client for 30 sec - exiting 15:34:42 (276): No heartbeat from core client for 30 sec - exiting 11:23:01 (4728): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:24:37 (2888): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:06:36 (4000): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:07:01 (4000): No heartbeat from core client for 30 sec - exiting 15:07:02 (4000): No heartbeat from core client for 30 sec - exiting 15:07:03 (4000): No heartbeat from core client for 30 sec - exiting 15:07:05 (4000): No heartbeat from core client for 30 sec - exiting 15:07:06 (4000): No heartbeat from core client for 30 sec - exiting 15:07:07 (4000): No heartbeat from core client for 30 sec - exiting 15:07:08 (4000): No heartbeat from core client for 30 sec - exiting 15:09:36 (3456): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:12:34 (5592): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:12:43 (5592): No heartbeat from core client for 30 sec - exiting 15:12:44 (5592): No heartbeat from core client for 30 sec - exiting 15:12:45 (5592): No heartbeat from core client for 30 sec - exiting 15:12:46 (5592): No heartbeat from core client for 30 sec - exiting 15:12:47 (5592): No heartbeat from core client for 30 sec - exiting 15:12:48 (5592): No heartbeat from core client for 30 sec - exiting 15:12:49 (5592): No heartbeat from core client for 30 sec - exiting 15:12:50 (5592): No heartbeat from core client for 30 sec - exiting 15:12:51 (5592): No heartbeat from core client for 30 sec - exiting 15:12:53 (5592): No heartbeat from core client for 30 sec - exiting 08:53:50 (2668): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:55:43 (2768): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:56:34 (308): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:57:17 (5996): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:58:00 (5556): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:41:18 (4568): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:41:19 (4568): No heartbeat from core client for 30 sec - exiting 10:26:19 (3620): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:26:22 (3620): No heartbeat from core client for 30 sec - exiting 10:26:23 (3620): No heartbeat from core client for 30 sec - exiting 10:26:25 (3620): No heartbeat from core client for 30 sec - exiting 10:27:55 (1232): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:29:20 (4316): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:31:39 (4208): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:24:25 (5640): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:24:28 (5640): No heartbeat from core client for 30 sec - exiting 17:24:29 (5640): No heartbeat from core client for 30 sec - exiting 17:24:30 (5640): No heartbeat from core client for 30 sec - exiting 17:24:31 (5640): No heartbeat from core client for 30 sec - exiting 17:26:27 (6096): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:26:29 (6096): No heartbeat from core client for 30 sec - exiting 17:26:31 (6096): No heartbeat from core client for 30 sec - exiting 17:26:32 (6096): No heartbeat from core client for 30 sec - exiting 17:26:33 (6096): No heartbeat from core client for 30 sec - exiting 17:26:34 (6096): No heartbeat from core client for 30 sec - exiting 17:28:17 (4572): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:28:19 (4572): No heartbeat from core client for 30 sec - exiting 17:28:21 (4572): No heartbeat from core client for 30 sec - exiting 17:28:22 (4572): No heartbeat from core client for 30 sec - exiting 17:28:23 (4572): No heartbeat from core client for 30 sec - exiting 17:28:24 (4572): No heartbeat from core client for 30 sec - exiting 17:28:25 (4572): No heartbeat from core client for 30 sec - exiting 17:28:26 (4572): No heartbeat from core client for 30 sec - exiting 13:26:04 (1992): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:26:05 (1992): No heartbeat from core client for 30 sec - exiting 13:26:06 (1992): No heartbeat from core client for 30 sec - exiting 13:26:07 (1992): No heartbeat from core client for 30 sec - exiting 13:26:08 (1992): No heartbeat from core client for 30 sec - exiting 13:26:09 (1992): No heartbeat from core client for 30 sec - exiting 13:26:10 (1992): No heartbeat from core client for 30 sec - exiting 13:26:12 (1992): No heartbeat from core client for 30 sec - exiting 13:26:13 (1992): No heartbeat from core client for 30 sec - exiting 13:26:14 (1992): No heartbeat from core client for 30 sec - exiting 13:26:15 (1992): No heartbeat from core client for 30 sec - exiting 13:26:16 (1992): No heartbeat from core client for 30 sec - exiting 13:26:17 (1992): No heartbeat from core client for 30 sec - exiting 09:51:33 (536): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:51:34 (536): No heartbeat from core client for 30 sec - exiting 09:51:35 (536): No heartbeat from core client for 30 sec - exiting 09:51:36 (536): No heartbeat from core client for 30 sec - exiting 09:51:37 (536): No heartbeat from core client for 30 sec - exiting 09:54:03 (6028): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:55:02 (4376): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:56:30 (492): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 13:59:28 (832): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:59:33 (832): No heartbeat from core client for 30 sec - exiting 13:59:34 (832): No heartbeat from core client for 30 sec - exiting 13:59:35 (832): No heartbeat from core client for 30 sec - exiting 13:59:36 (832): No heartbeat from core client for 30 sec - exiting 13:59:37 (832): No heartbeat from core client for 30 sec - exiting 13:59:38 (832): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77B0411D read attempt to address 0x41841D9A Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77156E5F read attempt to address 0x40AAEE5F Engaging BOINC Windows Runtime Debugger... Cannot serialize file C:\ProgramData\BOINC/projects/climatepredictio </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
22 Jan 2013 23:41:29	984012	15427076	hadcm3n_yhkv_1980_40_008244106_1	777,600	1,831,156	2.3549
20 Jan 2013 00:02:55	984012	15427076	hadcm3n_yhkv_1980_40_008244106_1	751,680	1,770,863	2.3559
17 Jan 2013 22:41:55	984012	15427076	hadcm3n_yhkv_1980_40_008244106_1	725,760	1,711,311	2.3580
16 Jan 2013 14:10:37	984012	15427076	hadcm3n_yhkv_1980_40_008244106_1	699,840	1,652,283	2.3609
11 Jan 2013 21:41:41	984012	15427076	hadcm3n_yhkv_1980_40_008244106_1	673,920	1,591,415	2.3614
09 Jan 2013 22:16:40	984012	15427076	hadcm3n_yhkv_1980_40_008244106_1	648,000	1,530,408	2.3617
07 Jan 2013 22:04:22	984012	15427076	hadcm3n_yhkv_1980_40_008244106_1	622,080	1,467,196	2.3585
04 Jan 2013 16:49:36	984012	15427076	hadcm3n_yhkv_1980_40_008244106_1	596,160	1,405,993	2.3584
04 Jan 2013 16:49:36	984012	15427076	hadcm3n_yhkv_1980_40_008244106_1	570,240	1,352,776	2.3723
04 Jan 2013 16:49:36	984012	15427076	hadcm3n_yhkv_1980_40_008244106_1	544,320	1,297,594	2.3839
04 Jan 2013 16:49:36	984012	15427076	hadcm3n_yhkv_1980_40_008244106_1	518,400	1,236,989	2.3862
20 Dec 2012 21:37:12	984012	15427076	hadcm3n_yhkv_1980_40_008244106_1	492,480	1,177,664	2.3913
18 Dec 2012 21:54:29	984012	15427076	hadcm3n_yhkv_1980_40_008244106_1	466,560	1,117,595	2.3954
15 Dec 2012 19:35:10	984012	15427076	hadcm3n_yhkv_1980_40_008244106_1	440,640	1,057,899	2.4008
13 Dec 2012 22:57:49	984012	15427076	hadcm3n_yhkv_1980_40_008244106_1	414,720	999,350	2.4097
13 Dec 2012 22:57:49	984012	15427076	hadcm3n_yhkv_1980_40_008244106_1	388,800	936,375	2.4084
07 Dec 2012 21:53:44	984012	15427076	hadcm3n_yhkv_1980_40_008244106_1	362,880	875,578	2.4129
05 Dec 2012 22:01:04	984012	15427076	hadcm3n_yhkv_1980_40_008244106_1	336,960	813,616	2.4146
04 Dec 2012 14:43:33	984012	15427076	hadcm3n_yhkv_1980_40_008244106_1	311,040	753,317	2.4219
01 Dec 2012 16:07:54	984012	15427076	hadcm3n_yhkv_1980_40_008244106_1	285,120	692,250	2.4279