Task 15686664

Name	hadcm3n_u6sb_2020_40_008337640_0
Workunit	8488501
Created	27 Mar 2013, 10:33:07 UTC
Sent	27 Mar 2013, 10:33:53 UTC
Report deadline	26 Jun 2013, 18:01:04 UTC
Received	29 May 2013, 14:54:50 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	-1073741819 (0xC0000005) STATUS_ACCESS_VIOLATION
Computer ID	1213970
Run time	17 days 15 hours 3 min 40 sec
CPU time	16 days 15 hours 6 min 16 sec
Validate state	Invalid
Credit	9,331.20
Device peak FLOPS	2.34 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.1.1</core_client_version> <![CDATA[ <message> - exit code -1073741819 (0xc0000005) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... 08:59:17 (2720): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4012, iMonCtr=1 Model crash detected, will try to restart... 09:40:00 (6132): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4044, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... 08:25:38 (2876): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:25:42 (2876): No heartbeat from core client for 30 sec - exiting 08:25:43 (2876): No heartbeat from core client for 30 sec - exiting 08:25:44 (2876): No heartbeat from core client for 30 sec - exiting 08:25:45 (2876): No heartbeat from core client for 30 sec - exiting 11:29:00 (11032): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:29:01 (11032): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 09:53:44 (2800): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:53:49 (2800): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 17:22:23 (4060): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:22:27 (4060): No heartbeat from core client for 30 sec - exiting 17:22:28 (4060): No heartbeat from core client for 30 sec - exiting 17:22:29 (4060): No heartbeat from core client for 30 sec - exiting 17:22:30 (4060): No heartbeat from core client for 30 sec - exiting 17:22:31 (4060): No heartbeat from core client for 30 sec - exiting CSuspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 09:42:00 (1108): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:49:29 (4952): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:49:33 (4952): No heartbeat from core client for 30 sec - exiting 10:49:34 (4952): No heartbeat from core client for 30 sec - exiting 10:49:35 (4952): No heartbeat from core client for 30 sec - exiting Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3836, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3780, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3760, iMonCtr=1 Model crash detected, will try to restart... 10:04:23 (3108): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:10:39 (2168): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:44:21 (7404): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:04:26 (8136): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=11496, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... 11:39:30 (4020): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:39:35 (4020): No heartbeat from core client for 30 sec - exiting 11:39:36 (4020): No heartbeat from core client for 30 sec - exiting 11:39:37 (4020): No heartbeat from core client for 30 sec - exiting 11:39:39 (4020): No heartbeat from core client for 30 sec - exiting 11:39:40 (4020): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... 19:55:56 (3388): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:25:30 (4048): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:25:36 (4048): No heartbeat from core client for 30 sec - exiting 09:25:37 (4048): No heartbeat from core client for 30 sec - exiting 09:25:38 (4048): No heartbeat from core client for 30 sec - exiting 09:25:39 (4048): No heartbeat from core client for 30 sec - exiting 09:25:40 (4048): No heartbeat from core client for 30 sec - exiting 09:28:29 (8112): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:33:45 (3004): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:33:49 (3004): No heartbeat from core client for 30 sec - exiting 11:33:50 (3004): No heartbeat from core client for 30 sec - exiting 11:33:51 (3004): No heartbeat from core client for 30 sec - exiting 11:33:52 (3004): No heartbeat from core client for 30 sec - exiting 11:33:53 (3004): No heartbeat from core client for 30 sec - exiting 11:33:55 (3004): No heartbeat from core client for 30 sec - exiting 11:33:56 (3004): No heartbeat from core client for 30 sec - exiting 11:33:57 (3004): No heartbeat from core client for 30 sec - exiting 11:33:58 (3004): No heartbeat from core client for 30 sec - exiting 13:01:55 (12208): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:01:59 (12208): No heartbeat from core client for 30 sec - exiting 13:02:00 (12208): No heartbeat from core client for 30 sec - exiting 13:03:16 (9744): No heartbeat from core client for 30 sec - exiting 13:03:17 (9744): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:13:25 (2612): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:13:26 (2612): No heartbeat from core client for 30 sec - exiting 18:48:10 (7576): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:00:56 (368): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:01:01 (368): No heartbeat from core client for 30 sec - exiting 10:01:02 (368): No heartbeat from core client for 30 sec - exiting 23:07:30 (3128): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:26:12 (2964): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:39:05 (9100): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:39:06 (9100): No heartbeat from core client for 30 sec - exiting 23:41:18 (17296): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:37:38 (18428): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:37:42 (18428): No heartbeat from core client for 30 sec - exiting 00:37:43 (18428): No heartbeat from core client for 30 sec - exiting 00:37:44 (18428): No heartbeat from core client for 30 sec - exiting 11:35:24 (4084): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:37:11 (9712): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:37:12 (9712): No heartbeat from core client for 30 sec - exiting 22:07:27 (8028): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:07:29 (8028): No heartbeat from core client for 30 sec - exiting 18:07:12 (2112): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:16:30 (6988): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:59:20 (3156): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:03:31 (364): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:29:14 (6880): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:29:18 (6880): No heartbeat from core client for 30 sec - exiting 10:29:19 (6880): No heartbeat from core client for 30 sec - exiting 10:29:20 (6880): No heartbeat from core client for 30 sec - exiting 10:29:21 (6880): No heartbeat from core client for 30 sec - exiting 10:34:56 (7156): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:26:04 (4572): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:26:07 (4572): No heartbeat from core client for 30 sec - exiting 11:26:08 (4572): No heartbeat from core client for 30 sec - exiting 11:26:09 (4572): No heartbeat from core client for 30 sec - exiting 11:26:10 (4572): No heartbeat from core client for 30 sec - exiting Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3100, iMonCtr=1 Model crash detected, will try to restart... 14:52:42 (3656): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3708, iMonCtr=1 Model crash detected, will try to restart... 17:51:27 (2012): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:15:20 (3680): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2064, iMonCtr=1 Model crash detected, will try to restart... 10:33:20 (4052): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 11:43:54 (4468): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:43:58 (4468): No heartbeat from core client for 30 sec - exiting 11:43:59 (4468): No heartbeat from core client for 30 sec - exiting C19:47:51 (4000): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:23:18 (3484): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:23:33 (2372): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:23:38 (2372): No heartbeat from core client for 30 sec - exiting 22:23:39 (2372): No heartbeat from core client for 30 sec - exiting 22:23:40 (2372): No heartbeat from core client for 30 sec - exiting 22:23:41 (2372): No heartbeat from core client for 30 sec - exiting 13:49:46 (3740): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:05:28 (5536): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 12:31:45 (3916): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:31:47 (3916): No heartbeat from core client for 30 sec - exiting 12:31:48 (3916): No heartbeat from core client for 30 sec - exiting 12:32:56 (7708): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:33:00 (7708): No heartbeat from core client for 30 sec - exiting 12:33:01 (7708): No heartbeat from core client for 30 sec - exiting 12:33:02 (7708): No heartbeat from core client for 30 sec - exiting 12:33:03 (7708): No heartbeat from core client for 30 sec - exiting 12:35:39 (4880): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:21:43 (7000): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2204, iMonCtr=1 Model crash detected, will try to restart... 20:08:54 (2568): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:27:34 (4468): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:32:00 (11156): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:13:00 (8624): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:13:02 (8624): No heartbeat from core client for 30 sec - exiting 21:14:43 (12256): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77D2C3EB write attempt to address 0x4051C491 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77765EAB read attempt to address 0x4051C499 Engaging BOINC Windows Runtime Debugger... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
28 May 2013 14:20:49	1213970	15686664	hadcm3n_u6sb_2020_40_008337640_0	777,600	1,425,992	1.8338
27 May 2013 08:08:43	1213970	15686664	hadcm3n_u6sb_2020_40_008337640_0	751,680	1,379,253	1.8349
26 May 2013 06:14:23	1213970	15686664	hadcm3n_u6sb_2020_40_008337640_0	725,760	1,330,379	1.8331
24 May 2013 14:39:48	1213970	15686664	hadcm3n_u6sb_2020_40_008337640_0	699,840	1,285,451	1.8368
23 May 2013 09:23:19	1213970	15686664	hadcm3n_u6sb_2020_40_008337640_0	673,920	1,237,980	1.8370
22 May 2013 09:38:41	1213970	15686664	hadcm3n_u6sb_2020_40_008337640_0	648,000	1,187,792	1.8330
21 May 2013 01:15:11	1213970	15686664	hadcm3n_u6sb_2020_40_008337640_0	622,080	1,137,208	1.8281
17 May 2013 13:42:17	1213970	15686664	hadcm3n_u6sb_2020_40_008337640_0	596,160	1,089,437	1.8274
16 May 2013 07:51:59	1213970	15686664	hadcm3n_u6sb_2020_40_008337640_0	570,240	1,040,329	1.8244
15 May 2013 09:25:45	1213970	15686664	hadcm3n_u6sb_2020_40_008337640_0	544,320	992,482	1.8233
14 May 2013 09:44:07	1213970	15686664	hadcm3n_u6sb_2020_40_008337640_0	518,400	944,068	1.8211
13 May 2013 04:31:11	1213970	15686664	hadcm3n_u6sb_2020_40_008337640_0	492,480	895,839	1.8190
12 May 2013 04:59:44	1213970	15686664	hadcm3n_u6sb_2020_40_008337640_0	466,560	848,433	1.8185
10 May 2013 01:29:08	1213970	15686664	hadcm3n_u6sb_2020_40_008337640_0	440,640	801,276	1.8184
09 May 2013 01:34:00	1213970	15686664	hadcm3n_u6sb_2020_40_008337640_0	414,720	754,153	1.8185
08 May 2013 01:07:05	1213970	15686664	hadcm3n_u6sb_2020_40_008337640_0	388,800	705,224	1.8138
06 May 2013 10:39:01	1213970	15686664	hadcm3n_u6sb_2020_40_008337640_0	362,880	657,506	1.8119
05 May 2013 15:10:29	1213970	15686664	hadcm3n_u6sb_2020_40_008337640_0	336,960	609,436	1.8086
03 May 2013 13:55:11	1213970	15686664	hadcm3n_u6sb_2020_40_008337640_0	311,040	560,681	1.8026
02 May 2013 12:01:47	1213970	15686664	hadcm3n_u6sb_2020_40_008337640_0	285,120	514,442	1.8043