Task 13552049

Name	hadcm3n_yfgg_1900_40_007523177_1
Workunit	7720652
Created	28 Oct 2011, 13:21:30 UTC
Sent	31 Oct 2011, 17:29:33 UTC
Report deadline	31 Jan 2012, 0:56:44 UTC
Received	15 Nov 2011, 18:29:22 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	-1073741819 (0xC0000005) STATUS_ACCESS_VIOLATION
Computer ID	1173373
Run time	6 days 9 hours 29 min 12 sec
CPU time	6 days 1 hours 51 min 27 sec
Validate state	Invalid
Credit	3,110.40
Device peak FLOPS	3.79 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.12.34</core_client_version> <![CDATA[ <message> - exit code -1073741819 (0xc0000005) </message> <stderr_txt> 06:40:08 (1208): No heartbeat from core client for 30 sec - exiting 06:40:09 (1208): No heartbeat from core client for 30 sec - exiting 06:40:10 (1208): No heartbeat from core client for 30 sec - exiting 06:40:11 (1208): No heartbeat from core client for 30 sec - exiting 06:40:12 (1208): No heartbeat from core client for 30 sec - exiting 06:40:13 (1208): No heartbeat from core client for 30 sec - exiting 06:40:14 (1208): No heartbeat from core client for 30 sec - exiting 06:40:15 (1208): No heartbeat from core client for 30 sec - exiting 06:40:16 (1208): No heartbeat from core client for 30 sec - exiting 06:40:17 (1208): No heartbeat from core client for 30 sec - exiting 06:40:18 (1208): No heartbeat from core client for 30 sec - exiting 06:40:19 (1208): No heartbeat from core client for 30 sec - exiting 06:40:20 (1208): No heartbeat from core client for 30 sec - exiting 06:40:21 (1208): No heartbeat from core client for 30 sec - exiting 06:40:22 (1208): No heartbeat from core client for 30 sec - exiting 06:40:23 (1208): No heartbeat from core client for 30 sec - exiting 06:40:24 (1208): No heartbeat from core client for 30 sec - exiting 06:40:25 (1208): No heartbeat from core client for 30 sec - exiting 06:40:26 (1208): No heartbeat from core client for 30 sec - exiting 06:40:27 (1208): No heartbeat from core client for 30 sec - exiting 06:40:28 (1208): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:40:29 (1208): No heartbeat from core client for 30 sec - exiting Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5060, iMonCtr=1 Model crash detected, will try to restart... 06:30:55 (3840): No heartbeat from core client for 30 sec - exiting 06:30:56 (3840): No heartbeat from core client for 30 sec - exiting 06:30:57 (3840): No heartbeat from core client for 30 sec - exiting 06:30:58 (3840): No heartbeat from core client for 30 sec - exiting 06:30:59 (3840): No heartbeat from core client for 30 sec - exiting 06:31:01 (3840): No heartbeat from core client for 30 sec - exiting 06:31:02 (3840): No heartbeat from core client for 30 sec - exiting 06:31:03 (3840): No heartbeat from core client for 30 sec - exiting 06:31:04 (3840): No heartbeat from core client for 30 sec - exiting 06:31:05 (3840): No heartbeat from core client for 30 sec - exiting 06:31:06 (3840): No heartbeat from core client for 30 sec - exiting 06:31:07 (3840): No heartbeat from core client for 30 sec - exiting 06:31:08 (3840): No heartbeat from core client for 30 sec - exiting 06:31:09 (3840): No heartbeat from core client for 30 sec - exiting 06:31:10 (3840): No heartbeat from core client for 30 sec - exiting 06:31:11 (3840): No heartbeat from core client for 30 sec - exiting 06:31:12 (3840): No heartbeat from core client for 30 sec - exiting 06:31:13 (3840): No heartbeat from core client for 30 sec - exiting 06:31:14 (3840): No heartbeat from core client for 30 sec - exiting 06:31:15 (3840): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 09:16:13 (3308): No heartbeat from core client for 30 sec - exiting 09:16:14 (3308): No heartbeat from core client for 30 sec - exiting 09:16:15 (3308): No heartbeat from core client for 30 sec - exiting 09:16:16 (3308): No heartbeat from core client for 30 sec - exiting 09:16:17 (3308): No heartbeat from core client for 30 sec - exiting 09:16:18 (3308): No heartbeat from core client for 30 sec - exiting 09:16:19 (3308): No heartbeat from core client for 30 sec - exiting 09:16:20 (3308): No heartbeat from core client for 30 sec - exiting 09:16:21 (3308): No heartbeat from core client for 30 sec - exiting 09:16:23 (3308): No heartbeat from core client for 30 sec - exiting 09:16:24 (3308): No heartbeat from core client for 30 sec - exiting 09:16:25 (3308): No heartbeat from core client for 30 sec - exiting 09:16:26 (3308): No heartbeat from core client for 30 sec - exiting 09:16:27 (3308): No heartbeat from core client for 30 sec - exiting 09:16:28 (3308): No heartbeat from core client for 30 sec - exiting 09:16:29 (3308): No heartbeat from core client for 30 sec - exiting 09:16:30 (3308): No heartbeat from core client for 30 sec - exiting 09:16:31 (3308): No heartbeat from core client for 30 sec - exiting 09:16:32 (3308): No heartbeat from core client for 30 sec - exiting 09:16:33 (3308): No heartbeat from core client for 30 sec - exiting 09:16:35 (3308): No heartbeat from core client for 30 sec - exiting 09:16:36 (3308): No heartbeat from core client for 30 sec - exiting 09:16:37 (3308): No heartbeat from core client for 30 sec - exiting 09:16:38 (3308): No heartbeat from core client for 30 sec - exiting 09:16:39 (3308): No heartbeat from core client for 30 sec - exiting 09:16:40 (3308): No heartbeat from core client for 30 sec - exiting 09:16:41 (3308): No heartbeat from core client for 30 sec - exiting 09:16:42 (3308): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3652, iMonCtr=1 Model crash detected, will try to restart... 09:53:59 (4288): No heartbeat from core client for 30 sec - exiting 09:54:00 (4288): No heartbeat from core client for 30 sec - exiting 09:54:01 (4288): No heartbeat from core client for 30 sec - exiting 09:54:03 (4288): No heartbeat from core client for 30 sec - exiting 09:54:04 (4288): No heartbeat from core client for 30 sec - exiting 09:54:05 (4288): No heartbeat from core client for 30 sec - exiting 09:54:06 (4288): No heartbeat from core client for 30 sec - exiting 09:54:07 (4288): No heartbeat from core client for 30 sec - exiting 09:54:08 (4288): No heartbeat from core client for 30 sec - exiting 09:54:09 (4288): No heartbeat from core client for 30 sec - exiting 09:54:10 (4288): No heartbeat from core client for 30 sec - exiting 09:54:11 (4288): No heartbeat from core client for 30 sec - exiting 09:54:12 (4288): No heartbeat from core client for 30 sec - exiting 09:54:13 (4288): No heartbeat from core client for 30 sec - exiting 09:54:15 (4288): No heartbeat from core client for 30 sec - exiting 09:54:16 (4288): No heartbeat from core client for 30 sec - exiting 09:54:17 (4288): No heartbeat from core client for 30 sec - exiting 09:54:18 (4288): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:54:19 (4288): No heartbeat from core client for 30 sec - exiting 09:54:20 (4288): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 14:03:30 (4964): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:03:31 (4964): No heartbeat from core client for 30 sec - exiting Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1940, iMonCtr=1 Model crash detected, will try to restart... 08:36:45 (3460): No heartbeat from core client for 30 sec - exiting 08:36:46 (3460): No heartbeat from core client for 30 sec - exiting 08:36:47 (3460): No heartbeat from core client for 30 sec - exiting 08:36:48 (3460): No heartbeat from core client for 30 sec - exiting 08:36:50 (3460): No heartbeat from core client for 30 sec - exiting 08:36:51 (3460): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:36:52 (3460): No heartbeat from core client for 30 sec - exiting 08:36:53 (3460): No heartbeat from core client for 30 sec - exiting 08:36:54 (3460): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 06:29:15 (2808): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:29:16 (2808): No heartbeat from core client for 30 sec - exiting 06:29:17 (2808): No heartbeat from core client for 30 sec - exiting 06:29:18 (2808): No heartbeat from core client for 30 sec - exiting 06:49:13 (3800): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 06:24:07 (3924): No heartbeat from core client for 30 sec - exiting 06:24:08 (3924): No heartbeat from core client for 30 sec - exiting 06:24:09 (3924): No heartbeat from core client for 30 sec - exiting 06:24:10 (3924): No heartbeat from core client for 30 sec - exiting 06:24:11 (3924): No heartbeat from core client for 30 sec - exiting 06:24:12 (3924): No heartbeat from core client for 30 sec - exiting 06:24:13 (3924): No heartbeat from core client for 30 sec - exiting 06:24:15 (3924): No heartbeat from core client for 30 sec - exiting 06:24:16 (3924): No heartbeat from core client for 30 sec - exiting 06:24:17 (3924): No heartbeat from core client for 30 sec - exiting 06:24:18 (3924): No heartbeat from core client for 30 sec - exiting 06:24:19 (3924): No heartbeat from core client for 30 sec - exiting 06:24:20 (3924): No heartbeat from core client for 30 sec - exiting 06:24:21 (3924): No heartbeat from core client for 30 sec - exiting 06:24:22 (3924): No heartbeat from core client for 30 sec - exiting 06:24:23 (3924): No heartbeat from core client for 30 sec - exiting 06:24:24 (3924): No heartbeat from core client for 30 sec - exiting 06:24:25 (3924): No heartbeat from core client for 30 sec - exiting 06:24:27 (3924): No heartbeat from core client for 30 sec - exiting 06:24:28 (3924): No heartbeat from core client for 30 sec - exiting 06:24:29 (3924): No heartbeat from core client for 30 sec - exiting 06:24:30 (3924): No heartbeat from core client for 30 sec - exiting 06:24:31 (3924): No heartbeat from core client for 30 sec - exiting 06:24:32 (3924): No heartbeat from core client for 30 sec - exiting 06:24:33 (3924): No heartbeat from core client for 30 sec - exiting 06:24:34 (3924): No heartbeat from core client for 30 sec - exiting 06:24:35 (3924): No heartbeat from core client for 30 sec - exiting 06:24:36 (3924): No heartbeat from core client for 30 sec - exiting 06:24:37 (3924): No heartbeat from core client for 30 sec - exiting 06:24:39 (3924): No heartbeat from core client for 30 sec - exiting 06:24:40 (3924): No heartbeat from core client for 30 sec - exiting 06:24:41 (3924): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:24:42 (3924): No heartbeat from core client for 30 sec - exiting 06:40:36 (3852): No heartbeat from core client for 30 sec - exiting 06:40:37 (3852): No heartbeat from core client for 30 sec - exiting 06:40:38 (3852): No heartbeat from core client for 30 sec - exiting 06:40:39 (3852): No heartbeat from core client for 30 sec - exiting 06:40:40 (3852): No heartbeat from core client for 30 sec - exiting 06:40:41 (3852): No heartbeat from core client for 30 sec - exiting 06:40:42 (3852): No heartbeat from core client for 30 sec - exiting 06:40:43 (3852): No heartbeat from core client for 30 sec - exiting 06:40:44 (3852): No heartbeat from core client for 30 sec - exiting 06:40:45 (3852): No heartbeat from core client for 30 sec - exiting 06:40:46 (3852): No heartbeat from core client for 30 sec - exiting 06:40:47 (3852): No heartbeat from core client for 30 sec - exiting 06:40:48 (3852): No heartbeat from core client for 30 sec - exiting 06:40:49 (3852): No heartbeat from core client for 30 sec - exiting 06:40:50 (3852): No heartbeat from core client for 30 sec - exiting 06:40:51 (3852): No heartbeat from core client for 30 sec - exiting 06:40:53 (3852): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:40:54 (3852): No heartbeat from core client for 30 sec - exiting 06:40:55 (3852): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x772871D3 read attempt to address 0xFFFFFFF8 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77923A93 read attempt to address 0x00000000 Engaging BOINC Windows Runtime Debugger... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
15 Nov 2011 18:32:45	1173373	13552049	hadcm3n_yfgg_1900_40_007523177_1	259,200	505,367	1.9497
09 Nov 2011 20:21:23	1173373	13552049	hadcm3n_yfgg_1900_40_007523177_1	233,280	454,476	1.9482
08 Nov 2011 20:57:40	1173373	13552049	hadcm3n_yfgg_1900_40_007523177_1	207,360	404,609	1.9512
07 Nov 2011 20:14:18	1173373	13552049	hadcm3n_yfgg_1900_40_007523177_1	181,440	351,790	1.9389
06 Nov 2011 19:28:22	1173373	13552049	hadcm3n_yfgg_1900_40_007523177_1	155,520	301,314	1.9375
05 Nov 2011 16:00:34	1173373	13552049	hadcm3n_yfgg_1900_40_007523177_1	129,600	251,574	1.9412
04 Nov 2011 15:21:50	1173373	13552049	hadcm3n_yfgg_1900_40_007523177_1	103,680	202,404	1.9522
03 Nov 2011 14:48:38	1173373	13552049	hadcm3n_yfgg_1900_40_007523177_1	77,760	152,038	1.9552
02 Nov 2011 14:38:30	1173373	13552049	hadcm3n_yfgg_1900_40_007523177_1	51,840	102,555	1.9783
01 Nov 2011 14:57:55	1173373	13552049	hadcm3n_yfgg_1900_40_007523177_1	25,920	50,798	1.9598