Task 13498792

Name	hadcm3n_yfdd_1900_40_007352779_2
Workunit	7550209
Created	15 Oct 2011, 3:31:44 UTC
Sent	15 Oct 2011, 3:38:54 UTC
Report deadline	14 Jan 2012, 11:06:05 UTC
Received	23 Nov 2011, 1:34:04 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	-1073741819 (0xC0000005) STATUS_ACCESS_VIOLATION
Computer ID	1017410
Run time	5 days 8 hours 46 min 42 sec
CPU time	5 days 8 hours 46 min 42 sec
Validate state	Invalid
Credit	3,110.40
Device peak FLOPS	2.61 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.10.18</core_client_version> <![CDATA[ <message> - exit code -1073741819 (0xc0000005) </message> <stderr_txt> 20:39:41 (4112): No heartbeat from core client for 30 sec - exiting 20:39:42 (4112): No heartbeat from core client for 30 sec - exiting 20:39:43 (4112): No heartbeat from core client for 30 sec - exiting 20:39:44 (4112): No heartbeat from core client for 30 sec - exiting 20:39:45 (4112): No heartbeat from core client for 30 sec - exiting 20:39:46 (4112): No heartbeat from core client for 30 sec - exiting 20:39:48 (4112): No heartbeat from core client for 30 sec - exiting 20:39:49 (4112): No heartbeat from core client for 30 sec - exiting 20:39:50 (4112): No heartbeat from core client for 30 sec - exiting 20:39:51 (4112): No heartbeat from core client for 30 sec - exiting 20:39:52 (4112): No heartbeat from core client for 30 sec - exiting 20:39:53 (4112): No heartbeat from core client for 30 sec - exiting 20:39:54 (4112): No heartbeat from core client for 30 sec - exiting 20:39:55 (4112): No heartbeat from core client for 30 sec - exiting 20:39:56 (4112): No heartbeat from core client for 30 sec - exiting 20:39:57 (4112): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4616, iMonCtr=1 Model crash detected, will try to restart... 22:05:02 (5516): No heartbeat from core client for 30 sec - exiting 22:05:03 (5516): No heartbeat from core client for 30 sec - exiting 22:05:04 (5516): No heartbeat from core client for 30 sec - exiting 22:05:05 (5516): No heartbeat from core client for 30 sec - exiting 22:05:06 (5516): No heartbeat from core client for 30 sec - exiting 22:05:07 (5516): No heartbeat from core client for 30 sec - exiting 22:05:08 (5516): No heartbeat from core client for 30 sec - exiting 22:05:09 (5516): No heartbeat from core client for 30 sec - exiting 22:05:11 (5516): No heartbeat from core client for 30 sec - exiting 22:05:12 (5516): No heartbeat from core client for 30 sec - exiting 22:05:13 (5516): No heartbeat from core client for 30 sec - exiting 22:05:14 (5516): No heartbeat from core client for 30 sec - exiting 22:05:15 (5516): No heartbeat from core client for 30 sec - exiting 22:05:16 (5516): No heartbeat from core client for 30 sec - exiting 22:05:17 (5516): No heartbeat from core client for 30 sec - exiting 22:05:18 (5516): No heartbeat from core client for 30 sec - exiting 22:05:19 (5516): No heartbeat from core client for 30 sec - exiting 22:05:20 (5516): No heartbeat from core client for 30 sec - exiting 22:05:21 (5516): No heartbeat from core client for 30 sec - exiting 22:05:23 (5516): No heartbeat from core client for 30 sec - exiting 22:05:24 (5516): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4196, iMonCtr=1 Model crash detected, will try to restart... 21:50:12 (6056): No heartbeat from core client for 30 sec - exiting 21:50:13 (6056): No heartbeat from core client for 30 sec - exiting 21:50:14 (6056): No heartbeat from core client for 30 sec - exiting 21:50:15 (6056): No heartbeat from core client for 30 sec - exiting 21:50:16 (6056): No heartbeat from core client for 30 sec - exiting 21:50:17 (6056): No heartbeat from core client for 30 sec - exiting 21:50:18 (6056): No heartbeat from core client for 30 sec - exiting 21:50:19 (6056): No heartbeat from core client for 30 sec - exiting 21:50:20 (6056): No heartbeat from core client for 30 sec - exiting 21:50:21 (6056): No heartbeat from core client for 30 sec - exiting 21:50:22 (6056): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:12:47 (5536): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:22:55 (4612): No heartbeat from core client for 30 sec - exiting 06:22:56 (4612): No heartbeat from core client for 30 sec - exiting 06:22:57 (4612): No heartbeat from core client for 30 sec - exiting 06:22:58 (4612): No heartbeat from core client for 30 sec - exiting 06:22:59 (4612): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3300, iMonCtr=1 Model crash detected, will try to restart... 23:30:25 (3656): No heartbeat from core client for 30 sec - exiting 23:30:26 (3656): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:49:48 (4888): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:09:47 (2304): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6064, iMonCtr=1 Model crash detected, will try to restart... 22:52:29 (5100): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:32:51 (4356): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5524, iMonCtr=1 Model crash detected, will try to restart... 22:20:00 (4276): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:37:58 (4368): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:08:17 (5772): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:14:43 (3880): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:45:54 (5048): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:03:56 (5104): No heartbeat from core client for 30 sec - exiting 17:03:57 (5104): No heartbeat from core client for 30 sec - exiting 17:03:59 (5104): No heartbeat from core client for 30 sec - exiting 17:04:00 (5104): No heartbeat from core client for 30 sec - exiting 17:04:01 (5104): No heartbeat from core client for 30 sec - exiting 17:04:02 (5104): No heartbeat from core client for 30 sec - exiting 17:04:03 (5104): No heartbeat from core client for 30 sec - exiting 17:04:04 (5104): No heartbeat from core client for 30 sec - exiting 17:04:05 (5104): No heartbeat from core client for 30 sec - exiting 17:04:06 (5104): No heartbeat from core client for 30 sec - exiting 17:04:07 (5104): No heartbeat from core client for 30 sec - exiting 17:04:08 (5104): No heartbeat from core client for 30 sec - exiting 17:04:09 (5104): No heartbeat from core client for 30 sec - exiting 17:04:11 (5104): No heartbeat from core client for 30 sec - exiting 17:04:12 (5104): No heartbeat from core client for 30 sec - exiting 17:04:13 (5104): No heartbeat from core client for 30 sec - exiting 17:04:14 (5104): No heartbeat from core client for 30 sec - exiting 17:04:15 (5104): No heartbeat from core client for 30 sec - exiting 17:04:16 (5104): No heartbeat from core client for 30 sec - exiting 17:04:17 (5104): No heartbeat from core client for 30 sec - exiting 17:04:18 (5104): No heartbeat from core client for 30 sec - exiting 17:04:19 (5104): No heartbeat from core client for 30 sec - exiting 17:04:20 (5104): No heartbeat from core client for 30 sec - exiting 17:04:21 (5104): No heartbeat from core client for 30 sec - exiting 17:04:23 (5104): No heartbeat from core client for 30 sec - exiting 17:04:24 (5104): No heartbeat from core client for 30 sec - exiting 17:04:25 (5104): No heartbeat from core client for 30 sec - exiting 17:04:26 (5104): No heartbeat from core client for 30 sec - exiting 17:04:27 (5104): No heartbeat from core client for 30 sec - exiting 17:04:28 (5104): No heartbeat from core client for 30 sec - exiting 17:04:29 (5104): No heartbeat from core client for 30 sec - exiting 17:04:30 (5104): No heartbeat from core client for 30 sec - exiting 17:04:31 (5104): No heartbeat from core client for 30 sec - exiting 17:04:32 (5104): No heartbeat from core client for 30 sec - exiting 17:04:34 (5104): No heartbeat from core client for 30 sec - exiting 17:04:35 (5104): No heartbeat from core client for 30 sec - exiting 17:04:36 (5104): No heartbeat from core client for 30 sec - exiting 17:04:37 (5104): No heartbeat from core client for 30 sec - exiting 17:04:38 (5104): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:43:03 (4112): No heartbeat from core client for 30 sec - exiting 20:43:04 (4112): No heartbeat from core client for 30 sec - exiting 20:43:05 (4112): No heartbeat from core client for 30 sec - exiting 20:43:06 (4112): No heartbeat from core client for 30 sec - exiting 20:43:07 (4112): No heartbeat from core client for 30 sec - exiting 20:43:08 (4112): No heartbeat from core client for 30 sec - exiting 20:43:10 (4112): No heartbeat from core client for 30 sec - exiting 20:43:11 (4112): No heartbeat from core client for 30 sec - exiting 20:43:12 (4112): No heartbeat from core client for 30 sec - exiting 20:43:13 (4112): No heartbeat from core client for 30 sec - exiting 20:43:14 (4112): No heartbeat from core client for 30 sec - exiting 20:43:15 (4112): No heartbeat from core client for 30 sec - exiting 20:43:16 (4112): No heartbeat from core client for 30 sec - exiting 20:43:17 (4112): No heartbeat from core client for 30 sec - exiting 20:43:18 (4112): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:29:16 (1616): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:56:33 (4732): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:19:35 (4524): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:19:37 (4524): No heartbeat from core client for 30 sec - exiting 18:07:03 (7996): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:43:05 (4892): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:31:44 (4316): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:23:39 (4996): No heartbeat from core client for 30 sec - exiting 15:23:40 (4996): No heartbeat from core client for 30 sec - exiting 15:23:42 (4996): No heartbeat from core client for 30 sec - exiting 15:23:43 (4996): No heartbeat from core client for 30 sec - exiting 15:23:44 (4996): No heartbeat from core client for 30 sec - exiting 15:23:45 (4996): No heartbeat from core client for 30 sec - exiting 15:23:46 (4996): No heartbeat from core client for 30 sec - exiting 15:23:47 (4996): No heartbeat from core client for 30 sec - exiting 15:23:48 (4996): No heartbeat from core client for 30 sec - exiting 15:23:49 (4996): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:23:50 (4996): No heartbeat from core client for 30 sec - exiting 20:20:35 (5112): No heartbeat from core client for 30 sec - exiting 20:20:36 (5112): No heartbeat from core client for 30 sec - exiting 20:20:37 (5112): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:22:21 (2660): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:33:03 (4920): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:17:49 (4172): No heartbeat from core client for 30 sec - exiting 11:17:50 (4172): No heartbeat from core client for 30 sec - exiting 11:17:51 (4172): No heartbeat from core client for 30 sec - exiting 11:17:52 (4172): No heartbeat from core client for 30 sec - exiting 11:17:53 (4172): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x770EC3CB write attempt to address 0xC22E2155 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x76EF6B0D read attempt to address 0x00000004 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x775A6B0D read attempt to address 0x00000004 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x775E6B0D read attempt to address 0x00000004 Engaging BOINC Windows Runtime Debugger... 08:07:41 (6004): No heartbeat from core client for 30 sec - exiting 08:07:42 (6004): No heartbeat from core client for 30 sec - exiting Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x771E5F1B read attempt to address 0x00000000 Engaging BOINC Windows Runtime Debugger... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
20 Nov 2011 07:27:32	1017410	13498792	hadcm3n_yfdd_1900_40_007352779_2	259,200	458,532	1.7690
16 Nov 2011 19:28:54	1017410	13498792	hadcm3n_yfdd_1900_40_007352779_2	233,280	413,577	1.7729
16 Nov 2011 19:28:54	1017410	13498792	hadcm3n_yfdd_1900_40_007352779_2	207,360	368,814	1.7786
10 Nov 2011 07:58:26	1017410	13498792	hadcm3n_yfdd_1900_40_007352779_2	181,440	322,544	1.7777
05 Nov 2011 09:59:59	1017410	13498792	hadcm3n_yfdd_1900_40_007352779_2	155,520	277,372	1.7835
03 Nov 2011 02:21:45	1017410	13498792	hadcm3n_yfdd_1900_40_007352779_2	129,600	231,432	1.7857
31 Oct 2011 17:12:37	1017410	13498792	hadcm3n_yfdd_1900_40_007352779_2	103,680	182,483	1.7601
31 Oct 2011 15:19:40	1017410	13498792	hadcm3n_yfdd_1900_40_007352779_2	77,760	137,244	1.7650
31 Oct 2011 15:19:40	1017410	13498792	hadcm3n_yfdd_1900_40_007352779_2	51,840	91,637	1.7677
15 Oct 2011 23:59:20	1017410	13498792	hadcm3n_yfdd_1900_40_007352779_2	25,920	45,674	1.7621