Task 15924911

Name	hadcm3n_4h2q_2020_40_008363256_4
Workunit	8514115
Created	18 Aug 2013, 2:32:41 UTC
Sent	18 Aug 2013, 2:32:50 UTC
Report deadline	17 Nov 2013, 10:00:01 UTC
Received	30 Aug 2013, 16:49:20 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	-1073741819 (0xC0000005) STATUS_ACCESS_VIOLATION
Computer ID	1035913
Run time	9 days 16 hours 45 min 36 sec
CPU time	6 days 16 hours 56 min 40 sec
Validate state	Invalid
Credit	3,110.40
Device peak FLOPS	2.49 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.10.18</core_client_version> <![CDATA[ <message> - exit code -1073741819 (0xc0000005) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=10132, iMonCtr=1 Model crash detected, will try to restart... 03:49:26 (5640): No heartbeat from core client for 30 sec - exiting 03:49:27 (5640): No heartbeat from core client for 30 sec - exiting 03:49:29 (5640): No heartbeat from core client for 30 sec - exiting 03:49:30 (5640): No heartbeat from core client for 30 sec - exiting 03:49:31 (5640): No heartbeat from core client for 30 sec - exiting 03:49:32 (5640): No heartbeat from core client for 30 sec - exiting 03:49:33 (5640): No heartbeat from core client for 30 sec - exiting 03:49:34 (5640): No heartbeat from core client for 30 sec - exiting 03:49:35 (5640): No heartbeat from core client for 30 sec - exiting 03:49:36 (5640): No heartbeat from core client for 30 sec - exiting 03:49:37 (5640): No heartbeat from core client for 30 sec - exiting 03:49:38 (5640): No heartbeat from core client for 30 sec - exiting 03:49:39 (5640): No heartbeat from core client for 30 sec - exiting 03:49:40 (5640): No heartbeat from core client for 30 sec - exiting 03:49:41 (5640): No heartbeat from core client for 30 sec - exiting 03:49:42 (5640): No heartbeat from core client for 30 sec - exiting 03:49:43 (5640): No heartbeat from core client for 30 sec - exiting 03:49:44 (5640): No heartbeat from core client for 30 sec - exiting 03:49:45 (5640): No heartbeat from core client for 30 sec - exiting 03:49:46 (5640): No heartbeat from core client for 30 sec - exiting 03:49:47 (5640): No heartbeat from core client for 30 sec - exiting 03:49:48 (5640): No heartbeat from core client for 30 sec - exiting 03:49:49 (5640): No heartbeat from core client for 30 sec - exiting 03:49:50 (5640): No heartbeat from core client for 30 sec - exiting 03:49:51 (5640): No heartbeat from core client for 30 sec - exiting 03:49:52 (5640): No heartbeat from core client for 30 sec - exiting 03:49:53 (5640): No heartbeat from core client for 30 sec - exiting 03:49:54 (5640): No heartbeat from core client for 30 sec - exiting 03:49:56 (5640): No heartbeat from core client for 30 sec - exiting 03:49:57 (5640): No heartbeat from core client for 30 sec - exiting 03:49:58 (5640): No heartbeat from core client for 30 sec - exiting 03:49:59 (5640): No heartbeat from core client for 30 sec - exiting 03:50:01 (5640): No heartbeat from core client for 30 sec - exiting 03:50:02 (5640): No heartbeat from core client for 30 sec - exiting 03:50:03 (5640): No heartbeat from core client for 30 sec - exiting 03:50:04 (5640): No heartbeat from core client for 30 sec - exiting 03:50:05 (5640): No heartbeat from core client for 30 sec - exiting 03:50:06 (5640): No heartbeat from core client for 30 sec - exiting 03:50:07 (5640): No heartbeat from core client for 30 sec - exiting 03:50:08 (5640): No heartbeat from core client for 30 sec - exiting 03:50:09 (5640): No heartbeat from core client for 30 sec - exiting 03:50:10 (5640): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Skipping gmts_generator due to netcdf error 13 - Permission denied Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 12:56:19 (7036): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 14:23:09 (14188): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:33:10 (8864): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:43:11 (5724): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:03:14 (13628): No heartbeat from core client for 30 sec - exiting 15:03:15 (13628): No heartbeat from core client for 30 sec - exiting 15:03:16 (13628): No heartbeat from core client for 30 sec - exiting 15:03:17 (13628): No heartbeat from core client for 30 sec - exiting 15:03:18 (13628): No heartbeat from core client for 30 sec - exiting 15:03:19 (13628): No heartbeat from core client for 30 sec - exiting 15:03:20 (13628): No heartbeat from core client for 30 sec - exiting 15:03:21 (13628): No heartbeat from core client for 30 sec - exiting 15:03:22 (13628): No heartbeat from core client for 30 sec - exiting 15:03:23 (13628): No heartbeat from core client for 30 sec - exiting 15:03:25 (13628): No heartbeat from core client for 30 sec - exiting 15:03:26 (13628): No heartbeat from core client for 30 sec - exiting 15:03:27 (13628): No heartbeat from core client for 30 sec - exiting 15:03:28 (13628): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:23:14 (12792): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:28:14 (10368): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:33:16 (13020): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:43:16 (14148): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:53:18 (11024): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:58:20 (13072): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:08:20 (4656): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:23:22 (1672): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:28:23 (5700): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:38:25 (13408): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:43:25 (12180): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:58:26 (11060): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:03:27 (13000): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:16:14 (12660): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:23:29 (7520): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:28:29 (9832): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:43:31 (9848): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:53:32 (13236): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:58:35 (10020): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:03:33 (11788): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:08:33 (13492): No heartbeat from core client for 30 sec - exiting 18:08:34 (13492): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:23:36 (13264): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:43:39 (9468): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:48:38 (8316): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:03:41 (2300): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:13:42 (9316): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:18:49 (12016): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:38:50 (13868): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:48:44 (14004): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:03:46 (8784): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:13:47 (12344): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:23:57 (11500): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:33:50 (4004): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:41:02 (3064): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:48:51 (13684): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:53:51 (9976): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 21:58:58 (5996): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 21:58:59 (5996): No heartbeat from core client for 30 sec - exiting 21:59:01 (5996): No heartbeat from core client for 30 sec - exiting 21:59:02 (5996): No heartbeat from core client for 30 sec - exiting 21:59:03 (5996): No heartbeat from core client for 30 sec - exiting 21:59:04 (5996): No heartbeat from core client for 30 sec - exiting 21:59:05 (5996): No heartbeat from core client for 30 sec - exiting 21:59:06 (5996): No heartbeat from core client for 30 sec - exiting 21:59:07 (5996): No heartbeat from core client for 30 sec - exiting 21:59:08 (5996): No heartbeat from core client for 30 sec - exiting 21:59:10 (5996): No heartbeat from core client for 30 sec - exiting 21:59:11 (5996): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 05:21:38 (9324): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 12:27:25 (10656): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 03:27:49 (7468): No heartbeat from core client for 30 sec - exiting 03:27:50 (7468): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:27:51 (7468): No heartbeat from core client for 30 sec - exiting Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Skipping gmts_generator due to netcdf error 13 - Permission denied Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x777DFF6B write attempt to address 0xC24A4DFB Engaging BOINC Windows Runtime Debugger... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
30 Aug 2013 16:50:02	1035913	15924911	hadcm3n_4h2q_2020_40_008363256_4	259,200	579,390	2.2353
29 Aug 2013 22:16:35	1035913	15924911	hadcm3n_4h2q_2020_40_008363256_4	233,280	524,332	2.2477
28 Aug 2013 12:44:31	1035913	15924911	hadcm3n_4h2q_2020_40_008363256_4	207,360	468,265	2.2582
27 Aug 2013 02:39:25	1035913	15924911	hadcm3n_4h2q_2020_40_008363256_4	181,440	408,641	2.2522
25 Aug 2013 19:50:24	1035913	15924911	hadcm3n_4h2q_2020_40_008363256_4	155,520	350,012	2.2506
24 Aug 2013 13:19:16	1035913	15924911	hadcm3n_4h2q_2020_40_008363256_4	129,600	293,688	2.2661
23 Aug 2013 10:22:28	1035913	15924911	hadcm3n_4h2q_2020_40_008363256_4	103,680	236,374	2.2798
21 Aug 2013 16:08:24	1035913	15924911	hadcm3n_4h2q_2020_40_008363256_4	77,760	175,712	2.2597
20 Aug 2013 04:42:12	1035913	15924911	hadcm3n_4h2q_2020_40_008363256_4	51,840	118,951	2.2946
19 Aug 2013 05:49:01	1035913	15924911	hadcm3n_4h2q_2020_40_008363256_4	25,920	58,922	2.2732