Task 16039287

Name	hadcm3n_oaeu_1900_40_008468617_0
Workunit	8619456
Created	27 Sep 2013, 9:41:11 UTC
Sent	5 Oct 2013, 2:02:06 UTC
Report deadline	4 Jan 2014, 9:29:17 UTC
Received	18 Nov 2013, 9:02:06 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1163325
Run time	9 days 14 hours 20 min 5 sec
CPU time	9 days 12 hours 39 min 36 sec
Validate state	Invalid
Credit	12,441.60
Device peak FLOPS	3.46 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.64</core_client_version> <![CDATA[ <message> (unknown error) - exit code 193 (0xc1) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 22:01:46 (5324): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:01:55 (5324): No heartbeat from core client for 30 sec - exiting 22:01:56 (5324): No heartbeat from core client for 30 sec - exiting 22:01:57 (5324): No heartbeat from core client for 30 sec - exiting 22:01:58 (5324): No heartbeat from core client for 30 sec - exiting 22:01:59 (5324): No heartbeat from core client for 30 sec - exiting 22:02:00 (5324): No heartbeat from core client for 30 sec - exiting 22:02:01 (5324): No heartbeat from core client for 30 sec - exiting 22:05:59 (5184): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:06:00 (5184): No heartbeat from core client for 30 sec - exiting 22:06:01 (5184): No heartbeat from core client for 30 sec - exiting 22:06:02 (5184): No heartbeat from core client for 30 sec - exiting 22:06:03 (5184): No heartbeat from core client for 30 sec - exiting 22:06:04 (5184): No heartbeat from core client for 30 sec - exiting 22:06:05 (5184): No heartbeat from core client for 30 sec - exiting 22:06:06 (5184): No heartbeat from core client for 30 sec - exiting 22:06:07 (5184): No heartbeat from core client for 30 sec - exiting 22:06:08 (5184): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 12:47:59 (7164): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:53:04 (848): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:53:24 (848): No heartbeat from core client for 30 sec - exiting 12:53:25 (848): No heartbeat from core client for 30 sec - exiting 12:53:26 (848): No heartbeat from core client for 30 sec - exiting Atmos Hold Restart file rename failed on atmos_restart.hold CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 18:53:27 (9672): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:53:34 (9672): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 09:38:13 (6484): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:38:23 (6484): No heartbeat from core client for 30 sec - exiting 09:38:28 (6484): No heartbeat from core client for 30 sec - exiting 09:38:29 (6484): No heartbeat from core client for 30 sec - exiting 09:38:30 (6484): No heartbeat from core client for 30 sec - exiting 09:38:31 (6484): No heartbeat from core client for 30 sec - exiting 09:39:19 (3820): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:39:20 (3820): No heartbeat from core client for 30 sec - exiting 09:39:21 (3820): No heartbeat from core client for 30 sec - exiting 09:43:23 (6964): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:43:25 (6964): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 11:30:51 (5208): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 13:48:53 (5016): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:49:08 (5016): No heartbeat from core client for 30 sec - exiting 13:49:09 (5016): No heartbeat from core client for 30 sec - exiting 13:49:10 (5016): No heartbeat from core client for 30 sec - exiting 13:51:01 (4188): No heartbeat from core client for 30 sec - exiting 13:51:07 (4188): No heartbeat from core client for 30 sec - exiting 13:51:08 (4188): No heartbeat from core client for 30 sec - exiting 13:51:09 (4188): No heartbeat from core client for 30 sec - exiting 13:51:10 (4188): No heartbeat from core client for 30 sec - exiting 13:52:11 (4188): No heartbeat from core client for 30 sec - exiting 13:52:12 (4188): No heartbeat from core client for 30 sec - exiting 13:52:13 (4188): No heartbeat from core client for 30 sec - exiting 13:52:14 (4188): No heartbeat from core client for 30 sec - exiting 13:52:15 (4188): No heartbeat from core client for 30 sec - exiting 13:52:16 (4188): No heartbeat from core client for 30 sec - exiting 13:52:17 (4188): No heartbeat from core client for 30 sec - exiting 13:52:18 (4188): No heartbeat from core client for 30 sec - exiting 13:52:19 (4188): No heartbeat from core client for 30 sec - exiting 13:52:20 (4188): No heartbeat from core client for 30 sec - exiting 13:52:21 (4188): No heartbeat from core client for 30 sec - exiting 13:52:22 (4188): No heartbeat from core client for 30 sec - exiting 13:52:23 (4188): No heartbeat from core client for 30 sec - exiting 13:52:24 (4188): No heartbeat from core client for 30 sec - exiting 13:52:25 (4188): No heartbeat from core client for 30 sec - exiting 13:52:26 (4188): No heartbeat from core client for 30 sec - exiting 13:52:27 (4188): No heartbeat from core client for 30 sec - exiting 13:52:28 (4188): No heartbeat from core client for 30 sec - exiting 13:52:29 (4188): No heartbeat from core client for 30 sec - exiting 13:52:30 (4188): No heartbeat from core client for 30 sec - exiting 13:52:31 (4188): No heartbeat from core client for 30 sec - exiting 13:52:32 (4188): No heartbeat from core client for 30 sec - exiting 13:52:33 (4188): No heartbeat from core client for 30 sec - exiting 13:52:34 (4188): No heartbeat from core client for 30 sec - exiting 13:52:35 (4188): No heartbeat from core client for 30 sec - exiting 13:52:36 (4188): No heartbeat from core client for 30 sec - exiting 13:52:37 (4188): No heartbeat from core client for 30 sec - exiting 13:52:38 (4188): No heartbeat from core client for 30 sec - exiting 13:52:39 (4188): No heartbeat from core client for 30 sec - exiting 13:52:40 (4188): No heartbeat from core client for 30 sec - exiting 13:52:41 (4188): No heartbeat from core client for 30 sec - exiting 13:52:42 (4188): No heartbeat from core client for 30 sec - exiting 13:52:43 (4188): No heartbeat from core client for 30 sec - exiting 13:52:44 (4188): No heartbeat from core client for 30 sec - exiting 13:52:45 (4188): No heartbeat from core client for 30 sec - exiting 13:52:46 (4188): No heartbeat from core client for 30 sec - exiting 13:52:47 (4188): No heartbeat from core client for 30 sec - exiting 13:52:48 (4188): No heartbeat from core client for 30 sec - exiting 13:52:49 (4188): No heartbeat from core client for 30 sec - exiting 13:52:50 (4188): No heartbeat from core client for 30 sec - exiting 13:52:51 (4188): No heartbeat from core client for 30 sec - exiting 13:52:52 (4188): No heartbeat from core client for 30 sec - exiting 13:52:53 (4188): No heartbeat from core client for 30 sec - exiting 13:52:54 (4188): No heartbeat from core client for 30 sec - exiting 13:52:55 (4188): No heartbeat from core client for 30 sec - exiting 13:52:56 (4188): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:53:58 (6508): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 21:18:49 (8400): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:19:05 (8400): No heartbeat from core client for 30 sec - exiting 21:19:06 (8400): No heartbeat from core client for 30 sec - exiting 21:19:07 (8400): No heartbeat from core client for 30 sec - exiting 21:19:08 (8400): No heartbeat from core client for 30 sec - exiting 21:19:09 (8400): No heartbeat from core client for 30 sec - exiting 21:19:10 (8400): No heartbeat from core client for 30 sec - exiting 21:19:11 (8400): No heartbeat from core client for 30 sec - exiting 21:19:12 (8400): No heartbeat from core client for 30 sec - exiting 21:20:01 (4104): No heartbeat from core client for 30 sec - exiting 21:20:02 (4104): No heartbeat from core client for 30 sec - exiting 21:20:03 (4104): No heartbeat from core client for 30 sec - exiting 21:20:04 (4104): No heartbeat from core client for 30 sec - exiting 21:20:05 (4104): No heartbeat from core client for 30 sec - exiting 21:20:06 (4104): No heartbeat from core client for 30 sec - exiting 21:20:07 (4104): No heartbeat from core client for 30 sec - exiting 21:20:10 (4104): No heartbeat from core client for 30 sec - exiting 21:20:11 (4104): No heartbeat from core client for 30 sec - exiting 21:20:12 (4104): No heartbeat from core client for 30 sec - exiting 21:20:13 (4104): No heartbeat from core client for 30 sec - exiting 21:20:14 (4104): No heartbeat from core client for 30 sec - exiting 21:20:15 (4104): No heartbeat from core client for 30 sec - exiting 21:20:16 (4104): No heartbeat from core client for 30 sec - exiting 21:20:17 (4104): No heartbeat from core client for 30 sec - exiting 21:20:18 (4104): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:20:20 (4104): No heartbeat from core client for 30 sec - exiting Atmos Hold Restart file rename failed on atmos_restart.hold 21:23:24 (9044): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:23:25 (9044): No heartbeat from core client for 30 sec - exiting 21:23:26 (9044): No heartbeat from core client for 30 sec - exiting 21:23:27 (9044): No heartbeat from core client for 30 sec - exiting 21:23:28 (9044): No heartbeat from core client for 30 sec - exiting 21:23:29 (9044): No heartbeat from core client for 30 sec - exiting 21:23:30 (9044): No heartbeat from core client for 30 sec - exiting 21:23:31 (9044): No heartbeat from core client for 30 sec - exiting 21:23:32 (9044): No heartbeat from core client for 30 sec - exiting 21:23:33 (9044): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6516, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6624, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6388, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x76EEFF6B write attempt to address 0xFFFFFFF8 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77CA7383 read attempt to address 0xFFFFFFF8 Engaging BOINC Windows Runtime Debugger... Cannot serialize file E:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_oaeu_1900_40_008468617/dataout/shmem_restart.day Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
13 Nov 2013 10:41:34	1163325	16039287	hadcm3n_oaeu_1900_40_008468617_0	1,036,800	822,809	0.7936
13 Nov 2013 01:09:35	1163325	16039287	hadcm3n_oaeu_1900_40_008468617_0	1,010,880	799,942	0.7913
12 Nov 2013 11:27:31	1163325	16039287	hadcm3n_oaeu_1900_40_008468617_0	984,960	780,915	0.7928
12 Nov 2013 06:11:12	1163325	16039287	hadcm3n_oaeu_1900_40_008468617_0	959,040	762,037	0.7946
12 Nov 2013 00:49:57	1163325	16039287	hadcm3n_oaeu_1900_40_008468617_0	933,120	742,810	0.7960
11 Nov 2013 03:25:29	1163325	16039287	hadcm3n_oaeu_1900_40_008468617_0	907,200	724,011	0.7981
09 Nov 2013 12:32:07	1163325	16039287	hadcm3n_oaeu_1900_40_008468617_0	881,280	703,482	0.7983
08 Nov 2013 06:36:42	1163325	16039287	hadcm3n_oaeu_1900_40_008468617_0	855,360	681,618	0.7969
06 Nov 2013 01:25:10	1163325	16039287	hadcm3n_oaeu_1900_40_008468617_0	829,440	659,669	0.7953
05 Nov 2013 07:17:51	1163325	16039287	hadcm3n_oaeu_1900_40_008468617_0	803,520	636,619	0.7923
05 Nov 2013 01:40:43	1163325	16039287	hadcm3n_oaeu_1900_40_008468617_0	777,600	616,661	0.7930
04 Nov 2013 08:54:26	1163325	16039287	hadcm3n_oaeu_1900_40_008468617_0	751,680	597,429	0.7948
01 Nov 2013 08:17:10	1163325	16039287	hadcm3n_oaeu_1900_40_008468617_0	725,760	577,594	0.7958
31 Oct 2013 11:09:40	1163325	16039287	hadcm3n_oaeu_1900_40_008468617_0	699,840	558,889	0.7986
31 Oct 2013 03:42:39	1163325	16039287	hadcm3n_oaeu_1900_40_008468617_0	673,920	538,632	0.7993
30 Oct 2013 13:30:04	1163325	16039287	hadcm3n_oaeu_1900_40_008468617_0	648,000	519,546	0.8018
30 Oct 2013 07:42:21	1163325	16039287	hadcm3n_oaeu_1900_40_008468617_0	622,080	499,082	0.8023
29 Oct 2013 23:15:43	1163325	16039287	hadcm3n_oaeu_1900_40_008468617_0	596,160	478,409	0.8025
29 Oct 2013 05:21:17	1163325	16039287	hadcm3n_oaeu_1900_40_008468617_0	570,240	458,835	0.8046
28 Oct 2013 09:56:27	1163325	16039287	hadcm3n_oaeu_1900_40_008468617_0	544,320	439,075	0.8066