Task 15960697

Name	hadcm3n_n5t1_1920_40_008377338_2
Workunit	8528197
Created	30 Aug 2013, 1:41:42 UTC
Sent	30 Aug 2013, 2:00:05 UTC
Report deadline	29 Nov 2013, 9:27:16 UTC
Received	28 Sep 2013, 20:55:53 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1276187
Run time	5 days 17 hours 22 min 45 sec
CPU time	5 days 10 hours 44 min 19 sec
Validate state	Invalid
Credit	3,110.40
Device peak FLOPS	2.92 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.64</core_client_version> <![CDATA[ <message> (unknown error) - exit code 193 (0xc1) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 10:29:10 (3992): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 11:05:56 (5788): No heartbeat from core client for 30 sec - exiting 11:05:57 (5788): No heartbeat from core client for 30 sec - exiting 11:05:58 (5788): No heartbeat from core client for 30 sec - exiting 11:05:59 (5788): No heartbeat from core client for 30 sec - exiting 11:06:00 (5788): No heartbeat from core client for 30 sec - exiting 11:06:01 (5788): No heartbeat from core client for 30 sec - exiting 11:06:02 (5788): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 17:58:23 (2648): No heartbeat from core client for 30 sec - exiting 17:58:24 (2648): No heartbeat from core client for 30 sec - exiting 17:58:25 (2648): No heartbeat from core client for 30 sec - exiting 17:58:26 (2648): No heartbeat from core client for 30 sec - exiting 17:58:27 (2648): No heartbeat from core client for 30 sec - exiting 17:58:28 (2648): No heartbeat from core client for 30 sec - exiting 17:58:30 (2648): No heartbeat from core client for 30 sec - exiting 17:58:31 (2648): No heartbeat from core client for 30 sec - exiting 17:58:32 (2648): No heartbeat from core client for 30 sec - exiting 17:58:33 (2648): No heartbeat from core client for 30 sec - exiting 17:58:34 (2648): No heartbeat from core client for 30 sec - exiting 17:58:35 (2648): No heartbeat from core client for 30 sec - exiting 17:58:36 (2648): No heartbeat from core client for 30 sec - exiting 17:58:37 (2648): No heartbeat from core client for 30 sec - exiting 17:58:38 (2648): No heartbeat from core client for 30 sec - exiting 17:58:39 (2648): No heartbeat from core client for 30 sec - exiting 17:58:40 (2648): No heartbeat from core client for 30 sec - exiting 17:58:41 (2648): No heartbeat from core client for 30 sec - exiting 17:58:42 (2648): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 15:21:06 (5164): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 07:36:07 (5444): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:36:08 (5444): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 19:30:16 (4800): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 19:50:30 (1228): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 17:31:04 (4848): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 20:30:53 (1740): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 18:11:17 (4872): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:11:56 (4756): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 18:23:50 (2620): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 07:00:48 (1508): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:01:23 (5496): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 09:56:00 (1740): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 16:53:56 (5224): No heartbeat from core client for 30 sec - exiting 16:53:57 (5224): No heartbeat from core client for 30 sec - exiting 16:53:58 (5224): No heartbeat from core client for 30 sec - exiting 16:53:59 (5224): No heartbeat from core client for 30 sec - exiting 16:54:00 (5224): No heartbeat from core client for 30 sec - exiting 16:54:01 (5224): No heartbeat from core client for 30 sec - exiting 16:54:02 (5224): No heartbeat from core client for 30 sec - exiting 16:54:03 (5224): No heartbeat from core client for 30 sec - exiting 16:54:04 (5224): No heartbeat from core client for 30 sec - exiting 16:54:05 (5224): No heartbeat from core client for 30 sec - exiting 16:54:06 (5224): No heartbeat from core client for 30 sec - exiting 16:54:07 (5224): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:54:08 (5224): No heartbeat from core client for 30 sec - exiting 16:54:50 (4232): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 22:24:19 (1384): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 23:26:34 (5840): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 09:11:47 (2604): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 18:37:53 (1324): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 19:09:25 (4536): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 20:48:45 (824): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:48:46 (824): No heartbeat from core client for 30 sec - exiting 20:48:47 (824): No heartbeat from core client for 30 sec - exiting 20:48:48 (824): No heartbeat from core client for 30 sec - exiting 20:48:49 (824): No heartbeat from core client for 30 sec - exiting 20:48:50 (824): No heartbeat from core client for 30 sec - exiting 20:48:51 (824): No heartbeat from core client for 30 sec - exiting 20:48:52 (824): No heartbeat from core client for 30 sec - exiting 20:48:53 (824): No heartbeat from core client for 30 sec - exiting 20:48:55 (824): No heartbeat from core client for 30 sec - exiting 20:48:56 (824): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 07:53:44 (4636): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:53:45 (4636): No heartbeat from core client for 30 sec - exiting 07:53:46 (4636): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 18:24:41 (5040): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:24:42 (5040): No heartbeat from core client for 30 sec - exiting 18:24:43 (5040): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 17:27:52 (1168): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:27:53 (1168): No heartbeat from core client for 30 sec - exiting 17:27:54 (1168): No heartbeat from core client for 30 sec - exiting 17:27:55 (1168): No heartbeat from core client for 30 sec - exiting 17:27:56 (1168): No heartbeat from core client for 30 sec - exiting 17:27:57 (1168): No heartbeat from core client for 30 sec - exiting 17:27:58 (1168): No heartbeat from core client for 30 sec - exiting 17:28:00 (1168): No heartbeat from core client for 30 sec - exiting 17:28:01 (1168): No heartbeat from core client for 30 sec - exiting 17:28:02 (1168): No heartbeat from core client for 30 sec - exiting 17:28:03 (1168): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 15:21:42 (5040): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:21:43 (5040): No heartbeat from core client for 30 sec - exiting 15:21:44 (5040): No heartbeat from core client for 30 sec - exiting 15:21:45 (5040): No heartbeat from core client for 30 sec - exiting 15:21:47 (5040): No heartbeat from core client for 30 sec - exiting 15:21:48 (5040): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 19:33:17 (1120): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77487373 read attempt to address 0xFFFFFFF8 Engaging BOINC Windows Runtime Debugger... Cannot serialize file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_n5t1_1920_40_008377338/dataout/shmem_restart.day Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
28 Sep 2013 21:00:05	1276187	15960697	hadcm3n_n5t1_1920_40_008377338_2	259,200	470,653	1.8158
28 Sep 2013 13:43:21	1276187	15960697	hadcm3n_n5t1_1920_40_008377338_2	233,280	423,399	1.8150
25 Sep 2013 09:26:55	1276187	15960697	hadcm3n_n5t1_1920_40_008377338_2	207,360	376,203	1.8143
25 Sep 2013 09:25:12	1276187	15960697	hadcm3n_n5t1_1920_40_008377338_2	181,440	328,972	1.8131
21 Sep 2013 15:37:16	1276187	15960697	hadcm3n_n5t1_1920_40_008377338_2	155,520	281,427	1.8096
16 Sep 2013 02:54:25	1276187	15960697	hadcm3n_n5t1_1920_40_008377338_2	129,600	233,644	1.8028
14 Sep 2013 22:01:15	1276187	15960697	hadcm3n_n5t1_1920_40_008377338_2	103,680	186,716	1.8009
11 Sep 2013 02:09:27	1276187	15960697	hadcm3n_n5t1_1920_40_008377338_2	77,760	140,578	1.8078
07 Sep 2013 03:27:47	1276187	15960697	hadcm3n_n5t1_1920_40_008377338_2	51,840	93,434	1.8024
02 Sep 2013 21:20:41	1276187	15960697	hadcm3n_n5t1_1920_40_008377338_2	25,920	47,305	1.8250