Task 15880508

Name	hadcm3n_o3an_2020_40_008398095_1
Workunit	8548951
Created	4 Jul 2013, 8:11:01 UTC
Sent	4 Jul 2013, 8:24:04 UTC
Report deadline	3 Oct 2013, 15:51:15 UTC
Received	14 Nov 2013, 4:00:17 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1173744
Run time	8 days 0 hours 9 min 51 sec
CPU time	6 days 21 hours 22 min 16 sec
Validate state	Invalid
Credit	3,110.40
Device peak FLOPS	2.64 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.64</core_client_version> <![CDATA[ <message> (unknown error) - exit code 193 (0xc1) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 21:03:33 (1608): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 21:22:43 (1724): No heartbeat from core client for 30 sec - exiting 21:22:44 (1724): No heartbeat from core client for 30 sec - exiting 21:22:45 (1724): No heartbeat from core client for 30 sec - exiting 21:22:46 (1724): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 13:12:54 (2324): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 20:07:38 (2764): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 12:45:12 (3092): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 18:14:52 (2244): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 12:25:55 (2124): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 12:31:24 (1512): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 14:38:45 (2264): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 19:56:25 (3372): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:56:26 (3372): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 11:09:59 (2624): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 14:35:16 (2372): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 20:16:08 (2320): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 20:38:01 (2216): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 21:15:12 (3572): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 20:56:50 (520): No heartbeat from core client for 30 sec - exiting 20:56:52 (520): No heartbeat from core client for 30 sec - exiting 20:56:53 (520): No heartbeat from core client for 30 sec - exiting 20:56:54 (520): No heartbeat from core client for 30 sec - exiting 20:56:55 (520): No heartbeat from core client for 30 sec - exiting 20:56:56 (520): No heartbeat from core client for 30 sec - exiting 20:56:57 (520): No heartbeat from core client for 30 sec - exiting 20:56:58 (520): No heartbeat from core client for 30 sec - exiting 20:56:59 (520): No heartbeat from core client for 30 sec - exiting 20:57:00 (520): No heartbeat from core client for 30 sec - exiting 20:57:01 (520): No heartbeat from core client for 30 sec - exiting 20:57:03 (520): No heartbeat from core client for 30 sec - exiting 20:57:04 (520): No heartbeat from core client for 30 sec - exiting 20:57:05 (520): No heartbeat from core client for 30 sec - exiting 20:57:06 (520): No heartbeat from core client for 30 sec - exiting 20:57:07 (520): No heartbeat from core client for 30 sec - exiting 20:57:08 (520): No heartbeat from core client for 30 sec - exiting 20:57:09 (520): No heartbeat from core client for 30 sec - exiting 20:57:10 (520): No heartbeat from core client for 30 sec - exiting 20:57:11 (520): No heartbeat from core client for 30 sec - exiting 20:57:12 (520): No heartbeat from core client for 30 sec - exiting 20:57:13 (520): No heartbeat from core client for 30 sec - exiting 20:57:15 (520): No heartbeat from core client for 30 sec - exiting 20:57:16 (520): No heartbeat from core client for 30 sec - exiting 20:57:17 (520): No heartbeat from core client for 30 sec - exiting 20:57:18 (520): No heartbeat from core client for 30 sec - exiting 20:57:19 (520): No heartbeat from core client for 30 sec - exiting 20:57:20 (520): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 10:52:24 (4048): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 10:24:26 (1452): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3164, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3164, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3164, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 07:02:28 (3204): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 21:35:09 (3444): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 15:19:43 (356): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 21:14:33 (2940): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 13:26:48 (3904): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:26:49 (3904): No heartbeat from core client for 30 sec - exiting 13:26:50 (3904): No heartbeat from core client for 30 sec - exiting 13:26:51 (3904): No heartbeat from core client for 30 sec - exiting 13:26:52 (3904): No heartbeat from core client for 30 sec - exiting 13:26:53 (3904): No heartbeat from core client for 30 sec - exiting 13:26:54 (3904): No heartbeat from core client for 30 sec - exiting 13:26:55 (3904): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 12:49:14 (572): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 19:21:58 (3356): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 13:00:53 (2384): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 21:08:38 (3812): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 18:35:13 (196): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:35:14 (196): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 22:16:42 (876): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
14 Nov 2013 03:01:56	1173744	15880508	hadcm3n_o3an_2020_40_008398095_1	259,200	595,332	2.2968
19 Oct 2013 15:14:35	1173744	15880508	hadcm3n_o3an_2020_40_008398095_1	233,280	531,341	2.2777
21 Sep 2013 03:38:58	1173744	15880508	hadcm3n_o3an_2020_40_008398095_1	207,360	476,405	2.2975
02 Sep 2013 05:47:52	1173744	15880508	hadcm3n_o3an_2020_40_008398095_1	181,440	408,854	2.2534
17 Aug 2013 14:00:34	1173744	15880508	hadcm3n_o3an_2020_40_008398095_1	155,520	346,951	2.2309
17 Aug 2013 10:54:58	1173744	15880508	hadcm3n_o3an_2020_40_008398095_1	129,600	287,904	2.2215
24 Jul 2013 14:48:25	1173744	15880508	hadcm3n_o3an_2020_40_008398095_1	103,680	229,422	2.2128
23 Jul 2013 19:28:42	1173744	15880508	hadcm3n_o3an_2020_40_008398095_1	77,760	171,721	2.2083
23 Jul 2013 19:28:42	1173744	15880508	hadcm3n_o3an_2020_40_008398095_1	51,840	114,191	2.2028
11 Jul 2013 04:43:51	1173744	15880508	hadcm3n_o3an_2020_40_008398095_1	25,920	56,942	2.1968