Task 13746431

Name	hadcm3n_t2d4_1940_40_007546758_4
Workunit	7743990
Created	7 Dec 2011, 15:17:06 UTC
Sent	7 Dec 2011, 18:40:27 UTC
Report deadline	8 Mar 2012, 2:07:38 UTC
Received	26 Dec 2011, 22:13:43 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1180674
Run time	7 days 14 hours 49 min 56 sec
CPU time	5 days 18 hours 5 min 14 sec
Validate state	Invalid
Credit	3,110.40
Device peak FLOPS	2.57 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.12.34</core_client_version> <![CDATA[ <message> - exit code 193 (0xc1) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 09:14:46 (5764): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 12:07:23 (5576): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6968, selfPID=6968, iMonCtr=1 BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/t2d4ko.pje1c10 Error converting file to netcdf: dataout/t2d4ko.pie1c10 Error converting file to netcdf: dataout/t2d4ko.pfe1c10 Error converting file to netcdf: dataout/t2d4ka.phe1c10 Error converting file to netcdf: dataout/t2d4ka.pge1c10 Error converting file to netcdf: dataout/t2d4ka.pee1c10 Error converting file to netcdf: dataout/t2d4ka.pde1c10 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 01:20:27 (6208): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:20:28 (6208): No heartbeat from core client for 30 sec - exiting 01:20:29 (6208): No heartbeat from core client for 30 sec - exiting 01:20:30 (6208): No heartbeat from core client for 30 sec - exiting 01:20:31 (6208): No heartbeat from core client for 30 sec - exiting 01:20:32 (6208): No heartbeat from core client for 30 sec - exiting 02:16:37 (5808): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:16:38 (5808): No heartbeat from core client for 30 sec - exiting 02:16:39 (5808): No heartbeat from core client for 30 sec - exiting 02:16:40 (5808): No heartbeat from core client for 30 sec - exiting 02:16:41 (5808): No heartbeat from core client for 30 sec - exiting 02:16:42 (5808): No heartbeat from core client for 30 sec - exiting 02:23:05 (6984): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:23:06 (6984): No heartbeat from core client for 30 sec - exiting 02:23:07 (6984): No heartbeat from core client for 30 sec - exiting 02:23:08 (6984): No heartbeat from core client for 30 sec - exiting 02:23:09 (6984): No heartbeat from core client for 30 sec - exiting 02:24:10 (8092): No heartbeat from core client for 30 sec - exiting 02:24:11 (8092): No heartbeat from core client for 30 sec - exiting 02:24:12 (8092): No heartbeat from core client for 30 sec - exiting 02:24:13 (8092): No heartbeat from core client for 30 sec - exiting 02:24:14 (8092): No heartbeat from core client for 30 sec - exiting 02:24:15 (8092): No heartbeat from core client for 30 sec - exiting 02:24:16 (8092): No heartbeat from core client for 30 sec - exiting 02:24:17 (8092): No heartbeat from core client for 30 sec - exiting 02:24:18 (8092): No heartbeat from core client for 30 sec - exiting 02:24:19 (8092): No heartbeat from core client for 30 sec - exiting 02:24:20 (8092): No heartbeat from core client for 30 sec - exiting 02:24:21 (8092): No heartbeat from core client for 30 sec - exiting 02:24:22 (8092): No heartbeat from core client for 30 sec - exiting 02:24:23 (8092): No heartbeat from core client for 30 sec - exiting 02:24:24 (8092): No heartbeat from core client for 30 sec - exiting 02:24:25 (8092): No heartbeat from core client for 30 sec - exiting 02:24:26 (8092): No heartbeat from core client for 30 sec - exiting 02:24:27 (8092): No heartbeat from core client for 30 sec - exiting 02:24:28 (8092): No heartbeat from core client for 30 sec - exiting 02:24:29 (8092): No heartbeat from core client for 30 sec - exiting 02:24:30 (8092): No heartbeat from core client for 30 sec - exiting 02:24:31 (8092): No heartbeat from core client for 30 sec - exiting 02:24:32 (8092): No heartbeat from core client for 30 sec - exiting 02:24:33 (8092): No heartbeat from core client for 30 sec - exiting 02:24:34 (8092): No heartbeat from core client for 30 sec - exiting 02:24:35 (8092): No heartbeat from core client for 30 sec - exiting 02:24:36 (8092): No heartbeat from core client for 30 sec - exiting 02:24:37 (8092): No heartbeat from core client for 30 sec - exiting 02:24:38 (8092): No heartbeat from core client for 30 sec - exiting 02:24:39 (8092): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:03:35 (5564): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:03:36 (5564): No heartbeat from core client for 30 sec - exiting 03:03:37 (5564): No heartbeat from core client for 30 sec - exiting 03:03:38 (5564): No heartbeat from core client for 30 sec - exiting 03:03:39 (5564): No heartbeat from core client for 30 sec - exiting 03:03:40 (5564): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 01:29:33 (5264): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 02:48:22 (5048): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:48:23 (5048): No heartbeat from core client for 30 sec - exiting 02:48:24 (5048): No heartbeat from core client for 30 sec - exiting 02:48:25 (5048): No heartbeat from core client for 30 sec - exiting 02:48:26 (5048): No heartbeat from core client for 30 sec - exiting 02:52:21 (5148): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:54:17 (5268): No heartbeat from core client for 30 sec - exiting 02:54:18 (5268): No heartbeat from core client for 30 sec - exiting 02:54:19 (5268): No heartbeat from core client for 30 sec - exiting 02:54:20 (5268): No heartbeat from core client for 30 sec - exiting 02:54:21 (5268): No heartbeat from core client for 30 sec - exiting 02:54:22 (5268): No heartbeat from core client for 30 sec - exiting 02:54:23 (5268): No heartbeat from core client for 30 sec - exiting 02:54:24 (5268): No heartbeat from core client for 30 sec - exiting 02:54:25 (5268): No heartbeat from core client for 30 sec - exiting 02:54:26 (5268): No heartbeat from core client for 30 sec - exiting 02:54:27 (5268): No heartbeat from core client for 30 sec - exiting 02:54:28 (5268): No heartbeat from core client for 30 sec - exiting 02:54:29 (5268): No heartbeat from core client for 30 sec - exiting 02:54:30 (5268): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:54:31 (5268): No heartbeat from core client for 30 sec - exiting 02:54:32 (5268): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 02:26:32 (1376): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:26:33 (1376): No heartbeat from core client for 30 sec - exiting 02:26:34 (1376): No heartbeat from core client for 30 sec - exiting 02:26:35 (1376): No heartbeat from core client for 30 sec - exiting 02:26:36 (1376): No heartbeat from core client for 30 sec - exiting 02:26:37 (1376): No heartbeat from core client for 30 sec - exiting 02:26:38 (1376): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 00:01:26 (6000): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7632, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 01:11:04 (7088): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:11:05 (7088): No heartbeat from core client for 30 sec - exiting 01:11:06 (7088): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 11:50:10 (5336): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
26 Dec 2011 21:14:16	1180674	13746431	hadcm3n_t2d4_1940_40_007546758_4	259,200	497,110	1.9179
24 Dec 2011 10:20:13	1180674	13746431	hadcm3n_t2d4_1940_40_007546758_4	233,280	449,505	1.9269
22 Dec 2011 19:09:53	1180674	13746431	hadcm3n_t2d4_1940_40_007546758_4	207,360	399,899	1.9285
21 Dec 2011 16:59:59	1180674	13746431	hadcm3n_t2d4_1940_40_007546758_4	181,440	351,648	1.9381
20 Dec 2011 14:25:42	1180674	13746431	hadcm3n_t2d4_1940_40_007546758_4	155,520	302,916	1.9478
19 Dec 2011 09:27:50	1180674	13746431	hadcm3n_t2d4_1940_40_007546758_4	129,600	252,747	1.9502
18 Dec 2011 03:16:29	1180674	13746431	hadcm3n_t2d4_1940_40_007546758_4	103,680	203,772	1.9654
15 Dec 2011 18:09:56	1180674	13746431	hadcm3n_t2d4_1940_40_007546758_4	77,760	152,328	1.9590
13 Dec 2011 17:28:51	1180674	13746431	hadcm3n_t2d4_1940_40_007546758_4	51,840	101,213	1.9524
09 Dec 2011 13:53:53	1180674	13746431	hadcm3n_t2d4_1940_40_007546758_4	25,920	52,027	2.0072