Task 13779552

Name	hadcm3n_yci9_1940_40_007545614_4
Workunit	7742846
Created	14 Dec 2011, 19:00:10 UTC
Sent	14 Dec 2011, 19:03:23 UTC
Report deadline	15 Mar 2012, 2:30:34 UTC
Received	12 Jan 2012, 8:31:08 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	-226 (0xFFFFFF1E) ERR_TOO_MANY_EXITS
Computer ID	1137123
Run time	3 days 9 hours 49 min 27 sec
CPU time	3 days 1 hours 0 min 7 sec
Validate state	Invalid
Credit	1,555.20
Device peak FLOPS	2.62 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.12.34</core_client_version> <![CDATA[ <message> too many exit(0)s </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 08:40:29 (14400): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7012, iMonCtr=1 Model crash detected, will try to restart... 09:35:23 (5864): No heartbeat from core client for 30 sec - exiting 09:35:25 (5864): No heartbeat from core client for 30 sec - exiting 09:35:26 (5864): No heartbeat from core client for 30 sec - exiting 09:35:27 (5864): No heartbeat from core client for 30 sec - exiting 09:35:28 (5864): No heartbeat from core client for 30 sec - exiting 09:35:29 (5864): No heartbeat from core client for 30 sec - exiting 09:35:31 (5864): No heartbeat from core client for 30 sec - exiting 09:35:32 (5864): No heartbeat from core client for 30 sec - exiting 09:35:33 (5864): No heartbeat from core client for 30 sec - exiting 09:35:34 (5864): No heartbeat from core client for 30 sec - exiting 09:35:35 (5864): No heartbeat from core client for 30 sec - exiting 09:35:36 (5864): No heartbeat from core client for 30 sec - exiting 09:35:37 (5864): No heartbeat from core client for 30 sec - exiting 09:35:38 (5864): No heartbeat from core client for 30 sec - exiting 09:35:39 (5864): No heartbeat from core client for 30 sec - exiting 09:35:40 (5864): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 09:30:33 (8484): No heartbeat from core client for 30 sec - exiting 09:30:35 (8484): No heartbeat from core client for 30 sec - exiting 09:30:36 (8484): No heartbeat from core client for 30 sec - exiting 09:30:37 (8484): No heartbeat from core client for 30 sec - exiting 09:30:38 (8484): No heartbeat from core client for 30 sec - exiting 09:30:39 (8484): No heartbeat from core client for 30 sec - exiting 09:30:40 (8484): No heartbeat from core client for 30 sec - exiting 09:30:41 (8484): No heartbeat from core client for 30 sec - exiting 09:30:42 (8484): No heartbeat from core client for 30 sec - exiting 09:30:43 (8484): No heartbeat from core client for 30 sec - exiting 09:30:44 (8484): No heartbeat from core client for 30 sec - exiting 09:30:45 (8484): No heartbeat from core client for 30 sec - exiting 09:30:46 (8484): No heartbeat from core client for 30 sec - exiting 09:30:47 (8484): No heartbeat from core client for 30 sec - exiting 09:30:48 (8484): No heartbeat from core client for 30 sec - exiting 09:30:49 (8484): No heartbeat from core client for 30 sec - exiting 09:30:50 (8484): No heartbeat from core client for 30 sec - exiting 09:30:51 (8484): No heartbeat from core client for 30 sec - exiting 09:30:52 (8484): No heartbeat from core client for 30 sec - exiting 09:30:53 (8484): No heartbeat from core client for 30 sec - exiting 09:30:54 (8484): No heartbeat from core client for 30 sec - exiting 09:30:55 (8484): No heartbeat from core client for 30 sec - exiting 09:30:56 (8484): No heartbeat from core client for 30 sec - exiting 09:30:57 (8484): No heartbeat from core client for 30 sec - exiting 09:30:58 (8484): No heartbeat from core client for 30 sec - exiting 09:30:59 (8484): No heartbeat from core client for 30 sec - exiting 09:31:00 (8484): No heartbeat from core client for 30 sec - exiting 09:31:01 (8484): No heartbeat from core client for 30 sec - exiting 09:31:02 (8484): No heartbeat from core client for 30 sec - exiting 09:31:03 (8484): No heartbeat from core client for 30 sec - exiting 09:31:04 (8484): No heartbeat from core client for 30 sec - exiting 09:31:05 (8484): No heartbeat from core client for 30 sec - exiting 09:31:06 (8484): No heartbeat from core client for 30 sec - exiting 09:31:07 (8484): No heartbeat from core client for 30 sec - exiting 09:31:08 (8484): No heartbeat from core client for 30 sec - exiting 09:31:09 (8484): No heartbeat from core client for 30 sec - exiting 09:31:10 (8484): No heartbeat from core client for 30 sec - exiting 09:31:11 (8484): No heartbeat from core client for 30 sec - exiting 09:31:12 (8484): No heartbeat from core client for 30 sec - exiting 09:31:13 (8484): No heartbeat from core client for 30 sec - exiting 09:31:14 (8484): No heartbeat from core client for 30 sec - exiting 09:31:15 (8484): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:31:16 (8484): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 09:27:47 (17104): No heartbeat from core client for 30 sec - exiting 09:27:48 (17104): No heartbeat from core client for 30 sec - exiting 09:27:49 (17104): No heartbeat from core client for 30 sec - exiting 09:27:50 (17104): No heartbeat from core client for 30 sec - exiting 09:27:51 (17104): No heartbeat from core client for 30 sec - exiting 09:27:52 (17104): No heartbeat from core client for 30 sec - exiting 09:27:53 (17104): No heartbeat from core client for 30 sec - exiting 09:27:54 (17104): No heartbeat from core client for 30 sec - exiting 09:27:55 (17104): No heartbeat from core client for 30 sec - exiting 09:27:56 (17104): No heartbeat from core client for 30 sec - exiting 09:27:57 (17104): No heartbeat from core client for 30 sec - exiting 09:27:58 (17104): No heartbeat from core client for 30 sec - exiting 09:27:59 (17104): No heartbeat from core client for 30 sec - exiting 09:28:00 (17104): No heartbeat from core client for 30 sec - exiting 09:28:01 (17104): No heartbeat from core client for 30 sec - exiting 09:28:02 (17104): No heartbeat from core client for 30 sec - exiting 09:28:03 (17104): No heartbeat from core client for 30 sec - exiting 09:28:04 (17104): No heartbeat from core client for 30 sec - exiting 09:28:05 (17104): No heartbeat from core client for 30 sec - exiting 09:28:06 (17104): No heartbeat from core client for 30 sec - exiting 09:28:07 (17104): No heartbeat from core client for 30 sec - exiting 09:28:08 (17104): No heartbeat from core client for 30 sec - exiting 09:28:09 (17104): No heartbeat from core client for 30 sec - exiting 09:28:10 (17104): No heartbeat from core client for 30 sec - exiting 09:28:11 (17104): No heartbeat from core client for 30 sec - exiting 09:28:12 (17104): No heartbeat from core client for 30 sec - exiting 09:28:13 (17104): No heartbeat from core client for 30 sec - exiting 09:28:14 (17104): No heartbeat from core client for 30 sec - exiting 09:28:15 (17104): No heartbeat from core client for 30 sec - exiting 09:28:16 (17104): No heartbeat from core client for 30 sec - exiting 09:28:17 (17104): No heartbeat from core client for 30 sec - exiting 09:28:18 (17104): No heartbeat from core client for 30 sec - exiting 09:28:19 (17104): No heartbeat from core client for 30 sec - exiting 09:28:20 (17104): No heartbeat from core client for 30 sec - exiting 09:28:21 (17104): No heartbeat from core client for 30 sec - exiting 09:28:23 (17104): No heartbeat from core client for 30 sec - exiting 09:28:24 (17104): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:28:25 (17104): No heartbeat from core client for 30 sec - exiting 09:29:24 (11416): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 08:03:21 (16968): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 09:27:50 (4908): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 10:47:17 (1528): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 10:47:18 (1528): No heartbeat from core client for 30 sec - exiting 10:47:19 (1528): No heartbeat from core client for 30 sec - exiting 10:47:20 (1528): No heartbeat from core client for 30 sec - exiting 10:47:21 (1528): No heartbeat from core client for 30 sec - exiting 10:47:22 (1528): No heartbeat from core client for 30 sec - exiting 10:47:23 (1528): No heartbeat from core client for 30 sec - exiting 10:47:24 (1528): No heartbeat from core client for 30 sec - exiting 10:47:25 (1528): No heartbeat from core client for 30 sec - exiting 10:47:26 (1528): No heartbeat from core client for 30 sec - exiting 10:48:54 (16204): No heartbeat from core client for 30 sec - exiting 10:48:55 (16204): No heartbeat from core client for 30 sec - exiting 10:48:56 (16204): No heartbeat from core client for 30 sec - exiting 10:48:57 (16204): No heartbeat from core client for 30 sec - exiting 10:48:58 (16204): No heartbeat from core client for 30 sec - exiting 10:48:59 (16204): No heartbeat from core client for 30 sec - exiting 10:49:00 (16204): No heartbeat from core client for 30 sec - exiting 10:49:01 (16204): No heartbeat from core client for 30 sec - exiting 10:49:02 (16204): No heartbeat from core client for 30 sec - exiting 10:49:03 (16204): No heartbeat from core client for 30 sec - exiting 10:49:04 (16204): No heartbeat from core client for 30 sec - exiting 10:49:05 (16204): No heartbeat from core client for 30 sec - exiting 10:49:06 (16204): No heartbeat from core client for 30 sec - exiting 10:49:07 (16204): No heartbeat from core client for 30 sec - exiting 10:49:08 (16204): No heartbeat from core client for 30 sec - exiting 10:49:09 (16204): No heartbeat from core client for 30 sec - exiting 10:49:10 (16204): No heartbeat from core client for 30 sec - exiting 10:49:11 (16204): No heartbeat from core client for 30 sec - exiting 10:49:12 (16204): No heartbeat from core client for 30 sec - exiting 10:49:13 (16204): No heartbeat from core client for 30 sec - exiting 10:49:14 (16204): No heartbeat from core client for 30 sec - exiting 10:49:15 (16204): No heartbeat from core client for 30 sec - exiting 10:49:16 (16204): No heartbeat from core client for 30 sec - exiting 10:49:17 (16204): No heartbeat from core client for 30 sec - exiting 10:49:18 (16204): No heartbeat from core client for 30 sec - exiting 10:49:19 (16204): No heartbeat from core client for 30 sec - exiting 10:49:20 (16204): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 10:55:12 (1628): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:55:13 (1628): No heartbeat from core client for 30 sec - exiting 10:55:59 (17036): No heartbeat from core client for 30 sec - exiting 10:56:00 (17036): No heartbeat from core client for 30 sec - exiting 10:56:01 (17036): No heartbeat from core client for 30 sec - exiting 10:56:02 (17036): No heartbeat from core client for 30 sec - exiting 10:56:03 (17036): No heartbeat from core client for 30 sec - exiting 10:56:04 (17036): No heartbeat from core client for 30 sec - exiting 10:56:05 (17036): No heartbeat from core client for 30 sec - exiting 10:56:06 (17036): No heartbeat from core client for 30 sec - exiting 10:56:07 (17036): No heartbeat from core client for 30 sec - exiting 10:56:08 (17036): No heartbeat from core client for 30 sec - exiting 10:56:09 (17036): No heartbeat from core client for 30 sec - exiting 10:56:10 (17036): No heartbeat from core client for 30 sec - exiting 10:56:11 (17036): No heartbeat from core client for 30 sec - exiting 10:56:12 (17036): No heartbeat from core client for 30 sec - exiting 10:56:13 (17036): No heartbeat from core client for 30 sec - exiting 10:56:14 (17036): No heartbeat from core client for 30 sec - exiting 10:56:15 (17036): No heartbeat from core client for 30 sec - exiting 10:56:16 (17036): No heartbeat from core client for 30 sec - exiting 10:56:17 (17036): No heartbeat from core client for 30 sec - exiting 10:56:18 (17036): No heartbeat from core client for 30 sec - exiting 10:56:19 (17036): No heartbeat from core client for 30 sec - exiting 10:56:20 (17036): No heartbeat from core client for 30 sec - exiting 10:56:21 (17036): No heartbeat from core client for 30 sec - exiting 10:56:22 (17036): No heartbeat from core client for 30 sec - exiting 10:56:23 (17036): No heartbeat from core client for 30 sec - exiting 10:56:24 (17036): No heartbeat from core client for 30 sec - exiting 10:56:25 (17036): No heartbeat from core client for 30 sec - exiting 10:56:26 (17036): No heartbeat from core client for 30 sec - exiting 10:56:27 (17036): No heartbeat from core client for 30 sec - exiting 10:56:28 (17036): No heartbeat from core client for 30 sec - exiting 10:56:29 (17036): No heartbeat from core client for 30 sec - exiting 10:56:30 (17036): No heartbeat from core client for 30 sec - exiting 10:56:31 (17036): No heartbeat from core client for 30 sec - exiting 10:56:32 (17036): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:57:16 (2180): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 10:58:33 (11844): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:58:34 (11844): No heartbeat from core client for 30 sec - exiting 10:59:10 (15192): No heartbeat from core client for 30 sec - exiting 10:59:11 (15192): No heartbeat from core client for 30 sec - exiting 10:59:12 (15192): No heartbeat from core client for 30 sec - exiting 10:59:13 (15192): No heartbeat from core client for 30 sec - exiting 10:59:14 (15192): No heartbeat from core client for 30 sec - exiting 10:59:15 (15192): No heartbeat from core client for 30 sec - exiting 10:59:16 (15192): No heartbeat from core client for 30 sec - exiting 10:59:17 (15192): No heartbeat from core client for 30 sec - exiting 10:59:18 (15192): No heartbeat from core client for 30 sec - exiting 10:59:19 (15192): No heartbeat from core client for 30 sec - exiting 10:59:20 (15192): No heartbeat from core client for 30 sec - exiting 10:59:21 (15192): No heartbeat from core client for 30 sec - exiting 10:59:22 (15192): No heartbeat from core client for 30 sec - exiting 10:59:23 (15192): No heartbeat from core client for 30 sec - exiting 10:59:24 (15192): No heartbeat from core client for 30 sec - exiting 10:59:25 (15192): No heartbeat from core client for 30 sec - exiting 10:59:26 (15192): No heartbeat from core client for 30 sec - exiting 10:59:27 (15192): No heartbeat from core client for 30 sec - exiting 10:59:28 (15192): No heartbeat from core client for 30 sec - exiting 10:59:29 (15192): No heartbeat from core client for 30 sec - exiting 10:59:30 (15192): No heartbeat from core client for 30 sec - exiting 10:59:31 (15192): No heartbeat from core client for 30 sec - exiting 10:59:32 (15192): No heartbeat from core client for 30 sec - exiting 10:59:33 (15192): No heartbeat from core client for 30 sec - exiting 10:59:34 (15192): No heartbeat from core client for 30 sec - exiting 10:59:35 (15192): No heartbeat from core client for 30 sec - exiting 10:59:36 (15192): No heartbeat from core client for 30 sec - exiting 10:59:37 (15192): No heartbeat from core client for 30 sec - exiting 10:59:38 (15192): No heartbeat from core client for 30 sec - exiting 10:59:39 (15192): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:59:40 (15192): No heartbeat from core client for 30 sec - exiting 10:59:41 (15192): No heartbeat from core client for 30 sec - exiting 10:59:42 (15192): No heartbeat from core client for 30 sec - exiting 11:00:42 (15180): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3988, iMonCtr=1 Model crash detected, will try to restart... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
11 Jan 2012 08:48:57	1137123	13779552	hadcm3n_yci9_1940_40_007545614_4	129,600	251,030	1.9370
09 Jan 2012 13:14:28	1137123	13779552	hadcm3n_yci9_1940_40_007545614_4	103,680	200,955	1.9382
03 Jan 2012 19:09:57	1137123	13779552	hadcm3n_yci9_1940_40_007545614_4	77,760	151,047	1.9425
22 Dec 2011 14:24:58	1137123	13779552	hadcm3n_yci9_1940_40_007545614_4	51,840	101,013	1.9486
20 Dec 2011 02:31:23	1137123	13779552	hadcm3n_yci9_1940_40_007545614_4	25,920	50,534	1.9496