Task 13020395

Name	hadcm3n_t17i_1940_40_007314023_0
Workunit	7511453
Created	28 Jun 2011, 13:27:06 UTC
Sent	28 Jun 2011, 13:27:36 UTC
Report deadline	27 Sep 2011, 20:54:47 UTC
Received	27 Aug 2011, 17:05:52 UTC
Server state	Over
Outcome	Success
Client state	Done
Exit status	0 (0x00000000)
Computer ID	1021113
Run time	22 days 14 hours 44 min 41 sec
CPU time	21 days 23 hours 13 min 46 sec
Validate state	Valid
Credit	12,441.60
Device peak FLOPS	2.53 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.12.33</core_client_version> <![CDATA[ <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... 15:31:53 (6808): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7184, iMonCtr=1 Model crash detected, will try to restart... 15:26:20 (5140): No heartbeat from core client for 30 sec - exiting 15:26:21 (5140): No heartbeat from core client for 30 sec - exiting 15:26:22 (5140): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 09:40:52 (6288): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6896, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5972, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5424, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3440, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5256, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5956, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3664, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4540, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4804, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5664, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5664, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6432, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5444, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6728, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3444, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6064, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7488, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6200, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6200, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6200, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6200, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6200, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5152, iMonCtr=1 Model crash detected, will try to restart... 13:51:48 (392): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3948, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3948, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 22:11:21 (2880): No heartbeat from core client for 30 sec - exiting 22:11:22 (2880): No heartbeat from core client for 30 sec - exiting 22:11:23 (2880): No heartbeat from core client for 30 sec - exiting 22:11:24 (2880): No heartbeat from core client for 30 sec - exiting 22:11:25 (2880): No heartbeat from core client for 30 sec - exiting 22:11:26 (2880): No heartbeat from core client for 30 sec - exiting 22:11:27 (2880): No heartbeat from core client for 30 sec - exiting 22:11:28 (2880): No heartbeat from core client for 30 sec - exiting 22:11:29 (2880): No heartbeat from core client for 30 sec - exiting 22:11:30 (2880): No heartbeat from core client for 30 sec - exiting 22:11:31 (2880): No heartbeat from core client for 30 sec - exiting 22:11:32 (2880): No heartbeat from core client for 30 sec - exiting 22:11:33 (2880): No heartbeat from core client for 30 sec - exiting 22:11:34 (2880): No heartbeat from core client for 30 sec - exiting 22:11:35 (2880): No heartbeat from core client for 30 sec - exiting 22:11:36 (2880): No heartbeat from core client for 30 sec - exiting 22:11:37 (2880): No heartbeat from core client for 30 sec - exiting 22:11:38 (2880): No heartbeat from core client for 30 sec - exiting 22:11:39 (2880): No heartbeat from core client for 30 sec - exiting 22:11:40 (2880): No heartbeat from core client for 30 sec - exiting 22:11:41 (2880): No heartbeat from core client for 30 sec - exiting 22:11:42 (2880): No heartbeat from core client for 30 sec - exiting 22:11:43 (2880): No heartbeat from core client for 30 sec - exiting 22:11:44 (2880): No heartbeat from core client for 30 sec - exiting 22:11:45 (2880): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3380, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4008, iMonCtr=1 Model crash detected, will try to restart... 08:00:24 (5936): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3076, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3288, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3144, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3152, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3644, iMonCtr=1 Model crash detected, will try to restart... 15:23:49 (6908): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7268, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5588, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5540, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5436, iMonCtr=1 Model crash detected, will try to restart... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5000, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6696, iMonCtr=1 Model crash detected, will try to restart... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
27 Aug 2011 17:05:07	1021113	13020395	hadcm3n_t17i_1940_40_007314023_0	1,036,800	1,897,391	1.8300
25 Aug 2011 17:33:41	1021113	13020395	hadcm3n_t17i_1940_40_007314023_0	1,010,880	1,850,099	1.8302
25 Aug 2011 06:07:35	1021113	13020395	hadcm3n_t17i_1940_40_007314023_0	984,960	1,809,046	1.8367
22 Aug 2011 18:05:25	1021113	13020395	hadcm3n_t17i_1940_40_007314023_0	959,040	1,767,542	1.8430
21 Aug 2011 13:56:02	1021113	13020395	hadcm3n_t17i_1940_40_007314023_0	933,120	1,725,447	1.8491
20 Aug 2011 15:07:14	1021113	13020395	hadcm3n_t17i_1940_40_007314023_0	907,200	1,685,183	1.8576
19 Aug 2011 16:16:50	1021113	13020395	hadcm3n_t17i_1940_40_007314023_0	881,280	1,644,237	1.8657
18 Aug 2011 19:32:25	1021113	13020395	hadcm3n_t17i_1940_40_007314023_0	855,360	1,603,337	1.8745
17 Aug 2011 14:17:25	1021113	13020395	hadcm3n_t17i_1940_40_007314023_0	829,440	1,561,935	1.8831
16 Aug 2011 17:35:48	1021113	13020395	hadcm3n_t17i_1940_40_007314023_0	803,520	1,521,025	1.8930
16 Aug 2011 06:15:14	1021113	13020395	hadcm3n_t17i_1940_40_007314023_0	777,600	1,480,500	1.9039
14 Aug 2011 15:57:43	1021113	13020395	hadcm3n_t17i_1940_40_007314023_0	751,680	1,439,389	1.9149
13 Aug 2011 18:01:30	1021113	13020395	hadcm3n_t17i_1940_40_007314023_0	725,760	1,398,357	1.9267
13 Aug 2011 06:36:46	1021113	13020395	hadcm3n_t17i_1940_40_007314023_0	699,840	1,357,915	1.9403
12 Aug 2011 19:16:22	1021113	13020395	hadcm3n_t17i_1940_40_007314023_0	673,920	1,317,304	1.9547
10 Aug 2011 16:54:06	1021113	13020395	hadcm3n_t17i_1940_40_007314023_0	648,000	1,271,784	1.9626
08 Aug 2011 14:46:39	1021113	13020395	hadcm3n_t17i_1940_40_007314023_0	622,080	1,222,683	1.9655
07 Aug 2011 07:10:16	1021113	13020395	hadcm3n_t17i_1940_40_007314023_0	596,160	1,177,014	1.9743
29 Jul 2011 18:55:49	1021113	13020395	hadcm3n_t17i_1940_40_007314023_0	570,240	1,131,405	1.9841
28 Jul 2011 19:28:08	1021113	13020395	hadcm3n_t17i_1940_40_007314023_0	544,320	1,080,399	1.9849