Task 15673272

Name	hadcm3n_3ce6_2020_40_008324288_2
Workunit	8475423
Created	19 Mar 2013, 18:20:16 UTC
Sent	19 Mar 2013, 18:20:28 UTC
Report deadline	19 Jun 2013, 1:47:39 UTC
Received	29 Jun 2013, 3:51:23 UTC
Server state	Over
Outcome	Success
Client state	Done
Exit status	0 (0x00000000)
Computer ID	1166628
Run time	19 days 8 hours 29 min 29 sec
CPU time	19 days 8 hours 29 min 29 sec
Validate state	Valid
Credit	12,441.60
Device peak FLOPS	2.80 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.4.5</core_client_version> <![CDATA[ <stderr_txt> Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2284, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5056, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4248, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4308, iMonCtr=1 Model crash detected, will try to restart... Ocean Restart file copy failed on 3ce6ko.dam67m0 CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4668, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4804, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4112, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2020, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running,Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1196, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4120, iMonCtr=1 Model crash detected, will try to restart... Ocean Restart file copy failed on 3ce6ko.dan3110 CPDN Monitor - Quit request from BOINC... 13:39:32 (4660): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:39:33 (4660): No heartbeat from core client for 30 sec - exiting 13:39:34 (4660): No heartbeat from core client for 30 sec - exiting 13:39:35 (4660): No heartbeat from core client for 30 sec - exiting 13:39:36 (4660): No heartbeat from core client for 30 sec - exiting 13:39:37 (4660): No heartbeat from core client for 30 sec - exiting 13:42:37 (5336): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:42:38 (5336): No heartbeat from core client for 30 sec - exiting 13:42:39 (5336): No heartbeat from core client for 30 sec - exiting 13:42:40 (5336): No heartbeat from core client for 30 sec - exiting 13:42:41 (5336): No heartbeat from core client for 30 sec - exiting Controller:: CPDN proceController:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4432, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... 13:49:15 (4568): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:59:40 (3444): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:27:14 (4376): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:20:19 (2592): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4496, iMonCtr=1 Model crash detected, will try to restart... 13:59:48 (5068): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:02:04 (5016): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:27:34 (5524): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:27:35 (5524): No heartbeat from core client for 30 sec - exiting 17:27:37 (5524): No heartbeat from core client for 30 sec - exiting 17:27:38 (5524): No heartbeat from core client for 30 sec - exiting 17:27:39 (5524): No heartbeat from core client for 30 sec - exiting 17:27:40 (5524): No heartbeat from core client for 30 sec - exiting Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4188, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4620, iMonCtr=1 Model crash detected, will try to restart... 10:26:49 (4296): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... C10:12:07 (4524): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:58:40 (4744): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:58:23 (4888): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:07:38 (6064): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:07:39 (6064): No heartbeat from core client for 30 sec - exiting 16:07:40 (6064): No heartbeat from core client for 30 sec - exiting 16:07:41 (6064): No heartbeat from core client for 30 sec - exiting 16:07:42 (6064): No heartbeat from core client for 30 sec - exiting 16:41:11 (5004): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:41:12 (5004): No heartbeat from core client for 30 sec - exiting 16:41:13 (5004): No heartbeat from core client for 30 sec - exiting 16:41:14 (5004): No heartbeat from core client for 30 sec - exiting 16:41:15 (5004): No heartbeat from core client for 30 sec - exiting 17:51:18 (1392): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4288, iMonCtr=1 Model crash detected, will try to restart... 09:59:50 (4208): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 12:33:26 (4692): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:51:52 (5880): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:51:53 (5880): No heartbeat from core client for 30 sec - exiting 13:25:17 (3320): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:28:20 (3492): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:28:21 (3492): No heartbeat from core client for 30 sec - exiting 13:28:22 (3492): No heartbeat from core client for 30 sec - exiting 13:28:23 (3492): No heartbeat from core client for 30 sec - exiting 13:28:24 (3492): No heartbeat from core client for 30 sec - exiting 13:37:29 (3984): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:37:30 (3984): No heartbeat from core client for 30 sec - exiting 13:37:31 (3984): No heartbeat from core client for 30 sec - exiting 13:37:32 (3984): No heartbeat from core client for 30 sec - exiting 13:37:33 (3984): No heartbeat from core client for 30 sec - exiting 13:37:34 (3984): No heartbeat from core client for 30 sec - exiting 13:37:35 (3984): No heartbeat from core client for 30 sec - exiting 14:38:05 (3760): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:02:12 (4772): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:05:53 (4604): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:14:59 (4360): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:15:00 (4360): No heartbeat from core client for 30 sec - exiting 16:15:01 (4360): No heartbeat from core client for 30 sec - exiting 16:15:02 (4360): No heartbeat from core client for 30 sec - exiting 17:39:33 (5952): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CController:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4252, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4776, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4304, iMonCtr=1 Model crash detected, will try to restart... 11:08:54 (4880): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPCPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3052, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4480, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... 19:14:43 (5932): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:54:20 (8188): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:06:44 (7524): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:06:46 (7524): No heartbeat from core client for 30 sec - exiting 11:06:47 (7524): No heartbeat from core client for 30 sec - exiting 11:06:48 (7524): No heartbeat from core client for 30 sec - exiting 11:06:49 (7524): No heartbeat from core client for 30 sec - exiting 11:06:50 (7524): No heartbeat from core client for 30 sec - exiting 11:06:51 (7524): No heartbeat from core client for 30 sec - exiting 11:21:58 (7320): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:21:59 (7320): No heartbeat from core client for 30 sec - exiting 11:22:00 (7320): No heartbeat from core client for 30 sec - exiting 11:22:01 (7320): No heartbeat from core client for 30 sec - exiting 11:22:02 (7320): No heartbeat from core client for 30 sec - exiting 12:13:31 (7364): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Atmos Hold Restart file rename failed on atmos_restart.hold CPDN Monitor - Quit request from BOINC... 12:34:41 (7280): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:34:42 (7280): No heartbeat from core client for 30 sec - exiting 12:34:43 (7280): No heartbeat from core client for 30 sec - exiting 12:34:44 (7280): No heartbeat from core client for 30 sec - exiting 12:34:45 (7280): No heartbeat from core client for 30 sec - exiting 12:34:46 (7280): No heartbeat from core client for 30 sec - exiting 12:40:50 (6608): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
02 Jul 2013 09:55:46	1166628	15673272	hadcm3n_3ce6_2020_40_008324288_2	1,036,800	1,671,427	1.6121
28 Jun 2013 14:35:58	1166628	15673272	hadcm3n_3ce6_2020_40_008324288_2	1,010,880	1,628,977	1.6114
28 Jun 2013 01:24:47	1166628	15673272	hadcm3n_3ce6_2020_40_008324288_2	984,960	1,583,417	1.6076
27 Jun 2013 11:25:08	1166628	15673272	hadcm3n_3ce6_2020_40_008324288_2	959,040	1,542,806	1.6087
26 Jun 2013 18:38:50	1166628	15673272	hadcm3n_3ce6_2020_40_008324288_2	933,120	1,506,327	1.6143
26 Jun 2013 05:02:57	1166628	15673272	hadcm3n_3ce6_2020_40_008324288_2	907,200	1,463,733	1.6135
25 Jun 2013 16:15:54	1166628	15673272	hadcm3n_3ce6_2020_40_008324288_2	881,280	1,419,157	1.6103
25 Jun 2013 03:18:45	1166628	15673272	hadcm3n_3ce6_2020_40_008324288_2	855,360	1,375,679	1.6083
24 Jun 2013 14:22:25	1166628	15673272	hadcm3n_3ce6_2020_40_008324288_2	829,440	1,330,538	1.6041
24 Jun 2013 01:31:35	1166628	15673272	hadcm3n_3ce6_2020_40_008324288_2	803,520	1,286,206	1.6007
23 Jun 2013 13:03:32	1166628	15673272	hadcm3n_3ce6_2020_40_008324288_2	777,600	1,242,813	1.5983
23 Jun 2013 00:13:07	1166628	15673272	hadcm3n_3ce6_2020_40_008324288_2	751,680	1,198,381	1.5943
22 Jun 2013 11:52:37	1166628	15673272	hadcm3n_3ce6_2020_40_008324288_2	725,760	1,155,275	1.5918
21 Jun 2013 23:54:47	1166628	15673272	hadcm3n_3ce6_2020_40_008324288_2	699,840	1,113,745	1.5914
21 Jun 2013 11:59:48	1166628	15673272	hadcm3n_3ce6_2020_40_008324288_2	673,920	1,071,981	1.5907
21 Jun 2013 00:03:40	1166628	15673272	hadcm3n_3ce6_2020_40_008324288_2	648,000	1,030,337	1.5900
19 Jun 2013 19:44:34	1166628	15673272	hadcm3n_3ce6_2020_40_008324288_2	622,080	989,918	1.5913
13 Jun 2013 16:53:00	1166628	15673272	hadcm3n_3ce6_2020_40_008324288_2	596,160	951,922	1.5968
23 May 2013 16:16:29	1166628	15673272	hadcm3n_3ce6_2020_40_008324288_2	570,240	913,957	1.6028
15 May 2013 19:40:36	1166628	15673272	hadcm3n_3ce6_2020_40_008324288_2	544,320	874,322	1.6063