Task 16202371

Name	hadcm3n_ob1s_1900_40_008469443_2
Workunit	8620282
Created	6 Jan 2014, 8:52:16 UTC
Sent	6 Jan 2014, 8:52:34 UTC
Report deadline	7 Apr 2014, 16:19:45 UTC
Received	13 Feb 2014, 15:54:52 UTC
Server state	Over
Outcome	Success
Client state	Done
Exit status	0 (0x00000000)
Computer ID	957844
Run time	18 days 0 hours 20 min 50 sec
CPU time	18 days 0 hours 20 min 50 sec
Validate state	Valid
Credit	12,441.60
Device peak FLOPS	2.97 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.4.7</core_client_version> <![CDATA[ <stderr_txt> Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4804, iMonCtr=1 Model crash detected, will try to restart... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 CPDN Monitor - Quit request from BOINC... 23:58:27 (4156): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:59:21 (5624): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5012, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5012, iMonCtr=1 Model crash detected, will try to restart... 17:07:04 (4528): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5692, iMonCtr=1 Model crash detected, will try to restart... 21:51:52 (4752): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:57:29 (5380): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:54:01 (5164): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5916, iMonCtr=1 Model crash detected, will try to restart... 23:18:22 (4564): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:09:09 (5652): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:34:46 (6020): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5656, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5656, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5656, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5656, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5656, iMonCtr=1 Model crash detected, will try to restart... 20:06:56 (6048): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=904, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3848, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5724, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5264, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... 00:29:29 (1248): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:09:52 (4160): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:09:53 (4160): No heartbeat from core client for 30 sec - exiting 13:09:54 (4160): No heartbeat from core client for 30 sec - exiting 13:09:55 (4160): No heartbeat from core client for 30 sec - exiting 13:09:56 (4160): No heartbeat from core client for 30 sec - exiting 13:09:57 (4160): No heartbeat from core client for 30 sec - exiting 13:09:58 (4160): No heartbeat from core client for 30 sec - exiting 13:10:00 (4160): No heartbeat from core client for 30 sec - exiting 13:10:01 (4160): No heartbeat from core client for 30 sec - exiting 13:10:02 (4160): No heartbeat from core client for 30 sec - exiting 21:27:53 (3428): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:58:25 (5292): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5592, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5340, iMonCtr=1 Model crash detected, will try to restart... 17:46:35 (5384): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:46:36 (5384): No heartbeat from core client for 30 sec - exiting 17:46:37 (5384): No heartbeat from core client for 30 sec - exiting 17:46:38 (5384): No heartbeat from core client for 30 sec - exiting 17:46:39 (5384): No heartbeat from core client for 30 sec - exiting 17:46:40 (5384): No heartbeat from core client for 30 sec - exiting 17:46:41 (5384): No heartbeat from core client for 30 sec - exiting 09:40:30 (4532): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:26:16 (5136): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:44:51 (5668): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:45:55 (5476): No heartbeat from core client for 30 sec - exiting 02:45:56 (5476): No heartbeat from core client for 30 sec - exiting 02:45:57 (5476): No heartbeat from core client for 30 sec - exiting 02:45:58 (5476): No heartbeat from core client for 30 sec - exiting 02:45:59 (5476): No heartbeat from core client for 30 sec - exiting 02:46:00 (5476): No heartbeat from core client for 30 sec - exiting 02:46:01 (5476): No heartbeat from core client for 30 sec - exiting 02:46:02 (5476): No heartbeat from core client for 30 sec - exiting 02:46:03 (5476): No heartbeat from core client for 30 sec - exiting 02:46:04 (5476): No heartbeat from core client for 30 sec - exiting 02:46:05 (5476): No heartbeat from core client for 30 sec - exiting 02:46:06 (5476): No heartbeat from core client for 30 sec - exiting 02:46:07 (5476): No heartbeat from core client for 30 sec - exiting 02:46:08 (5476): No heartbeat from core client for 30 sec - exiting 02:46:09 (5476): No heartbeat from core client for 30 sec - exiting 02:46:10 (5476): No heartbeat from core client for 30 sec - exiting 02:46:11 (5476): No heartbeat from core client for 30 sec - exiting 02:46:12 (5476): No heartbeat from core client for 30 sec - exiting 02:46:13 (5476): No heartbeat from core client for 30 sec - exiting 02:46:14 (5476): No heartbeat from core client for 30 sec - exiting 02:46:15 (5476): No heartbeat from core client for 30 sec - exiting 02:46:16 (5476): No heartbeat from core client for 30 sec - exiting 02:46:17 (5476): No heartbeat from core client for 30 sec - exiting 02:46:18 (5476): No heartbeat from core client for 30 sec - exiting 02:46:19 (5476): No heartbeat from core client for 30 sec - exiting 02:46:20 (5476): No heartbeat from core client for 30 sec - exiting 02:46:21 (5476): No heartbeat from core client for 30 sec - exiting 02:46:22 (5476): No heartbeat from core client for 30 sec - exiting 02:46:23 (5476): No heartbeat from core client for 30 sec - exiting 02:46:24 (5476): No heartbeat from core client for 30 sec - exiting 02:46:25 (5476): No heartbeat from core client for 30 sec - exiting 02:46:26 (5476): No heartbeat from core client for 30 sec - exiting 02:46:27 (5476): No heartbeat from core client for 30 sec - exiting 02:46:28 (5476): No heartbeat from core client for 30 sec - exiting 02:46:29 (5476): No heartbeat from core client for 30 sec - exiting 02:46:30 (5476): No heartbeat from core client for 30 sec - exiting 02:46:31 (5476): No heartbeat from core client for 30 sec - exiting 02:46:32 (5476): No heartbeat from core client for 30 sec - exiting 02:46:33 (5476): No heartbeat from core client for 30 sec - exiting 02:46:34 (5476): No heartbeat from core client for 30 sec - exiting 02:46:35 (5476): No heartbeat from core client for 30 sec - exiting 02:46:36 (5476): No heartbeat from core client for 30 sec - exiting 02:46:37 (5476): No heartbeat from core client for 30 sec - exiting 02:46:38 (5476): No heartbeat from core client for 30 sec - exiting 02:46:39 (5476): No heartbeat from core client for 30 sec - exiting 02:46:40 (5476): No heartbeat from core client for 30 sec - exiting 02:46:41 (5476): No heartbeat from core client for 30 sec - exiting 02:46:42 (5476): No heartbeat from core client for 30 sec - exiting 02:46:43 (5476): No heartbeat from core client for 30 sec - exiting 02:46:44 (5476): No heartbeat from core client for 30 sec - exiting 02:46:45 (5476): No heartbeat from core client for 30 sec - exiting 02:46:46 (5476): No heartbeat from core client for 30 sec - exiting 02:46:47 (5476): No heartbeat from core client for 30 sec - exiting 02:46:48 (5476): No heartbeat from core client for 30 sec - exiting 02:46:49 (5476): No heartbeat from core client for 30 sec - exiting 02:46:50 (5476): No heartbeat from core client for 30 sec - exiting 02:46:51 (5476): No heartbeat from core client for 30 sec - exiting 02:46:52 (5476): No heartbeat from core client for 30 sec - exiting 02:46:53 (5476): No heartbeat from core client for 30 sec - exiting 02:46:54 (5476): No heartbeat from core client for 30 sec - exiting 02:46:55 (5476): No heartbeat from core client for 30 sec - exiting 02:46:56 (5476): No heartbeat from core client for 30 sec - exiting 02:46:57 (5476): No heartbeat from core client for 30 sec - exiting 02:46:58 (5476): No heartbeat from core client for 30 sec - exiting 02:46:59 (5476): No heartbeat from core client for 30 sec - exiting 02:47:00 (5476): No heartbeat from core client for 30 sec - exiting 02:47:01 (5476): No heartbeat from core client for 30 sec - exiting 02:47:02 (5476): No heartbeat from core client for 30 sec - exiting 02:47:03 (5476): No heartbeat from core client for 30 sec - exiting 02:47:04 (5476): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:54:50 (5796): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:34:19 (4212): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6064, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5888, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4928, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
13 Feb 2014 09:37:36	957844	16202371	hadcm3n_ob1s_1900_40_008469443_2	1,036,800	1,555,919	1.5007
12 Feb 2014 20:30:28	957844	16202371	hadcm3n_ob1s_1900_40_008469443_2	1,010,880	1,518,122	1.5018
11 Feb 2014 02:29:47	957844	16202371	hadcm3n_ob1s_1900_40_008469443_2	984,960	1,476,626	1.4992
09 Feb 2014 00:59:09	957844	16202371	hadcm3n_ob1s_1900_40_008469443_2	959,040	1,436,195	1.4975
08 Feb 2014 14:37:30	957844	16202371	hadcm3n_ob1s_1900_40_008469443_2	933,120	1,399,667	1.5000
07 Feb 2014 12:10:57	957844	16202371	hadcm3n_ob1s_1900_40_008469443_2	907,200	1,357,345	1.4962
06 Feb 2014 10:20:46	957844	16202371	hadcm3n_ob1s_1900_40_008469443_2	881,280	1,317,042	1.4945
05 Feb 2014 11:01:19	957844	16202371	hadcm3n_ob1s_1900_40_008469443_2	855,360	1,276,728	1.4926
04 Feb 2014 04:48:24	957844	16202371	hadcm3n_ob1s_1900_40_008469443_2	829,440	1,236,346	1.4906
03 Feb 2014 02:24:59	957844	16202371	hadcm3n_ob1s_1900_40_008469443_2	803,520	1,195,936	1.4884
02 Feb 2014 03:22:33	957844	16202371	hadcm3n_ob1s_1900_40_008469443_2	777,600	1,157,833	1.4890
01 Feb 2014 16:24:48	957844	16202371	hadcm3n_ob1s_1900_40_008469443_2	751,680	1,119,931	1.4899
31 Jan 2014 11:29:45	957844	16202371	hadcm3n_ob1s_1900_40_008469443_2	725,760	1,078,774	1.4864
30 Jan 2014 16:06:57	957844	16202371	hadcm3n_ob1s_1900_40_008469443_2	699,840	1,038,870	1.4844
30 Jan 2014 16:06:57	957844	16202371	hadcm3n_ob1s_1900_40_008469443_2	673,920	998,681	1.4819
30 Jan 2014 16:06:57	957844	16202371	hadcm3n_ob1s_1900_40_008469443_2	648,000	959,792	1.4812
27 Jan 2014 17:57:47	957844	16202371	hadcm3n_ob1s_1900_40_008469443_2	622,080	916,908	1.4739
25 Jan 2014 23:10:11	957844	16202371	hadcm3n_ob1s_1900_40_008469443_2	596,160	879,742	1.4757
25 Jan 2014 04:36:25	957844	16202371	hadcm3n_ob1s_1900_40_008469443_2	570,240	839,885	1.4729
24 Jan 2014 11:21:29	957844	16202371	hadcm3n_ob1s_1900_40_008469443_2	544,320	799,977	1.4697