Task 16074810

Name	hadcm3n_od82_1900_40_008472261_1
Workunit	8623100
Created	27 Oct 2013, 18:52:50 UTC
Sent	27 Oct 2013, 18:53:01 UTC
Report deadline	27 Jan 2014, 2:20:12 UTC
Received	13 Jan 2014, 6:24:21 UTC
Server state	Over
Outcome	Success
Client state	Done
Exit status	0 (0x00000000)
Computer ID	1294309
Run time	20 days 7 hours 27 min 32 sec
CPU time	17 days 7 hours 24 min 30 sec
Validate state	Valid
Credit	12,441.60
Device peak FLOPS	2.51 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.2.33</core_client_version> <![CDATA[ <stderr_txt> 17:15:13 (1060): No heartbeat from core client for 30 sec - exiting 17:15:14 (1060): No heartbeat from core client for 30 sec - exiting 17:15:15 (1060): No heartbeat from core client for 30 sec - exiting 17:15:16 (1060): No heartbeat from core client for 30 sec - exiting 17:15:17 (1060): No heartbeat from core client for 30 sec - exiting 17:15:18 (1060): No heartbeat from core client for 30 sec - exiting 17:15:19 (1060): No heartbeat from core client for 30 sec - exiting 17:15:20 (1060): No heartbeat from core client for 30 sec - exiting 17:15:21 (1060): No heartbeat from core client for 30 sec - exiting 17:15:22 (1060): No heartbeat from core client for 30 sec - exiting 17:15:23 (1060): No heartbeat from core client for 30 sec - exiting 17:15:24 (1060): No heartbeat from core client for 30 sec - exiting 17:15:25 (1060): No heartbeat from core client for 30 sec - exiting 17:15:26 (1060): No heartbeat from core client for 30 sec - exiting 17:15:27 (1060): No heartbeat from core client for 30 sec - exiting 17:15:28 (1060): No heartbeat from core client for 30 sec - exiting 17:15:29 (1060): No heartbeat from core client for 30 sec - exiting 17:15:30 (1060): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 12:03:43 (3892): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:19:36 (8752): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:19:38 (8752): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 12:13:12 (5016): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 11:12:16 (3816): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 17:54:14 (5444): No heartbeat from core client for 30 sec - exiting 17:54:16 (5444): No heartbeat from core client for 30 sec - exiting 17:54:17 (5444): No heartbeat from core client for 30 sec - exiting 17:54:18 (5444): No heartbeat from core client for 30 sec - exiting 17:54:19 (5444): No heartbeat from core client for 30 sec - exiting 17:54:20 (5444): No heartbeat from core client for 30 sec - exiting 17:54:21 (5444): No heartbeat from core client for 30 sec - exiting 17:54:22 (5444): No heartbeat from core client for 30 sec - exiting 17:54:23 (5444): No heartbeat from core client for 30 sec - exiting 17:54:24 (5444): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 21:34:40 (5784): No heartbeat from core client for 30 sec - exiting 21:34:41 (5784): No heartbeat from core client for 30 sec - exiting 21:34:42 (5784): No heartbeat from core client for 30 sec - exiting 21:34:43 (5784): No heartbeat from core client for 30 sec - exiting 21:34:44 (5784): No heartbeat from core client for 30 sec - exiting 21:34:45 (5784): No heartbeat from core client for 30 sec - exiting 21:34:46 (5784): No heartbeat from core client for 30 sec - exiting 21:34:47 (5784): No heartbeat from core client for 30 sec - exiting 21:34:48 (5784): No heartbeat from core client for 30 sec - exiting 21:34:49 (5784): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 19:13:36 (4756): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:22:27 (1352): No heartbeat from core client for 30 sec - exiting 19:22:28 (1352): No heartbeat from core client for 30 sec - exiting 19:22:29 (1352): No heartbeat from core client for 30 sec - exiting 19:22:30 (1352): No heartbeat from core client for 30 sec - exiting 19:22:31 (1352): No heartbeat from core client for 30 sec - exiting 19:22:32 (1352): No heartbeat from core client for 30 sec - exiting 19:22:33 (1352): No heartbeat from core client for 30 sec - exiting 19:22:34 (1352): No heartbeat from core client for 30 sec - exiting 19:22:35 (1352): No heartbeat from core client for 30 sec - exiting 19:22:36 (1352): No heartbeat from core client for 30 sec - exiting 19:22:37 (1352): No heartbeat from core client for 30 sec - exiting 19:22:38 (1352): No heartbeat from core client for 30 sec - exiting 19:22:39 (1352): No heartbeat from core client for 30 sec - exiting 19:22:40 (1352): No heartbeat from core client for 30 sec - exiting 19:22:41 (1352): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:57:46 (5952): No heartbeat from core client for 30 sec - exiting 19:57:48 (5952): No heartbeat from core client for 30 sec - exiting 19:57:49 (5952): No heartbeat from core client for 30 sec - exiting 19:57:50 (5952): No heartbeat from core client for 30 sec - exiting 19:57:51 (5952): No heartbeat from core client for 30 sec - exiting 19:57:52 (5952): No heartbeat from core client for 30 sec - exiting 19:57:53 (5952): No heartbeat from core client for 30 sec - exiting 19:57:54 (5952): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 17:23:16 (5732): No heartbeat from core client for 30 sec - exiting 17:23:17 (5732): No heartbeat from core client for 30 sec - exiting 17:23:19 (5732): No heartbeat from core client for 30 sec - exiting 17:23:20 (5732): No heartbeat from core client for 30 sec - exiting 17:23:21 (5732): No heartbeat from core client for 30 sec - exiting 17:23:22 (5732): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 08:13:25 (5976): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 12:40:38 (5168): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 17:14:19 (4732): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 11:15:02 (5632): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 07:12:49 (5220): No heartbeat from core client for 30 sec - exiting 07:12:50 (5220): No heartbeat from core client for 30 sec - exiting 07:12:51 (5220): No heartbeat from core client for 30 sec - exiting 07:12:52 (5220): No heartbeat from core client for 30 sec - exiting 07:12:53 (5220): No heartbeat from core client for 30 sec - exiting 07:12:54 (5220): No heartbeat from core client for 30 sec - exiting 07:12:55 (5220): No heartbeat from core client for 30 sec - exiting 07:12:56 (5220): No heartbeat from core client for 30 sec - exiting 07:12:57 (5220): No heartbeat from core client for 30 sec - exiting 07:12:58 (5220): No heartbeat from core client for 30 sec - exiting 07:12:59 (5220): No heartbeat from core client for 30 sec - exiting 07:13:00 (5220): No heartbeat from core client for 30 sec - exiting 07:13:01 (5220): No heartbeat from core client for 30 sec - exiting 07:13:02 (5220): No heartbeat from core client for 30 sec - exiting 07:13:03 (5220): No heartbeat from core client for 30 sec - exiting 07:13:04 (5220): No heartbeat from core client for 30 sec - exiting 07:13:05 (5220): No heartbeat from core client for 30 sec - exiting 07:13:06 (5220): No heartbeat from core client for 30 sec - exiting 07:13:07 (5220): No heartbeat from core client for 30 sec - exiting 07:13:08 (5220): No heartbeat from core client for 30 sec - exiting 07:13:09 (5220): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 07:14:35 (1508): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 07:00:35 (5112): No heartbeat from core client for 30 sec - exiting 07:00:36 (5112): No heartbeat from core client for 30 sec - exiting 07:00:37 (5112): No heartbeat from core client for 30 sec - exiting 07:00:38 (5112): No heartbeat from core client for 30 sec - exiting 07:00:39 (5112): No heartbeat from core client for 30 sec - exiting 07:00:40 (5112): No heartbeat from core client for 30 sec - exiting 07:00:41 (5112): No heartbeat from core client for 30 sec - exiting 07:00:42 (5112): No heartbeat from core client for 30 sec - exiting 07:00:44 (5112): No heartbeat from core client for 30 sec - exiting 07:00:45 (5112): No heartbeat from core client for 30 sec - exiting 07:00:46 (5112): No heartbeat from core client for 30 sec - exiting 07:00:47 (5112): No heartbeat from core client for 30 sec - exiting 07:00:48 (5112): No heartbeat from core client for 30 sec - exiting 07:00:49 (5112): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5832, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5004, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5004, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5004, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5004, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5972, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5972, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5600, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5600, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5600, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5600, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5940, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5940, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5412, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5412, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5412, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6104, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6104, iMonCtr=1 Model crash detected, will try to restart... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
12 Jan 2014 21:45:20	1294309	16074810	hadcm3n_od82_1900_40_008472261_1	1,036,800	1,494,847	1.4418
12 Jan 2014 10:33:56	1294309	16074810	hadcm3n_od82_1900_40_008472261_1	1,010,880	1,456,592	1.4409
11 Jan 2014 15:09:23	1294309	16074810	hadcm3n_od82_1900_40_008472261_1	984,960	1,419,143	1.4408
11 Jan 2014 06:12:01	1294309	16074810	hadcm3n_od82_1900_40_008472261_1	959,040	1,382,274	1.4413
09 Jan 2014 18:57:50	1294309	16074810	hadcm3n_od82_1900_40_008472261_1	933,120	1,338,923	1.4349
09 Jan 2014 09:22:18	1294309	16074810	hadcm3n_od82_1900_40_008472261_1	907,200	1,302,954	1.4362
08 Jan 2014 14:38:12	1294309	16074810	hadcm3n_od82_1900_40_008472261_1	881,280	1,267,886	1.4387
07 Jan 2014 19:11:29	1294309	16074810	hadcm3n_od82_1900_40_008472261_1	855,360	1,232,367	1.4408
07 Jan 2014 06:47:29	1294309	16074810	hadcm3n_od82_1900_40_008472261_1	829,440	1,196,270	1.4423
05 Jan 2014 12:00:39	1294309	16074810	hadcm3n_od82_1900_40_008472261_1	803,520	1,154,681	1.4370
29 Dec 2013 09:47:28	1294309	16074810	hadcm3n_od82_1900_40_008472261_1	777,600	1,116,650	1.4360
25 Dec 2013 06:45:15	1294309	16074810	hadcm3n_od82_1900_40_008472261_1	751,680	1,078,874	1.4353
23 Dec 2013 18:39:36	1294309	16074810	hadcm3n_od82_1900_40_008472261_1	725,760	1,039,526	1.4323
23 Dec 2013 06:35:42	1294309	16074810	hadcm3n_od82_1900_40_008472261_1	699,840	999,286	1.4279
22 Dec 2013 12:05:20	1294309	16074810	hadcm3n_od82_1900_40_008472261_1	673,920	963,050	1.4290
21 Dec 2013 17:10:19	1294309	16074810	hadcm3n_od82_1900_40_008472261_1	648,000	927,677	1.4316
21 Dec 2013 06:20:48	1294309	16074810	hadcm3n_od82_1900_40_008472261_1	622,080	892,519	1.4347
20 Dec 2013 11:36:37	1294309	16074810	hadcm3n_od82_1900_40_008472261_1	596,160	857,502	1.4384
19 Dec 2013 15:36:11	1294309	16074810	hadcm3n_od82_1900_40_008472261_1	570,240	819,804	1.4376
18 Dec 2013 19:01:12	1294309	16074810	hadcm3n_od82_1900_40_008472261_1	544,320	779,415	1.4319