Task 15902628

Name	hadcm3n_o4ls_1940_40_008380190_1
Workunit	8531049
Created	23 Jul 2013, 11:00:39 UTC
Sent	24 Jul 2013, 14:35:25 UTC
Report deadline	23 Oct 2013, 22:02:36 UTC
Received	23 Oct 2013, 11:45:31 UTC
Server state	Over
Outcome	Success
Client state	Done
Exit status	0 (0x00000000)
Computer ID	1061327
Run time	28 days 14 hours 5 min 57 sec
CPU time	28 days 3 hours 10 min 50 sec
Validate state	Valid
Credit	12,441.60
Device peak FLOPS	1.98 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.10.60</core_client_version> <![CDATA[ <stderr_txt> 21:53:40 (5752): No heartbeat from core client for 30 sec - exiting 21:53:42 (5752): No heartbeat from core client for 30 sec - exiting 21:53:43 (5752): No heartbeat from core client for 30 sec - exiting 21:53:44 (5752): No heartbeat from core client for 30 sec - exiting 21:53:45 (5752): No heartbeat from core client for 30 sec - exiting 21:53:46 (5752): No heartbeat from core client for 30 sec - exiting 21:53:47 (5752): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1396, selfPID=1396, iMonCtr=1 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/o4lsko.pje1c10 Error converting file to netcdf: dataout/o4lsko.pie1c10 Error converting file to netcdf: dataout/o4lsko.pfe1c10 Error converting file to netcdf: dataout/o4lska.phe1c10 Error converting file to netcdf: dataout/o4lska.pge1c10 Error converting file to netcdf: dataout/o4lska.pee1c10 19:16:15 (4924): No heartbeat from core client for 30 sec - exiting 19:16:16 (4924): No heartbeat from core client for 30 sec - exiting 19:16:17 (4924): No heartbeat from core client for 30 sec - exiting 19:16:18 (4924): No heartbeat from core client for 30 sec - exiting 19:16:19 (4924): No heartbeat from core client for 30 sec - exiting 19:16:20 (4924): No heartbeat from core client for 30 sec - exiting 19:16:21 (4924): No heartbeat from core client for 30 sec - exiting 19:16:22 (4924): No heartbeat from core client for 30 sec - exiting 19:16:23 (4924): No heartbeat from core client for 30 sec - exiting 19:16:24 (4924): No heartbeat from core client for 30 sec - exiting 19:16:26 (4924): No heartbeat from core client for 30 sec - exiting 19:16:27 (4924): No heartbeat from core client for 30 sec - exiting 19:16:28 (4924): No heartbeat from core client for 30 sec - exiting 19:16:29 (4924): No heartbeat from core client for 30 sec - exiting 19:16:30 (4924): No heartbeat from core client for 30 sec - exiting 19:16:31 (4924): No heartbeat from core client for 30 sec - exiting 19:16:32 (4924): No heartbeat from core client for 30 sec - exiting 19:16:33 (4924): No heartbeat from core client for 30 sec - exiting 19:16:34 (4924): No heartbeat from core client for 30 sec - exiting 19:16:35 (4924): No heartbeat from core client for 30 sec - exiting 19:16:36 (4924): No heartbeat from core client for 30 sec - exiting 19:16:38 (4924): No heartbeat from core client for 30 sec - exiting 19:16:39 (4924): No heartbeat from core client for 30 sec - exiting 19:16:40 (4924): No heartbeat from core client for 30 sec - exiting 19:16:41 (4924): No heartbeat from core client for 30 sec - exiting 19:16:42 (4924): No heartbeat from core client for 30 sec - exiting 19:16:43 (4924): No heartbeat from core client for 30 sec - exiting 19:16:44 (4924): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 12:34:20 (4916): No heartbeat from core client for 30 sec - exiting 12:34:22 (4916): No heartbeat from core client for 30 sec - exiting 12:34:23 (4916): No heartbeat from core client for 30 sec - exiting 12:34:24 (4916): No heartbeat from core client for 30 sec - exiting 12:34:25 (4916): No heartbeat from core client for 30 sec - exiting 12:34:26 (4916): No heartbeat from core client for 30 sec - exiting 12:34:27 (4916): No heartbeat from core client for 30 sec - exiting 12:34:28 (4916): No heartbeat from core client for 30 sec - exiting 12:34:29 (4916): No heartbeat from core client for 30 sec - exiting 12:34:30 (4916): No heartbeat from core client for 30 sec - exiting 12:34:31 (4916): No heartbeat from core client for 30 sec - exiting 12:34:32 (4916): No heartbeat from core client for 30 sec - exiting 12:34:34 (4916): No heartbeat from core client for 30 sec - exiting 12:34:35 (4916): No heartbeat from core client for 30 sec - exiting 12:34:36 (4916): No heartbeat from core client for 30 sec - exiting 12:34:37 (4916): No heartbeat from core client for 30 sec - exiting 12:34:38 (4916): No heartbeat from core client for 30 sec - exiting 12:34:39 (4916): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4984, iMonCtr=1 Model crash detected, will try to restart... 13:09:44 (904): No heartbeat from core client for 30 sec - exiting 13:09:45 (904): No heartbeat from core client for 30 sec - exiting 13:09:46 (904): No heartbeat from core client for 30 sec - exiting 13:09:47 (904): No heartbeat from core client for 30 sec - exiting 13:09:48 (904): No heartbeat from core client for 30 sec - exiting 13:09:50 (904): No heartbeat from core client for 30 sec - exiting 13:09:51 (904): No heartbeat from core client for 30 sec - exiting 13:09:52 (904): No heartbeat from core client for 30 sec - exiting 13:09:53 (904): No heartbeat from core client for 30 sec - exiting 13:09:54 (904): No heartbeat from core client for 30 sec - exiting 13:09:55 (904): No heartbeat from core client for 30 sec - exiting 13:09:56 (904): No heartbeat from core client for 30 sec - exiting 13:09:57 (904): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:35:46 (4872): No heartbeat from core client for 30 sec - exiting 21:35:47 (4872): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:08:05 (4580): No heartbeat from core client for 30 sec - exiting 21:08:06 (4580): No heartbeat from core client for 30 sec - exiting 21:08:07 (4580): No heartbeat from core client for 30 sec - exiting 21:08:08 (4580): No heartbeat from core client for 30 sec - exiting 21:08:09 (4580): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 04:53:56 (27488): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:20:52 (20116): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:16:16 (25732): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Atmos Hold Restart file rename failed on atmos_restart.hold 20:43:35 (4256): No heartbeat from core client for 30 sec - exiting 20:43:36 (4256): No heartbeat from core client for 30 sec - exiting 20:43:37 (4256): No heartbeat from core client for 30 sec - exiting 20:43:39 (4256): No heartbeat from core client for 30 sec - exiting 20:43:40 (4256): No heartbeat from core client for 30 sec - exiting 20:43:41 (4256): No heartbeat from core client for 30 sec - exiting 20:43:42 (4256): No heartbeat from core client for 30 sec - exiting 20:43:43 (4256): No heartbeat from core client for 30 sec - exiting 20:43:44 (4256): No heartbeat from core client for 30 sec - exiting 20:43:45 (4256): No heartbeat from core client for 30 sec - exiting 20:43:46 (4256): No heartbeat from core client for 30 sec - exiting 20:43:47 (4256): No heartbeat from core client for 30 sec - exiting 20:43:48 (4256): No heartbeat from core client for 30 sec - exiting 20:43:50 (4256): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:07:41 (4840): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:16:32 (4152): No heartbeat from core client for 30 sec - exiting 17:16:33 (4152): No heartbeat from core client for 30 sec - exiting 17:16:34 (4152): No heartbeat from core client for 30 sec - exiting 17:16:36 (4152): No heartbeat from core client for 30 sec - exiting 17:16:37 (4152): No heartbeat from core client for 30 sec - exiting 17:16:38 (4152): No heartbeat from core client for 30 sec - exiting 17:16:39 (4152): No heartbeat from core client for 30 sec - exiting 17:16:40 (4152): No heartbeat from core client for 30 sec - exiting 17:16:41 (4152): No heartbeat from core client for 30 sec - exiting 17:16:42 (4152): No heartbeat from core client for 30 sec - exiting 17:16:43 (4152): No heartbeat from core client for 30 sec - exiting 17:16:44 (4152): No heartbeat from core client for 30 sec - exiting 17:16:45 (4152): No heartbeat from core client for 30 sec - exiting 17:16:46 (4152): No heartbeat from core client for 30 sec - exiting 17:16:48 (4152): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:45:56 (4416): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:19:11 (14116): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:39:56 (9016): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2132, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5024, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5024, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... 20:37:36 (3740): No heartbeat from core client for 30 sec - exiting 20:37:37 (3740): No heartbeat from core client for 30 sec - exiting 20:37:38 (3740): No heartbeat from core client for 30 sec - exiting 20:37:39 (3740): No heartbeat from core client for 30 sec - exiting 20:37:40 (3740): No heartbeat from core client for 30 sec - exiting 20:37:41 (3740): No heartbeat from core client for 30 sec - exiting 20:37:42 (3740): No heartbeat from core client for 30 sec - exiting 20:37:43 (3740): No heartbeat from core client for 30 sec - exiting 20:37:44 (3740): No heartbeat from core client for 30 sec - exiting 20:37:45 (3740): No heartbeat from core client for 30 sec - exiting 20:37:47 (3740): No heartbeat from core client for 30 sec - exiting 20:37:48 (3740): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7704, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... 19:14:48 (4360): No heartbeat from core client for 30 sec - exiting 19:14:49 (4360): No heartbeat from core client for 30 sec - exiting 19:14:50 (4360): No heartbeat from core client for 30 sec - exiting 19:14:51 (4360): No heartbeat from core client for 30 sec - exiting 19:14:52 (4360): No heartbeat from core client for 30 sec - exiting 19:14:53 (4360): No heartbeat from core client for 30 sec - exiting 19:14:54 (4360): No heartbeat from core client for 30 sec - exiting 19:14:55 (4360): No heartbeat from core client for 30 sec - exiting 19:14:56 (4360): No heartbeat from core client for 30 sec - exiting 19:14:58 (4360): No heartbeat from core client for 30 sec - exiting 19:14:59 (4360): No heartbeat from core client for 30 sec - exiting 19:15:00 (4360): No heartbeat from core client for 30 sec - exiting 19:15:01 (4360): No heartbeat from core client for 30 sec - exiting 19:15:02 (4360): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 20:26:28 (2412): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:44:32 (4628): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 00:32:59 (25040): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 16:41:05 (4412): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 21:08:20 (4716): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
23 Oct 2013 10:47:29	1061327	15902628	hadcm3n_o4ls_1940_40_008380190_1	1,036,800	2,429,392	2.3432
20 Oct 2013 07:43:57	1061327	15902628	hadcm3n_o4ls_1940_40_008380190_1	1,010,880	2,335,215	2.3101
18 Oct 2013 09:35:32	1061327	15902628	hadcm3n_o4ls_1940_40_008380190_1	984,960	2,290,456	2.3254
14 Oct 2013 07:26:21	1061327	15902628	hadcm3n_o4ls_1940_40_008380190_1	959,040	2,201,348	2.2954
10 Oct 2013 03:23:12	1061327	15902628	hadcm3n_o4ls_1940_40_008380190_1	933,120	2,146,740	2.3006
05 Oct 2013 11:14:47	1061327	15902628	hadcm3n_o4ls_1940_40_008380190_1	907,200	2,074,378	2.2866
28 Sep 2013 07:24:05	1061327	15902628	hadcm3n_o4ls_1940_40_008380190_1	881,280	2,010,664	2.2815
27 Sep 2013 20:54:48	1061327	15902628	hadcm3n_o4ls_1940_40_008380190_1	855,360	1,973,359	2.3071
27 Sep 2013 10:43:26	1061327	15902628	hadcm3n_o4ls_1940_40_008380190_1	829,440	1,936,739	2.3350
26 Sep 2013 23:54:30	1061327	15902628	hadcm3n_o4ls_1940_40_008380190_1	803,520	1,897,912	2.3620
26 Sep 2013 14:19:58	1061327	15902628	hadcm3n_o4ls_1940_40_008380190_1	777,600	1,860,731	2.3929
26 Sep 2013 02:19:30	1061327	15902628	hadcm3n_o4ls_1940_40_008380190_1	751,680	1,809,235	2.4069
18 Sep 2013 05:26:19	1061327	15902628	hadcm3n_o4ls_1940_40_008380190_1	725,760	1,755,045	2.4182
18 Sep 2013 05:26:19	1061327	15902628	hadcm3n_o4ls_1940_40_008380190_1	725,760	1,755,045	2.4182
16 Sep 2013 20:09:55	1061327	15902628	hadcm3n_o4ls_1940_40_008380190_1	699,840	1,707,846	2.4403
15 Sep 2013 20:17:23	1061327	15902628	hadcm3n_o4ls_1940_40_008380190_1	673,920	1,652,093	2.4515
14 Sep 2013 00:44:54	1061327	15902628	hadcm3n_o4ls_1940_40_008380190_1	648,000	1,594,833	2.4612
13 Sep 2013 11:56:47	1061327	15902628	hadcm3n_o4ls_1940_40_008380190_1	622,080	1,548,895	2.4899
12 Sep 2013 11:25:13	1061327	15902628	hadcm3n_o4ls_1940_40_008380190_1	596,160	1,502,665	2.5206
10 Sep 2013 04:52:55	1061327	15902628	hadcm3n_o4ls_1940_40_008380190_1	570,240	1,453,420	2.5488