Task 15928716

Name	hadcm3n_o7cg_2060_40_008407097_0
Workunit	8557953
Created	20 Aug 2013, 9:50:03 UTC
Sent	20 Aug 2013, 9:54:32 UTC
Report deadline	19 Nov 2013, 17:21:43 UTC
Received	15 Oct 2013, 3:27:06 UTC
Server state	Over
Outcome	Success
Client state	Done
Exit status	0 (0x00000000)
Computer ID	1103902
Run time	54 days 12 hours 40 min 24 sec
CPU time	49 days 1 hours 23 min 48 sec
Validate state	Valid
Credit	12,441.60
Device peak FLOPS	1.52 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.10.58</core_client_version> <![CDATA[ <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... 09:06:42 (4188): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:06:44 (4188): No heartbeat from core client for 30 sec - exiting 09:06:45 (4188): No heartbeat from core client for 30 sec - exiting 09:06:46 (4188): No heartbeat from core client for 30 sec - exiting 09:06:47 (4188): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Ocean Restart file copy failed on o7cgko.daq4bf0 Ocean Restart file copy failed on o7cgko.daq57g0 Suspended CPDN Monitor - Suspend request from BOINC... 13:16:50 (4400): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:16:51 (4400): No heartbeat from core client for 30 sec - exiting 13:16:52 (4400): No heartbeat from core client for 30 sec - exiting 15:35:36 (6088): No heartbeat from core client for 30 sec - exiting 15:35:37 (6088): No heartbeat from core client for 30 sec - exiting 15:35:38 (6088): No heartbeat from core client for 30 sec - exiting 15:35:39 (6088): No heartbeat from core client for 30 sec - exiting 15:35:40 (6088): No heartbeat from core client for 30 sec - exiting 15:35:41 (6088): No heartbeat from core client for 30 sec - exiting 15:35:42 (6088): No heartbeat from core client for 30 sec - exiting 15:35:43 (6088): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... zip error: Could not create output file (was replacing the original zip file) Suspended CPDN Monitor - Suspend request from BOINC... 18:04:52 (6136): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:04:53 (6136): No heartbeat from core client for 30 sec - exiting 18:04:54 (6136): No heartbeat from core client for 30 sec - exiting Atmos Hold Restart file rename failed on atmos_restart.hold 22:05:10 (2440): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:05:11 (2440): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Ocean Restart file copy failed on o7cgko.dar7310 05:59:06 (580): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:59:08 (580): No heartbeat from core client for 30 sec - exiting 05:59:09 (580): No heartbeat from core client for 30 sec - exiting 05:59:10 (580): No heartbeat from core client for 30 sec - exiting 06:49:32 (2500): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:49:33 (2500): No heartbeat from core client for 30 sec - exiting 06:49:34 (2500): No heartbeat from core client for 30 sec - exiting Ocean Restart file copy failed on o7cgko.dar88s0 23:41:28 (4608): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:41:30 (4608): No heartbeat from core client for 30 sec - exiting 23:41:31 (4608): No heartbeat from core client for 30 sec - exiting 23:41:32 (4608): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... 00:58:03 (596): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:58:04 (596): No heartbeat from core client for 30 sec - exiting 00:58:05 (596): No heartbeat from core client for 30 sec - exiting 09:09:19 (5052): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:09:21 (5052): No heartbeat from core client for 30 sec - exiting 09:09:22 (5052): No heartbeat from core client for 30 sec - exiting 09:09:23 (5052): No heartbeat from core client for 30 sec - exiting 09:09:24 (5052): No heartbeat from core client for 30 sec - exiting 14:16:38 (480): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:16:40 (480): No heartbeat from core client for 30 sec - exiting 14:16:41 (480): No heartbeat from core client for 30 sec - exiting 14:16:42 (480): No heartbeat from core client for 30 sec - exiting 14:16:43 (480): No heartbeat from core client for 30 sec - exiting 14:16:44 (480): No heartbeat from core client for 30 sec - exiting 14:16:45 (480): No heartbeat from core client for 30 sec - exiting 14:16:46 (480): No heartbeat from core client for 30 sec - exiting 14:16:47 (480): No heartbeat from core client for 30 sec - exiting 14:16:48 (480): No heartbeat from core client for 30 sec - exiting 14:16:49 (480): No heartbeat from core client for 30 sec - exiting 14:16:50 (480): No heartbeat from core client for 30 sec - exiting 02:22:05 (5868): No heartbeat from core client for 30 sec - exiting 02:22:06 (5868): No heartbeat from core client for 30 sec - exiting 02:22:07 (5868): No heartbeat from core client for 30 sec - exiting 02:22:08 (5868): No heartbeat from core client for 30 sec - exiting 02:22:09 (5868): No heartbeat from core client for 30 sec - exiting 02:22:10 (5868): No heartbeat from core client for 30 sec - exiting 02:22:11 (5868): No heartbeat from core client for 30 sec - exiting 02:22:12 (5868): No heartbeat from core client for 30 sec - exiting 02:22:13 (5868): No heartbeat from core client for 30 sec - exiting 02:22:14 (5868): No heartbeat from core client for 30 sec - exiting 02:22:15 (5868): No heartbeat from core client for 30 sec - exiting 02:22:16 (5868): No heartbeat from core client for 30 sec - exiting 02:22:17 (5868): No heartbeat from core client for 30 sec - exiting 02:22:18 (5868): No heartbeat from core client for 30 sec - exiting 02:22:19 (5868): No heartbeat from core client for 30 sec - exiting 02:22:20 (5868): No heartbeat from core client for 30 sec - exiting 02:22:21 (5868): No heartbeat from core client for 30 sec - exiting 02:22:22 (5868): No heartbeat from core client for 30 sec - exiting 02:22:23 (5868): No heartbeat from core client for 30 sec - exiting 02:22:24 (5868): No heartbeat from core client for 30 sec - exiting 02:22:25 (5868): No heartbeat from core client for 30 sec - exiting 02:22:26 (5868): No heartbeat from core client for 30 sec - exiting 02:22:27 (5868): No heartbeat from core client for 30 sec - exiting 02:22:28 (5868): No heartbeat from core client for 30 sec - exiting 02:22:29 (5868): No heartbeat from core client for 30 sec - exiting 02:22:30 (5868): No heartbeat from core client for 30 sec - exiting 02:22:31 (5868): No heartbeat from core client for 30 sec - exiting 02:22:32 (5868): No heartbeat from core client for 30 sec - exiting 02:22:33 (5868): No heartbeat from core client for 30 sec - exiting 02:22:34 (5868): No heartbeat from core client for 30 sec - exiting 02:22:35 (5868): No heartbeat from core client for 30 sec - exiting 02:22:36 (5868): No heartbeat from core client for 30 sec - exiting 02:22:37 (5868): No heartbeat from core client for 30 sec - exiting 02:22:38 (5868): No heartbeat from core client for 30 sec - exiting 02:22:39 (5868): No heartbeat from core client for 30 sec - exiting 02:22:40 (5868): No heartbeat from core client for 30 sec - exiting 02:22:41 (5868): No heartbeat from core client for 30 sec - exiting 02:22:42 (5868): No heartbeat from core client for 30 sec - exiting 02:22:43 (5868): No heartbeat from core client for 30 sec - exiting 02:22:44 (5868): No heartbeat from core client for 30 sec - exiting 02:22:45 (5868): No heartbeat from core client for 30 sec - exiting 02:22:46 (5868): No heartbeat from core client for 30 sec - exiting 02:22:47 (5868): No heartbeat from core client for 30 sec - exiting 02:22:48 (5868): No heartbeat from core client for 30 sec - exiting 02:22:49 (5868): No heartbeat from core client for 30 sec - exiting 02:22:50 (5868): No heartbeat from core client for 30 sec - exiting 02:22:51 (5868): No heartbeat from core client for 30 sec - exiting 02:22:52 (5868): No heartbeat from core client for 30 sec - exiting 02:22:53 (5868): No heartbeat from core client for 30 sec - exiting 02:22:54 (5868): No heartbeat from core client for 30 sec - exiting 02:22:55 (5868): No heartbeat from core client for 30 sec - exiting 02:22:56 (5868): No heartbeat from core client for 30 sec - exiting 02:22:57 (5868): No heartbeat from core client for 30 sec - exiting 02:22:58 (5868): No heartbeat from core client for 30 sec - exiting 02:22:59 (5868): No heartbeat from core client for 30 sec - exiting 02:23:00 (5868): No heartbeat from core client for 30 sec - exiting 02:23:01 (5868): No heartbeat from core client for 30 sec - exiting 02:23:02 (5868): No heartbeat from core client for 30 sec - exiting 02:23:03 (5868): No heartbeat from core client for 30 sec - exiting 02:23:04 (5868): No heartbeat from core client for 30 sec - exiting 02:23:05 (5868): No heartbeat from core client for 30 sec - exiting 02:23:06 (5868): No heartbeat from core client for 30 sec - exiting 02:23:07 (5868): No heartbeat from core client for 30 sec - exiting 02:23:08 (5868): No heartbeat from core client for 30 sec - exiting 02:23:09 (5868): No heartbeat from core client for 30 sec - exiting 02:23:10 (5868): No heartbeat from core client for 30 sec - exiting 02:23:11 (5868): No heartbeat from core client for 30 sec - exiting 02:23:12 (5868): No heartbeat from core client for 30 sec - exiting 02:23:13 (5868): No heartbeat from core client for 30 sec - exiting 02:23:14 (5868): No heartbeat from core client for 30 sec - exiting 02:23:15 (5868): No heartbeat from core client for 30 sec - exiting 02:23:16 (5868): No heartbeat from core client for 30 sec - exiting 02:23:17 (5868): No heartbeat from core client for 30 sec - exiting 02:23:18 (5868): No heartbeat from core client for 30 sec - exiting 02:23:19 (5868): No heartbeat from core client for 30 sec - exiting 02:23:20 (5868): No heartbeat from core client for 30 sec - exiting 02:23:21 (5868): No heartbeat from core client for 30 sec - exiting 02:23:22 (5868): No heartbeat from core client for 30 sec - exiting 02:23:23 (5868): No heartbeat from core client for 30 sec - exiting 02:23:24 (5868): No heartbeat from core client for 30 sec - exiting 02:23:25 (5868): No heartbeat from core client for 30 sec - exiting 02:23:26 (5868): No heartbeat from core client for 30 sec - exiting 02:23:27 (5868): No heartbeat from core client for 30 sec - exiting 02:23:28 (5868): No heartbeat from core client for 30 sec - exiting 02:23:29 (5868): No heartbeat from core client for 30 sec - exiting 02:23:30 (5868): No heartbeat from core client for 30 sec - exiting 02:23:31 (5868): No heartbeat from core client for 30 sec - exiting 02:23:32 (5868): No heartbeat from core client for 30 sec - exiting 02:23:33 (5868): No heartbeat from core client for 30 sec - exiting 02:23:34 (5868): No heartbeat from core client for 30 sec - exiting 02:23:35 (5868): No heartbeat from core client for 30 sec - exiting 02:23:36 (5868): No heartbeat from core client for 30 sec - exiting 02:23:37 (5868): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... zip error: Could not create output file (was replacing the original zip file) Atmos Hold Restart file rename failed on atmos_restart.hold Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5088, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... 22:30:07 (4624): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/o7cgko.pjs6c10 Error converting file to netcdf: dataout/o7cgko.pis6c10 Error converting file to netcdf: dataout/o7cgko.pfs6c10 Error converting file to netcdf: dataout/o7cgka.phs6c10 Error converting file to netcdf: dataout/o7cgka.pgs6c10 Error converting file to netcdf: dataout/o7cgka.pes6c10 Error converting file to netcdf: dataout/o7cgka.pds6c10 Suspended CPDN Monitor - Suspend request from BOINC... Ocean Restart file copy failed on o7cgko.das71e0 Ocean Restart file copy failed on o7cgko.das86f0 Suspended CPDN Monitor - Suspend request from BOINC... Ocean Restart file copy failed on o7cgko.dat0860 13:41:25 (5072): No heartbeat from core client for 30 sec - exiting 13:41:26 (5072): No heartbeat from core client for 30 sec - exiting 13:41:27 (5072): No heartbeat from core client for 30 sec - exiting 13:41:28 (5072): No heartbeat from core client for 30 sec - exiting 13:41:29 (5072): No heartbeat from core client for 30 sec - exiting 13:41:30 (5072): No heartbeat from core client for 30 sec - exiting 13:41:31 (5072): No heartbeat from core client for 30 sec - exiting 13:41:32 (5072): No heartbeat from core client for 30 sec - exiting 13:41:33 (5072): No heartbeat from core client for 30 sec - exiting 13:41:34 (5072): No heartbeat from core client for 30 sec - exiting 13:41:35 (5072): No heartbeat from core client for 30 sec - exiting 13:41:36 (5072): No heartbeat from core client for 30 sec - exiting 13:41:37 (5072): No heartbeat from core client for 30 sec - exiting 13:41:38 (5072): No heartbeat from core client for 30 sec - exiting 13:41:39 (5072): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... zip error: Could not create output file (was replacing the original zip file) Atmos Hold Restart file rename failed on atmos_restart.hold 18:58:12 (3612): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:58:13 (3612): No heartbeat from core client for 30 sec - exiting Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=376, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Ocean Restart file copy failed on o7cgko.dat71h0 Suspended CPDN Monitor - Suspend request from BOINC... 17:16:27 (2812): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:16:29 (2812): No heartbeat from core client for 30 sec - exiting 17:16:30 (2812): No heartbeat from core client for 30 sec - exiting 17:16:31 (2812): No heartbeat from core client for 30 sec - exiting 17:16:32 (2812): No heartbeat from core client for 30 sec - exiting 17:16:33 (2812): No heartbeat from core client for 30 sec - exiting 17:16:34 (2812): No heartbeat from core client for 30 sec - exiting 22:42:47 (5476): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:42:48 (5476): No heartbeat from core client for 30 sec - exiting Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5636, iMonCtr=1 Model crash detected, will try to restart... 23:53:39 (5272): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:53:40 (5272): No heartbeat from core client for 30 sec - exiting 00:44:31 (5932): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:44:32 (5932): No heartbeat from core client for 30 sec - exiting </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
14 Oct 2013 01:08:19	1103902	15928716	hadcm3n_o7cg_2060_40_008407097_0	1,036,800	4,237,188	4.0868
12 Oct 2013 17:37:31	1103902	15928716	hadcm3n_o7cg_2060_40_008407097_0	1,010,880	4,136,071	4.0916
11 Oct 2013 12:22:56	1103902	15928716	hadcm3n_o7cg_2060_40_008407097_0	984,960	4,041,798	4.1035
10 Oct 2013 05:59:27	1103902	15928716	hadcm3n_o7cg_2060_40_008407097_0	959,040	3,949,423	4.1181
08 Oct 2013 21:37:21	1103902	15928716	hadcm3n_o7cg_2060_40_008407097_0	933,120	3,843,865	4.1194
07 Oct 2013 11:27:27	1103902	15928716	hadcm3n_o7cg_2060_40_008407097_0	907,200	3,731,977	4.1137
06 Oct 2013 01:50:56	1103902	15928716	hadcm3n_o7cg_2060_40_008407097_0	881,280	3,622,387	4.1104
04 Oct 2013 16:19:05	1103902	15928716	hadcm3n_o7cg_2060_40_008407097_0	855,360	3,512,591	4.1066
03 Oct 2013 05:54:55	1103902	15928716	hadcm3n_o7cg_2060_40_008407097_0	829,440	3,399,240	4.0982
01 Oct 2013 21:20:54	1103902	15928716	hadcm3n_o7cg_2060_40_008407097_0	803,520	3,292,207	4.0972
30 Sep 2013 12:42:45	1103902	15928716	hadcm3n_o7cg_2060_40_008407097_0	777,600	3,186,881	4.0984
29 Sep 2013 03:16:53	1103902	15928716	hadcm3n_o7cg_2060_40_008407097_0	751,680	3,077,685	4.0944
27 Sep 2013 17:38:14	1103902	15928716	hadcm3n_o7cg_2060_40_008407097_0	725,760	2,967,975	4.0895
26 Sep 2013 08:11:59	1103902	15928716	hadcm3n_o7cg_2060_40_008407097_0	699,840	2,858,826	4.0850
25 Sep 2013 09:24:36	1103902	15928716	hadcm3n_o7cg_2060_40_008407097_0	673,920	2,746,369	4.0752
23 Sep 2013 11:42:40	1103902	15928716	hadcm3n_o7cg_2060_40_008407097_0	648,000	2,635,971	4.0679
22 Sep 2013 01:23:15	1103902	15928716	hadcm3n_o7cg_2060_40_008407097_0	622,080	2,524,390	4.0580
20 Sep 2013 18:15:54	1103902	15928716	hadcm3n_o7cg_2060_40_008407097_0	596,160	2,422,086	4.0628
19 Sep 2013 12:38:23	1103902	15928716	hadcm3n_o7cg_2060_40_008407097_0	570,240	2,324,776	4.0768
18 Sep 2013 07:07:08	1103902	15928716	hadcm3n_o7cg_2060_40_008407097_0	544,320	2,227,115	4.0916