Task 15928719

Name	hadcm3n_n5mw_1920_40_008407100_0
Workunit	8557956
Created	20 Aug 2013, 9:51:10 UTC
Sent	20 Aug 2013, 9:54:32 UTC
Report deadline	19 Nov 2013, 17:21:43 UTC
Received	15 Oct 2013, 3:27:06 UTC
Server state	Over
Outcome	Success
Client state	Done
Exit status	0 (0x00000000)
Computer ID	1103902
Run time	54 days 20 hours 59 min 43 sec
CPU time	49 days 5 hours 13 min 18 sec
Validate state	Valid
Credit	12,441.60
Device peak FLOPS	1.52 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.10.58</core_client_version> <![CDATA[ <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... 09:06:42 (408): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:06:44 (408): No heartbeat from core client for 30 sec - exiting 09:06:45 (408): No heartbeat from core client for 30 sec - exiting 09:06:46 (408): No heartbeat from core client for 30 sec - exiting 09:06:47 (408): No heartbeat from core client for 30 sec - exiting 09:06:48 (408): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 13:16:51 (4356): No heartbeat from core client for 30 sec - exiting 13:16:53 (4356): No heartbeat from core client for 30 sec - exiting 13:16:54 (4356): No heartbeat from core client for 30 sec - exiting 13:16:55 (4356): No heartbeat from core client for 30 sec - exiting 13:18:05 (4356): No heartbeat from core client for 30 sec - exiting 13:18:06 (4356): No heartbeat from core client for 30 sec - exiting 13:18:07 (4356): No heartbeat from core client for 30 sec - exiting 13:18:08 (4356): No heartbeat from core client for 30 sec - exiting 13:18:09 (4356): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... zip error: Could not create output file (was replacing the original zip file) 15:35:37 (5872): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:35:38 (5872): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... 18:04:52 (3080): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:04:53 (3080): No heartbeat from core client for 30 sec - exiting 18:04:54 (3080): No heartbeat from core client for 30 sec - exiting 18:04:55 (3080): No heartbeat from core client for 30 sec - exiting Atmos Hold Restart file rename failed on atmos_restart.hold 22:05:11 (4736): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/n5mwko.pjd3c10 Error converting file to netcdf: dataout/n5mwko.pid3c10 Error converting file to netcdf: dataout/n5mwko.pfd3c10 Error converting file to netcdf: dataout/n5mwka.phd3c10 Error converting file to netcdf: dataout/n5mwka.pgd3c10 Error converting file to netcdf: dataout/n5mwka.ped3c10 Error converting file to netcdf: dataout/n5mwka.pdd3c10 Suspended CPDN Monitor - Suspend request from BOINC... 05:59:06 (840): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:59:07 (840): No heartbeat from core client for 30 sec - exiting 05:59:08 (840): No heartbeat from core client for 30 sec - exiting 05:59:09 (840): No heartbeat from core client for 30 sec - exiting 06:49:32 (4648): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:49:33 (4648): No heartbeat from core client for 30 sec - exiting 23:41:28 (5132): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:41:30 (5132): No heartbeat from core client for 30 sec - exiting 23:41:31 (5132): No heartbeat from core client for 30 sec - exiting 23:41:32 (5132): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... 00:58:03 (5340): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:58:04 (5340): No heartbeat from core client for 30 sec - exiting 09:09:19 (5128): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:09:21 (5128): No heartbeat from core client for 30 sec - exiting 09:09:22 (5128): No heartbeat from core client for 30 sec - exiting 09:09:23 (5128): No heartbeat from core client for 30 sec - exiting 09:09:24 (5128): No heartbeat from core client for 30 sec - exiting 14:16:38 (1540): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:16:40 (1540): No heartbeat from core client for 30 sec - exiting 02:22:05 (4772): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:22:07 (4772): No heartbeat from core client for 30 sec - exiting 02:22:08 (4772): No heartbeat from core client for 30 sec - exiting Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4768, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... 22:30:07 (4632): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:30:08 (4632): No heartbeat from core client for 30 sec - exiting 22:30:09 (4632): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 13:41:24 (5172): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:41:25 (5172): No heartbeat from core client for 30 sec - exiting 13:41:26 (5172): No heartbeat from core client for 30 sec - exiting 18:58:13 (4564): No heartbeat from core client for 30 sec - exiting 18:58:14 (4564): No heartbeat from core client for 30 sec - exiting 18:58:15 (4564): No heartbeat from core client for 30 sec - exiting 18:58:16 (4564): No heartbeat from core client for 30 sec - exiting 18:58:17 (4564): No heartbeat from core client for 30 sec - exiting 18:58:18 (4564): No heartbeat from core client for 30 sec - exiting 18:58:19 (4564): No heartbeat from core client for 30 sec - exiting 18:58:20 (4564): No heartbeat from core client for 30 sec - exiting 18:58:21 (4564): No heartbeat from core client for 30 sec - exiting 18:58:22 (4564): No heartbeat from core client for 30 sec - exiting 18:58:23 (4564): No heartbeat from core client for 30 sec - exiting 18:58:24 (4564): No heartbeat from core client for 30 sec - exiting 18:58:25 (4564): No heartbeat from core client for 30 sec - exiting 18:58:26 (4564): No heartbeat from core client for 30 sec - exiting 18:58:27 (4564): No heartbeat from core client for 30 sec - exiting 18:58:28 (4564): No heartbeat from core client for 30 sec - exiting 18:58:29 (4564): No heartbeat from core client for 30 sec - exiting 18:58:30 (4564): No heartbeat from core client for 30 sec - exiting 18:58:31 (4564): No heartbeat from core client for 30 sec - exiting 18:58:32 (4564): No heartbeat from core client for 30 sec - exiting 18:58:33 (4564): No heartbeat from core client for 30 sec - exiting 18:58:34 (4564): No heartbeat from core client for 30 sec - exiting 18:58:35 (4564): No heartbeat from core client for 30 sec - exiting 18:58:36 (4564): No heartbeat from core client for 30 sec - exiting 18:58:37 (4564): No heartbeat from core client for 30 sec - exiting 18:58:38 (4564): No heartbeat from core client for 30 sec - exiting 18:58:39 (4564): No heartbeat from core client for 30 sec - exiting 18:58:40 (4564): No heartbeat from core client for 30 sec - exiting 18:58:41 (4564): No heartbeat from core client for 30 sec - exiting 18:58:42 (4564): No heartbeat from core client for 30 sec - exiting 18:58:43 (4564): No heartbeat from core client for 30 sec - exiting 18:58:44 (4564): No heartbeat from core client for 30 sec - exiting 18:58:45 (4564): No heartbeat from core client for 30 sec - exiting 18:58:46 (4564): No heartbeat from core client for 30 sec - exiting 18:58:47 (4564): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... zip error: Could not create output file (was replacing the original zip file) Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4528, iMonCtr=1 Model crash detected, will try to restart... Ocean Restart file copy failed on n5mwko.daf28d0 Suspended CPDN Monitor - Suspend request from BOINC... Ocean Restart file copy failed on n5mwko.daf3cp0 Suspended CPDN Monitor - Suspend request from BOINC... 17:16:27 (4088): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:16:29 (4088): No heartbeat from core client for 30 sec - exiting 17:16:30 (4088): No heartbeat from core client for 30 sec - exiting 17:16:31 (4088): No heartbeat from core client for 30 sec - exiting 17:16:32 (4088): No heartbeat from core client for 30 sec - exiting 17:16:33 (4088): No heartbeat from core client for 30 sec - exiting 17:16:34 (4088): No heartbeat from core client for 30 sec - exiting 22:42:47 (6092): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:42:48 (6092): No heartbeat from core client for 30 sec - exiting 22:42:49 (6092): No heartbeat from core client for 30 sec - exiting Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4864, iMonCtr=1 Model crash detected, will try to restart... 23:53:39 (5280): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:53:41 (5280): No heartbeat from core client for 30 sec - exiting 00:44:31 (2040): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:44:32 (2040): No heartbeat from core client for 30 sec - exiting </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
14 Oct 2013 09:47:24	1103902	15928719	hadcm3n_n5mw_1920_40_008407100_0	1,036,800	4,250,868	4.1000
13 Oct 2013 03:27:28	1103902	15928719	hadcm3n_n5mw_1920_40_008407100_0	1,010,880	4,151,883	4.1072
11 Oct 2013 21:08:53	1103902	15928719	hadcm3n_n5mw_1920_40_008407100_0	984,960	4,052,290	4.1142
10 Oct 2013 14:45:59	1103902	15928719	hadcm3n_n5mw_1920_40_008407100_0	959,040	3,959,492	4.1286
09 Oct 2013 07:21:47	1103902	15928719	hadcm3n_n5mw_1920_40_008407100_0	933,120	3,857,507	4.1340
07 Oct 2013 19:40:38	1103902	15928719	hadcm3n_n5mw_1920_40_008407100_0	907,200	3,744,697	4.1278
06 Oct 2013 09:20:14	1103902	15928719	hadcm3n_n5mw_1920_40_008407100_0	881,280	3,633,793	4.1233
04 Oct 2013 23:39:52	1103902	15928719	hadcm3n_n5mw_1920_40_008407100_0	855,360	3,524,524	4.1205
03 Oct 2013 12:22:53	1103902	15928719	hadcm3n_n5mw_1920_40_008407100_0	829,440	3,411,070	4.1125
02 Oct 2013 03:01:22	1103902	15928719	hadcm3n_n5mw_1920_40_008407100_0	803,520	3,302,939	4.1106
30 Sep 2013 18:03:18	1103902	15928719	hadcm3n_n5mw_1920_40_008407100_0	777,600	3,196,701	4.1110
29 Sep 2013 08:08:22	1103902	15928719	hadcm3n_n5mw_1920_40_008407100_0	751,680	3,087,533	4.1075
27 Sep 2013 22:35:22	1103902	15928719	hadcm3n_n5mw_1920_40_008407100_0	725,760	2,978,630	4.1042
26 Sep 2013 12:39:05	1103902	15928719	hadcm3n_n5mw_1920_40_008407100_0	699,840	2,867,442	4.0973
25 Sep 2013 09:27:18	1103902	15928719	hadcm3n_n5mw_1920_40_008407100_0	673,920	2,757,104	4.0911
23 Sep 2013 15:39:10	1103902	15928719	hadcm3n_n5mw_1920_40_008407100_0	648,000	2,643,808	4.0800
23 Sep 2013 10:35:20	1103902	15928719	hadcm3n_n5mw_1920_40_008407100_0	622,080	2,532,320	4.0707
20 Sep 2013 21:41:56	1103902	15928719	hadcm3n_n5mw_1920_40_008407100_0	596,160	2,428,140	4.0730
19 Sep 2013 15:45:42	1103902	15928719	hadcm3n_n5mw_1920_40_008407100_0	570,240	2,329,981	4.0860
18 Sep 2013 10:08:44	1103902	15928719	hadcm3n_n5mw_1920_40_008407100_0	544,320	2,232,396	4.1013