Task 15870857

Name	hadcm3n_n7lj_1960_40_008340829_1
Workunit	8491690
Created	28 Jun 2013, 10:53:56 UTC
Sent	28 Jun 2013, 11:08:54 UTC
Report deadline	27 Sep 2013, 18:36:05 UTC
Received	20 Aug 2013, 9:54:32 UTC
Server state	Over
Outcome	Success
Client state	Done
Exit status	0 (0x00000000)
Computer ID	1103902
Run time	52 days 15 hours 34 min 13 sec
CPU time	47 days 9 hours 25 min 55 sec
Validate state	Valid
Credit	12,441.60
Device peak FLOPS	1.61 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.10.58</core_client_version> <![CDATA[ <stderr_txt> 04:34:52 (4776): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:34:54 (4776): No heartbeat from core client for 30 sec - exiting 04:34:55 (4776): No heartbeat from core client for 30 sec - exiting 04:34:56 (4776): No heartbeat from core client for 30 sec - exiting 16:36:20 (5268): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:36:22 (5268): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... 09:30:20 (4012): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:30:21 (4012): No heartbeat from core client for 30 sec - exiting 09:30:22 (4012): No heartbeat from core client for 30 sec - exiting 09:30:23 (4012): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Ocean Restart file copy failed on n7ljko.dag8980 Ocean Restart file copy failed on n7ljko.dah0ad0 07:44:43 (1500): No heartbeat from core client for 30 sec - exiting 07:44:44 (1500): No heartbeat from core client for 30 sec - exiting 07:44:45 (1500): No heartbeat from core client for 30 sec - exiting 07:44:46 (1500): No heartbeat from core client for 30 sec - exiting 07:44:47 (1500): No heartbeat from core client for 30 sec - exiting 07:44:48 (1500): No heartbeat from core client for 30 sec - exiting 07:44:49 (1500): No heartbeat from core client for 30 sec - exiting 07:44:50 (1500): No heartbeat from core client for 30 sec - exiting 07:44:51 (1500): No heartbeat from core client for 30 sec - exiting 07:44:52 (1500): No heartbeat from core client for 30 sec - exiting 07:44:53 (1500): No heartbeat from core client for 30 sec - exiting 07:44:54 (1500): No heartbeat from core client for 30 sec - exiting 07:44:55 (1500): No heartbeat from core client for 30 sec - exiting 07:44:56 (1500): No heartbeat from core client for 30 sec - exiting 07:44:57 (1500): No heartbeat from core client for 30 sec - exiting 07:44:58 (1500): No heartbeat from core client for 30 sec - exiting 07:44:59 (1500): No heartbeat from core client for 30 sec - exiting 07:45:00 (1500): No heartbeat from core client for 30 sec - exiting 07:45:01 (1500): No heartbeat from core client for 30 sec - exiting 07:45:02 (1500): No heartbeat from core client for 30 sec - exiting 07:45:03 (1500): No heartbeat from core client for 30 sec - exiting 07:45:04 (1500): No heartbeat from core client for 30 sec - exiting 07:45:05 (1500): No heartbeat from core client for 30 sec - exiting 07:45:06 (1500): No heartbeat from core client for 30 sec - exiting 07:45:07 (1500): No heartbeat from core client for 30 sec - exiting 07:45:08 (1500): No heartbeat from core client for 30 sec - exiting 07:45:09 (1500): No heartbeat from core client for 30 sec - exiting 07:45:10 (1500): No heartbeat from core client for 30 sec - exiting 07:45:11 (1500): No heartbeat from core client for 30 sec - exiting 07:45:12 (1500): No heartbeat from core client for 30 sec - exiting 07:45:13 (1500): No heartbeat from core client for 30 sec - exiting 07:45:14 (1500): No heartbeat from core client for 30 sec - exiting 07:45:15 (1500): No heartbeat from core client for 30 sec - exiting 07:45:16 (1500): No heartbeat from core client for 30 sec - exiting 07:45:17 (1500): No heartbeat from core client for 30 sec - exiting 07:45:18 (1500): No heartbeat from core client for 30 sec - exiting 07:45:19 (1500): No heartbeat from core client for 30 sec - exiting 07:45:20 (1500): No heartbeat from core client for 30 sec - exiting 07:45:21 (1500): No heartbeat from core client for 30 sec - exiting 07:45:22 (1500): No heartbeat from core client for 30 sec - exiting 07:45:23 (1500): No heartbeat from core client for 30 sec - exiting 07:45:24 (1500): No heartbeat from core client for 30 sec - exiting 07:45:25 (1500): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... zip error: Could not create output file (was replacing the original zip file) Suspended CPDN Monitor - Suspend request from BOINC... Atmos Hold Restart file rename failed on atmos_restart.hold 05:32:52 (4660): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:32:54 (4660): No heartbeat from core client for 30 sec - exiting 05:32:55 (4660): No heartbeat from core client for 30 sec - exiting 05:32:56 (4660): No heartbeat from core client for 30 sec - exiting 05:32:57 (4660): No heartbeat from core client for 30 sec - exiting 06:43:16 (4540): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:43:19 (4540): No heartbeat from core client for 30 sec - exiting 06:43:20 (4540): No heartbeat from core client for 30 sec - exiting 06:43:21 (4540): No heartbeat from core client for 30 sec - exiting 06:47:27 (532): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:48:31 (5988): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:12:29 (4600): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:12:31 (4600): No heartbeat from core client for 30 sec - exiting 07:12:32 (4600): No heartbeat from core client for 30 sec - exiting 07:17:42 (512): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:17:44 (512): No heartbeat from core client for 30 sec - exiting 07:17:45 (512): No heartbeat from core client for 30 sec - exiting 07:17:46 (512): No heartbeat from core client for 30 sec - exiting 18:13:06 (4184): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:13:07 (4184): No heartbeat from core client for 30 sec - exiting 18:13:08 (4184): No heartbeat from core client for 30 sec - exiting 18:13:09 (4184): No heartbeat from core client for 30 sec - exiting 01:19:07 (4900): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:19:08 (4900): No heartbeat from core client for 30 sec - exiting 01:19:09 (4900): No heartbeat from core client for 30 sec - exiting 08:25:02 (2212): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:25:03 (2212): No heartbeat from core client for 30 sec - exiting 08:25:04 (2212): No heartbeat from core client for 30 sec - exiting 08:25:05 (2212): No heartbeat from core client for 30 sec - exiting 19:46:23 (5440): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:46:24 (5440): No heartbeat from core client for 30 sec - exiting 19:46:25 (5440): No heartbeat from core client for 30 sec - exiting 19:46:26 (5440): No heartbeat from core client for 30 sec - exiting 20:22:03 (5496): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:22:05 (5496): No heartbeat from core client for 30 sec - exiting 20:22:06 (5496): No heartbeat from core client for 30 sec - exiting 20:24:12 (4444): No heartbeat from core client for 30 sec - exiting 20:24:13 (4444): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:02:59 (5632): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:03:01 (5632): No heartbeat from core client for 30 sec - exiting 10:03:02 (5632): No heartbeat from core client for 30 sec - exiting 10:03:03 (5632): No heartbeat from core client for 30 sec - exiting 10:03:04 (5632): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Ocean Restart file copy failed on n7ljko.dah58b0 09:53:01 (5268): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:53:03 (5268): No heartbeat from core client for 30 sec - exiting 09:53:04 (5268): No heartbeat from core client for 30 sec - exiting 04:06:17 (5244): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:06:19 (5244): No heartbeat from core client for 30 sec - exiting 04:06:20 (5244): No heartbeat from core client for 30 sec - exiting 04:06:21 (5244): No heartbeat from core client for 30 sec - exiting 04:06:22 (5244): No heartbeat from core client for 30 sec - exiting 04:06:23 (5244): No heartbeat from core client for 30 sec - exiting 04:06:24 (5244): No heartbeat from core client for 30 sec - exiting 12:33:04 (288): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:08:41 (5680): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:01:59 (5968): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:02:00 (5968): No heartbeat from core client for 30 sec - exiting 01:02:01 (5968): No heartbeat from core client for 30 sec - exiting 01:02:02 (5968): No heartbeat from core client for 30 sec - exiting 01:02:03 (5968): No heartbeat from core client for 30 sec - exiting 01:02:04 (5968): No heartbeat from core client for 30 sec - exiting 01:02:05 (5968): No heartbeat from core client for 30 sec - exiting 01:02:06 (5968): No heartbeat from core client for 30 sec - exiting 01:02:07 (5968): No heartbeat from core client for 30 sec - exiting 01:02:08 (5968): No heartbeat from core client for 30 sec - exiting 01:02:09 (5968): No heartbeat from core client for 30 sec - exiting 01:02:10 (5968): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... 02:02:55 (3400): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:02:56 (3400): No heartbeat from core client for 30 sec - exiting 02:13:12 (5712): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:13:13 (5712): No heartbeat from core client for 30 sec - exiting 02:13:14 (5712): No heartbeat from core client for 30 sec - exiting Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3612, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... 18:11:52 (4304): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:11:54 (4304): No heartbeat from core client for 30 sec - exiting 18:11:55 (4304): No heartbeat from core client for 30 sec - exiting 18:11:56 (4304): No heartbeat from core client for 30 sec - exiting 04:53:07 (5252): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:53:09 (5252): No heartbeat from core client for 30 sec - exiting 04:53:10 (5252): No heartbeat from core client for 30 sec - exiting 00:40:22 (2636): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:40:23 (2636): No heartbeat from core client for 30 sec - exiting 00:40:24 (2636): No heartbeat from core client for 30 sec - exiting 00:40:25 (2636): No heartbeat from core client for 30 sec - exiting 00:40:26 (2636): No heartbeat from core client for 30 sec - exiting 00:40:27 (2636): No heartbeat from core client for 30 sec - exiting Atmos Hold Restart file rename failed on atmos_restart.hold Suspended CPDN Monitor - Suspend request from BOINC... 00:48:17 (2596): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:48:18 (2596): No heartbeat from core client for 30 sec - exiting 00:48:19 (2596): No heartbeat from core client for 30 sec - exiting 00:48:20 (2596): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Ocean Restart file copy failed on n7ljko.dai95o0 21:03:47 (6004): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:03:48 (6004): No heartbeat from core client for 30 sec - exiting 21:03:49 (6004): No heartbeat from core client for 30 sec - exiting 21:03:50 (6004): No heartbeat from core client for 30 sec - exiting 21:11:52 (5080): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:11:54 (5080): No heartbeat from core client for 30 sec - exiting 21:11:55 (5080): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Atmos Hold Restart file rename failed on atmos_restart.hold Ocean Restart file copy failed on n7ljko.daj57n0 Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5404, iMonCtr=1 Model crash detected, will try to restart... 07:44:18 (5384): No heartbeat from core client for 30 sec - exiting 07:44:19 (5384): No heartbeat from core client for 30 sec - exiting 07:44:20 (5384): No heartbeat from core client for 30 sec - exiting 07:44:21 (5384): No heartbeat from core client for 30 sec - exiting 07:44:22 (5384): No heartbeat from core client for 30 sec - exiting 07:44:23 (5384): No heartbeat from core client for 30 sec - exiting 07:44:24 (5384): No heartbeat from core client for 30 sec - exiting 07:44:25 (5384): No heartbeat from core client for 30 sec - exiting 07:44:26 (5384): No heartbeat from core client for 30 sec - exiting 07:44:27 (5384): No heartbeat from core client for 30 sec - exiting 07:44:28 (5384): No heartbeat from core client for 30 sec - exiting 07:44:29 (5384): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:44:30 (5384): No heartbeat from core client for 30 sec - exiting </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
20 Aug 2013 08:58:12	1103902	15870857	hadcm3n_n7lj_1960_40_008340829_1	1,036,800	4,093,779	3.9485
19 Aug 2013 14:20:48	1103902	15870857	hadcm3n_n7lj_1960_40_008340829_1	1,010,880	4,028,625	3.9853
18 Aug 2013 19:46:55	1103902	15870857	hadcm3n_n7lj_1960_40_008340829_1	984,960	3,963,298	4.0238
18 Aug 2013 01:23:09	1103902	15870857	hadcm3n_n7lj_1960_40_008340829_1	959,040	3,897,639	4.0641
17 Aug 2013 03:53:46	1103902	15870857	hadcm3n_n7lj_1960_40_008340829_1	933,120	3,824,316	4.0984
15 Aug 2013 13:47:25	1103902	15870857	hadcm3n_n7lj_1960_40_008340829_1	907,200	3,712,270	4.0920
14 Aug 2013 17:38:32	1103902	15870857	hadcm3n_n7lj_1960_40_008340829_1	881,280	3,606,850	4.0927
14 Aug 2013 17:38:32	1103902	15870857	hadcm3n_n7lj_1960_40_008340829_1	855,360	3,509,121	4.1025
14 Aug 2013 17:38:32	1103902	15870857	hadcm3n_n7lj_1960_40_008340829_1	829,440	3,408,322	4.1092
14 Aug 2013 17:38:32	1103902	15870857	hadcm3n_n7lj_1960_40_008340829_1	803,520	3,308,107	4.1170
14 Aug 2013 17:38:32	1103902	15870857	hadcm3n_n7lj_1960_40_008340829_1	777,600	3,204,324	4.1208
14 Aug 2013 17:38:32	1103902	15870857	hadcm3n_n7lj_1960_40_008340829_1	751,680	3,096,310	4.1192
14 Aug 2013 17:38:32	1103902	15870857	hadcm3n_n7lj_1960_40_008340829_1	725,760	2,992,050	4.1226
14 Aug 2013 17:38:32	1103902	15870857	hadcm3n_n7lj_1960_40_008340829_1	699,840	2,885,788	4.1235
14 Aug 2013 17:38:32	1103902	15870857	hadcm3n_n7lj_1960_40_008340829_1	673,920	2,774,937	4.1176
14 Aug 2013 17:38:32	1103902	15870857	hadcm3n_n7lj_1960_40_008340829_1	648,000	2,655,915	4.0986
14 Aug 2013 17:38:32	1103902	15870857	hadcm3n_n7lj_1960_40_008340829_1	622,080	2,542,832	4.0876
30 Jul 2013 09:40:00	1103902	15870857	hadcm3n_n7lj_1960_40_008340829_1	596,160	2,435,563	4.0854
29 Jul 2013 13:22:46	1103902	15870857	hadcm3n_n7lj_1960_40_008340829_1	570,240	2,325,014	4.0773
29 Jul 2013 13:22:46	1103902	15870857	hadcm3n_n7lj_1960_40_008340829_1	544,320	2,206,880	4.0544