Task 15806566

Name	hadcm3n_4bpw_1940_40_008309762_3
Workunit	8460897
Created	30 May 2013, 10:59:16 UTC
Sent	20 Jun 2013, 6:56:19 UTC
Report deadline	19 Sep 2013, 14:23:30 UTC
Received	16 Aug 2013, 13:53:01 UTC
Server state	Over
Outcome	Computation error
Client state	Aborted by user
Exit status	203 (0x000000CB) EXIT_ABORTED_VIA_GUI
Computer ID	1369613
Run time	33 days 13 hours 33 min 41 sec
CPU time	22 days 2 hours 26 min 19 sec
Validate state	Invalid
Credit	9,953.28
Device peak FLOPS	2.67 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.64</core_client_version> <![CDATA[ <message> aborted by user </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4632, iMonCtr=1 Model crash detected, will try to restart... CSuspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 12:29:12 (5084): No heartbeat from core client for 30 sec - exiting 12:29:13 (5084): No heartbeat from core client for 30 sec - exiting 12:29:14 (5084): No heartbeat from core client for 30 sec - exiting 12:29:15 (5084): No heartbeat from core client for 30 sec - exiting 12:29:16 (5084): No heartbeat from core client for 30 sec - exiting 12:29:17 (5084): No heartbeat from core client for 30 sec - exiting 12:29:18 (5084): No heartbeat from core client for 30 sec - exiting 12:29:19 (5084): No heartbeat from core client for 30 sec - exiting 12:29:20 (5084): No heartbeat from core client for 30 sec - exiting 12:29:21 (5084): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:29:22 (5084): No heartbeat from core client for 30 sec - exiting 12:29:23 (5084): No heartbeat from core client for 30 sec - exiting 12:29:24 (5084): No heartbeat from core client for 30 sec - exiting 12:29:25 (5084): No heartbeat from core client for 30 sec - exiting 12:29:26 (5084): No heartbeat from core client for 30 sec - exiting 12:29:27 (5084): No heartbeat from core client for 30 sec - exiting 12:29:28 (5084): No heartbeat from core client for 30 sec - exiting 12:29:29 (5084): No heartbeat from core client for 30 sec - exiting 12:29:30 (5084): No heartbeat from core client for 30 sec - exiting 12:29:31 (5084): No heartbeat from core client for 30 sec - exiting 12:29:32 (5084): No heartbeat from core client for 30 sec - exiting 12:29:33 (5084): No heartbeat from core client for 30 sec - exiting 12:29:34 (5084): No heartbeat from core client for 30 sec - exiting 12:29:35 (5084): No heartbeat from core client for 30 sec - exiting 12:29:36 (5084): No heartbeat from core client for 30 sec - exiting 12:29:37 (5084): No heartbeat from core client for 30 sec - exiting 12:29:38 (5084): No heartbeat from core client for 30 sec - exiting 12:29:39 (5084): No heartbeat from core client for 30 sec - exiting 12:29:40 (5084): No heartbeat from core client for 30 sec - exiting 12:29:41 (5084): No heartbeat from core client for 30 sec - exiting 12:29:42 (5084): No heartbeat from core client for 30 sec - exiting 12:29:43 (5084): No heartbeat from core client for 30 sec - exiting 12:29:44 (5084): No heartbeat from core client for 30 sec - exiting 12:29:45 (5084): No heartbeat from core client for 30 sec - exiting 12:29:46 (5084): No heartbeat from core client for 30 sec - exiting 12:29:47 (5084): No heartbeat from core client for 30 sec - exiting 12:29:48 (5084): No heartbeat from core client for 30 sec - exiting 12:29:49 (5084): No heartbeat from core client for 30 sec - exiting 12:29:50 (5084): No heartbeat from core client for 30 sec - exiting 12:29:51 (5084): No heartbeat from core client for 30 sec - exiting 12:29:52 (5084): No heartbeat from core client for 30 sec - exiting 12:29:53 (5084): No heartbeat from core client for 30 sec - exiting 12:29:54 (5084): No heartbeat from core client for 30 sec - exiting 12:29:55 (5084): No heartbeat from core client for 30 sec - exiting 12:29:56 (5084): No heartbeat from core client for 30 sec - exiting 12:29:57 (5084): No heartbeat from core client for 30 sec - exiting 12:29:58 (5084): No heartbeat from core client for 30 sec - exiting 12:29:59 (5084): No heartbeat from core client for 30 sec - exiting 12:30:00 (5084): No heartbeat from core client for 30 sec - exiting 12:30:01 (5084): No heartbeat from core client for 30 sec - exiting 12:30:02 (5084): No heartbeat from core client for 30 sec - exiting 12:30:03 (5084): No heartbeat from core client for 30 sec - exiting 12:30:04 (5084): No heartbeat from core client for 30 sec - exiting 12:30:05 (5084): No heartbeat from core client for 30 sec - exiting 12:30:06 (5084): No heartbeat from core client for 30 sec - exiting 12:30:07 (5084): No heartbeat from core client for 30 sec - exiting 12:30:08 (5084): No heartbeat from core client for 30 sec - exiting 12:30:09 (5084): No heartbeat from core client for 30 sec - exiting 12:30:10 (5084): No heartbeat from core client for 30 sec - exiting 12:30:11 (5084): No heartbeat from core client for 30 sec - exiting 12:30:12 (5084): No heartbeat from core client for 30 sec - exiting 12:30:13 (5084): No heartbeat from core client for 30 sec - exiting 12:30:14 (5084): No heartbeat from core client for 30 sec - exiting 12:30:15 (5084): No heartbeat from core client for 30 sec - exiting 12:30:16 (5084): No heartbeat from core client for 30 sec - exiting 12:30:17 (5084): No heartbeat from core client for 30 sec - exiting 12:30:18 (5084): No heartbeat from core client for 30 sec - exiting 12:30:19 (5084): No heartbeat from core client for 30 sec - exiting 12:30:20 (5084): No heartbeat from core client for 30 sec - exiting 12:30:21 (5084): No heartbeat from core client for 30 sec - exiting 12:30:22 (5084): No heartbeat from core client for 30 sec - exiting 12:30:23 (5084): No heartbeat from core client for 30 sec - exiting 12:30:24 (5084): No heartbeat from core client for 30 sec - exiting 12:30:25 (5084): No heartbeat from core client for 30 sec - exiting 12:30:26 (5084): No heartbeat from core client for 30 sec - exiting 12:30:27 (5084): No heartbeat from core client for 30 sec - exiting 12:30:28 (5084): No heartbeat from core client for 30 sec - exiting 12:30:29 (5084): No heartbeat from core client for 30 sec - exiting 12:30:30 (5084): No heartbeat from core client for 30 sec - exiting 12:30:31 (5084): No heartbeat from core client for 30 sec - exiting 12:30:32 (5084): No heartbeat from core client for 30 sec - exiting 12:30:33 (5084): No heartbeat from core client for 30 sec - exiting 12:30:34 (5084): No heartbeat from core client for 30 sec - exiting 12:30:35 (5084): No heartbeat from core client for 30 sec - exiting 12:30:36 (5084): No heartbeat from core client for 30 sec - exiting 12:30:37 (5084): No heartbeat from core client for 30 sec - exiting 12:30:38 (5084): No heartbeat from core client for 30 sec - exiting 12:30:39 (5084): No heartbeat from core client for 30 sec - exiting 12:30:40 (5084): No heartbeat from core client for 30 sec - exiting 12:30:41 (5084): No heartbeat from core client for 30 sec - exiting 12:30:42 (5084): No heartbeat from core client for 30 sec - exiting 12:30:43 (5084): No heartbeat from core client for 30 sec - exiting 12:30:44 (5084): No heartbeat from core client for 30 sec - exiting 12:30:45 (5084): No heartbeat from core client for 30 sec - exiting 12:34:23 (3532): No heartbeat from core client for 30 sec - exiting 12:34:24 (3532): No heartbeat from core client for 30 sec - exiting 12:34:25 (3532): No heartbeat from core client for 30 sec - exiting 12:34:26 (3532): No heartbeat from core client for 30 sec - exiting 12:34:27 (3532): No heartbeat from core client for 30 sec - exiting 12:34:28 (3532): No heartbeat from core client for 30 sec - exiting 12:34:29 (3532): No heartbeat from core client for 30 sec - exiting 12:34:30 (3532): No heartbeat from core client for 30 sec - exiting 12:34:31 (3532): No heartbeat from core client for 30 sec - exiting 12:34:32 (3532): No heartbeat from core client for 30 sec - exiting 12:34:33 (3532): No heartbeat from core client for 30 sec - exiting 12:34:34 (3532): No heartbeat from core client for 30 sec - exiting 12:34:35 (3532): No heartbeat from core client for 30 sec - exiting 12:34:36 (3532): No heartbeat from core client for 30 sec - exiting 12:34:37 (3532): No heartbeat from core client for 30 sec - exiting 12:34:38 (3532): No heartbeat from core client for 30 sec - exiting 12:34:39 (3532): No heartbeat from core client for 30 sec - exiting 12:34:40 (3532): No heartbeat from core client for 30 sec - exiting 12:34:41 (3532): No heartbeat from core client for 30 sec - exiting 12:34:42 (3532): No heartbeat from core client for 30 sec - exiting 12:34:43 (3532): No heartbeat from core client for 30 sec - exiting 12:34:44 (3532): No heartbeat from core client for 30 sec - exiting 12:34:45 (3532): No heartbeat from core client for 30 sec - exiting 12:34:46 (3532): No heartbeat from core client for 30 sec - exiting 12:34:47 (3532): No heartbeat from core client for 30 sec - exiting 12:34:49 (3532): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/4bpwko.pjf5c10 Error converting file to netcdf: dataout/4bpwko.pif5c10 Error converting file to netcdf: dataout/4bpwko.pff5c10 Error converting file to netcdf: dataout/4bpwka.phf5c10 Error converting file to netcdf: dataout/4bpwka.pgf5c10 Error converting file to netcdf: dataout/4bpwka.pef5c10 Error converting file to netcdf: dataout/4bpwka.pdf5c10 05:18:50 (4348): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:18:54 (4348): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 14:24:15 (4608): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:24:16 (4608): No heartbeat from core client for 30 sec - exiting 14:24:17 (4608): No heartbeat from core client for 30 sec - exiting 14:24:18 (4608): No heartbeat from core client for 30 sec - exiting 14:24:19 (4608): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Ocean Restart file copy failed on 4bpwko.dag0c20 Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/4bpwko.pjg2c10 Error converting file to netcdf: dataout/4bpwko.pig2c10 Error converting file to netcdf: dataout/4bpwko.pfg2c10 Error converting file to netcdf: dataout/4bpwka.phg2c10 Error converting file to netcdf: dataout/4bpwka.pgg2c10 Error converting file to netcdf: dataout/4bpwka.peg2c10 Error converting file to netcdf: dataout/4bpwka.pdg2c10 CPDN Monitor - Quit request from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 CPDN Monitor - Quit request from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 09:11:09 (1864): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4440, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Abort request from BOINC... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
16 Aug 2013 05:18:43	1038662	15806566	hadcm3n_4bpw_1940_40_008309762_3	829,440	1,881,003	2.2678
14 Aug 2013 15:47:25	1038662	15806566	hadcm3n_4bpw_1940_40_008309762_3	803,520	1,822,385	2.2680
14 Aug 2013 15:47:25	1038662	15806566	hadcm3n_4bpw_1940_40_008309762_3	777,600	1,764,333	2.2689
14 Aug 2013 15:47:25	1038662	15806566	hadcm3n_4bpw_1940_40_008309762_3	751,680	1,705,433	2.2688
14 Aug 2013 15:47:25	1038662	15806566	hadcm3n_4bpw_1940_40_008309762_3	725,760	1,648,752	2.2718
14 Aug 2013 15:47:25	1038662	15806566	hadcm3n_4bpw_1940_40_008309762_3	699,840	1,588,918	2.2704
14 Aug 2013 15:47:25	1038662	15806566	hadcm3n_4bpw_1940_40_008309762_3	673,920	1,530,887	2.2716
14 Aug 2013 15:47:25	1038662	15806566	hadcm3n_4bpw_1940_40_008309762_3	648,000	1,473,625	2.2741
14 Aug 2013 15:47:25	1038662	15806566	hadcm3n_4bpw_1940_40_008309762_3	622,080	1,415,012	2.2746
14 Aug 2013 15:47:25	1038662	15806566	hadcm3n_4bpw_1940_40_008309762_3	596,160	1,360,616	2.2823
14 Aug 2013 15:47:25	1038662	15806566	hadcm3n_4bpw_1940_40_008309762_3	570,240	1,306,222	2.2907
14 Aug 2013 15:47:25	1038662	15806566	hadcm3n_4bpw_1940_40_008309762_3	544,320	1,251,591	2.2994
30 Jul 2013 09:47:17	1038662	15806566	hadcm3n_4bpw_1940_40_008309762_3	518,400	1,197,830	2.3106
29 Jul 2013 13:02:44	1038662	15806566	hadcm3n_4bpw_1940_40_008309762_3	492,480	1,143,899	2.3227
29 Jul 2013 13:02:44	1038662	15806566	hadcm3n_4bpw_1940_40_008309762_3	466,560	1,085,180	2.3259
26 Jul 2013 04:40:57	1038662	15806566	hadcm3n_4bpw_1940_40_008309762_3	440,640	1,024,149	2.3242
25 Jul 2013 00:05:50	1038662	15806566	hadcm3n_4bpw_1940_40_008309762_3	414,720	964,202	2.3249
23 Jul 2013 22:15:38	1038662	15806566	hadcm3n_4bpw_1940_40_008309762_3	388,800	906,257	2.3309
23 Jul 2013 21:41:54	1038662	15806566	hadcm3n_4bpw_1940_40_008309762_3	362,880	844,864	2.3282
23 Jul 2013 20:52:58	1038662	15806566	hadcm3n_4bpw_1940_40_008309762_3	336,960	781,647	2.3197