Task 13610655

Name	hadcm3n_o4np_1980_40_007540011_0
Workunit	7737243
Created	6 Nov 2011, 4:53:15 UTC
Sent	6 Nov 2011, 13:20:20 UTC
Report deadline	5 Feb 2012, 20:47:31 UTC
Received	13 Dec 2011, 23:33:43 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	25 (0x00000019) Unknown error code
Computer ID	1065995
Run time	11 days 16 hours 44 min 52 sec
CPU time	8 days 5 hours 13 min 51 sec
Validate state	Invalid
Credit	5,287.68
Device peak FLOPS	2.77 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.10.58</core_client_version> <![CDATA[ <message> The drive cannot locate a specific area or track on the disk. (0x19) - exit code 25 (0x19) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... 22:29:06 (5672): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:35:12 (5892): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 10:23:09 (7508): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:17:03 (6776): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:17:30 (4536): No heartbeat from core client for 30 sec - exiting 20:17:31 (4536): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... C14:56:05 (4956): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 17:38:10 (7120): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:51:05 (5044): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6880, iMonCtr=1 Model crash detected, will try to restart... 02:02:45 (4968): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 11:14:54 (5384): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:54:13 (7724): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:00:31 (7440): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:25:39 (2864): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 19:09:06 (1296): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:10:50 (1128): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:49:11 (5812): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:51:03 (4536): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:29:57 (4380): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:59:41 (3536): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:28:51 (5532): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:44:49 (4216): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 17:29:50 (2680): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:01:08 (5872): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:01:47 (4892): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:58:52 (7324): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:13:01 (4484): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:28:09 (5936): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:51:01 (6788): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:54:54 (7620): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:57:47 (7236): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:37:43 (2736): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 00:24:35 (1232): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:36:26 (4740): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7488, iMonCtr=1 Model crash detected, will try to restart... 09:35:23 (4916): No heartbeat from core client for 30 sec - exiting 09:35:24 (4916): No heartbeat from core client for 30 sec - exiting 09:35:25 (4916): No heartbeat from core client for 30 sec - exiting 09:35:26 (4916): No heartbeat from core client for 30 sec - exiting 09:35:27 (4916): No heartbeat from core client for 30 sec - exiting 09:35:28 (4916): No heartbeat from core client for 30 sec - exiting 09:35:29 (4916): No heartbeat from core client for 30 sec - exiting 09:35:30 (4916): No heartbeat from core client for 30 sec - exiting 09:35:31 (4916): No heartbeat from core client for 30 sec - exiting 09:35:32 (4916): No heartbeat from core client for 30 sec - exiting 09:35:33 (4916): No heartbeat from core client for 30 sec - exiting 09:35:34 (4916): No heartbeat from core client for 30 sec - exiting 09:35:35 (4916): No heartbeat from core client for 30 sec - exiting 09:35:36 (4916): No heartbeat from core client for 30 sec - exiting 09:35:37 (4916): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:37:54 (4328): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:21:33 (7920): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:38:27 (6076): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:44:20 (3728): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:23:37 (6720): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6992, iMonCtr=1 Model crash detected, will try to restart... 20:08:02 (5708): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Atmos Hold Restart file rename failed on atmos_restart.hold 00:36:02 (7300): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 14:16:14 (3592): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:25:16 (7312): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:25:46 (7360): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:50:02 (1240): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:50:18 (7388): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:53:20 (7656): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:17:46 (5816): No heartbeat from core client for 30 sec - exiting 19:17:47 (5816): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/o4npko.pjj5c10 Error converting file to netcdf: dataout/o4npko.pij5c10 Error converting file to netcdf: dataout/o4npko.pfj5c10 Error converting file to netcdf: dataout/o4npka.phj5c10 Error converting file to netcdf: dataout/o4npka.pgj5c10 Error converting file to netcdf: dataout/o4npka.pej5c10 Error converting file to netcdf: dataout/o4npka.pdj5c10 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 16:02:46 (6184): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 18:21:25 (6644): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 10:24:34 (6568): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:03:45 (1420): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 11:51:44 (6536): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:51:45 (6536): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 09:30:07 (5820): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
13 Dec 2011 14:38:02	1065995	13610655	hadcm3n_o4np_1980_40_007540011_0	440,640	700,251	1.5892
11 Dec 2011 14:48:19	1065995	13610655	hadcm3n_o4np_1980_40_007540011_0	414,720	659,354	1.5899
08 Dec 2011 19:28:41	1065995	13610655	hadcm3n_o4np_1980_40_007540011_0	388,800	618,636	1.5911
04 Dec 2011 14:16:18	1065995	13610655	hadcm3n_o4np_1980_40_007540011_0	362,880	574,385	1.5829
03 Dec 2011 11:11:48	1065995	13610655	hadcm3n_o4np_1980_40_007540011_0	336,960	532,288	1.5797
02 Dec 2011 08:47:00	1065995	13610655	hadcm3n_o4np_1980_40_007540011_0	311,040	491,142	1.5790
01 Dec 2011 10:41:53	1065995	13610655	hadcm3n_o4np_1980_40_007540011_0	285,120	450,919	1.5815
30 Nov 2011 09:36:50	1065995	13610655	hadcm3n_o4np_1980_40_007540011_0	259,200	411,372	1.5871
28 Nov 2011 23:48:03	1065995	13610655	hadcm3n_o4np_1980_40_007540011_0	233,280	371,083	1.5907
28 Nov 2011 07:50:17	1065995	13610655	hadcm3n_o4np_1980_40_007540011_0	207,360	330,071	1.5918
27 Nov 2011 00:23:11	1065995	13610655	hadcm3n_o4np_1980_40_007540011_0	181,440	289,987	1.5983
25 Nov 2011 23:27:14	1065995	13610655	hadcm3n_o4np_1980_40_007540011_0	155,520	248,703	1.5992
24 Nov 2011 21:01:48	1065995	13610655	hadcm3n_o4np_1980_40_007540011_0	129,600	207,513	1.6012
23 Nov 2011 17:49:38	1065995	13610655	hadcm3n_o4np_1980_40_007540011_0	103,680	165,707	1.5983
22 Nov 2011 15:30:35	1065995	13610655	hadcm3n_o4np_1980_40_007540011_0	77,760	125,067	1.6084
21 Nov 2011 11:04:07	1065995	13610655	hadcm3n_o4np_1980_40_007540011_0	51,840	85,473	1.6488
20 Nov 2011 08:42:47	1065995	13610655	hadcm3n_o4np_1980_40_007540011_0	25,920	43,244	1.6684