Task 13663018

Name	hadcm3n_ymk6_1900_40_007513532_4
Workunit	7711007
Created	26 Nov 2011, 6:49:35 UTC
Sent	26 Nov 2011, 6:51:57 UTC
Report deadline	25 Feb 2012, 14:19:08 UTC
Received	12 Dec 2011, 11:46:33 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	25 (0x00000019) Unknown error code
Computer ID	1109605
Run time	2 days 22 hours 13 min 56 sec
CPU time	2 days 21 hours 7 min 19 sec
Validate state	Invalid
Credit	2,799.36
Device peak FLOPS	3.35 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.10.60</core_client_version> <![CDATA[ <message> The drive cannot locate a specific area or track on the disk. (0x19) - exit code 25 (0x19) </message> <stderr_txt> 20:40:32 (6804): No heartbeat from core client for 30 sec - exiting 20:40:33 (6804): No heartbeat from core client for 30 sec - exiting 20:40:34 (6804): No heartbeat from core client for 30 sec - exiting 20:40:35 (6804): No heartbeat from core client for 30 sec - exiting 20:40:36 (6804): No heartbeat from core client for 30 sec - exiting 20:40:37 (6804): No heartbeat from core client for 30 sec - exiting 20:40:38 (6804): No heartbeat from core client for 30 sec - exiting 20:40:39 (6804): No heartbeat from core client for 30 sec - exiting 20:40:40 (6804): No heartbeat from core client for 30 sec - exiting 20:40:41 (6804): No heartbeat from core client for 30 sec - exiting 20:40:43 (6804): No heartbeat from core client for 30 sec - exiting 20:40:44 (6804): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:35:15 (5908): No heartbeat from core client for 30 sec - exiting 18:35:16 (5908): No heartbeat from core client for 30 sec - exiting 18:35:17 (5908): No heartbeat from core client for 30 sec - exiting 18:35:19 (5908): No heartbeat from core client for 30 sec - exiting 18:35:20 (5908): No heartbeat from core client for 30 sec - exiting 18:35:21 (5908): No heartbeat from core client for 30 sec - exiting 18:35:22 (5908): No heartbeat from core client for 30 sec - exiting 18:35:23 (5908): No heartbeat from core client for 30 sec - exiting 18:35:24 (5908): No heartbeat from core client for 30 sec - exiting 18:35:25 (5908): No heartbeat from core client for 30 sec - exiting 18:35:26 (5908): No heartbeat from core client for 30 sec - exiting 18:35:27 (5908): No heartbeat from core client for 30 sec - exiting 18:35:28 (5908): No heartbeat from core client for 30 sec - exiting 18:35:29 (5908): No heartbeat from core client for 30 sec - exiting 18:35:30 (5908): No heartbeat from core client for 30 sec - exiting 18:35:31 (5908): No heartbeat from core client for 30 sec - exiting 18:35:32 (5908): No heartbeat from core client for 30 sec - exiting 18:35:33 (5908): No heartbeat from core client for 30 sec - exiting 18:35:35 (5908): No heartbeat from core client for 30 sec - exiting 18:35:36 (5908): No heartbeat from core client for 30 sec - exiting 18:35:37 (5908): No heartbeat from core client for 30 sec - exiting 18:35:38 (5908): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:38:17 (6020): No heartbeat from core client for 30 sec - exiting 10:38:18 (6020): No heartbeat from core client for 30 sec - exiting 10:38:19 (6020): No heartbeat from core client for 30 sec - exiting 10:38:20 (6020): No heartbeat from core client for 30 sec - exiting 10:38:21 (6020): No heartbeat from core client for 30 sec - exiting 10:38:22 (6020): No heartbeat from core client for 30 sec - exiting 10:38:23 (6020): No heartbeat from core client for 30 sec - exiting 10:38:24 (6020): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 19:33:18 (4108): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:33:19 (4108): No heartbeat from core client for 30 sec - exiting 20:46:27 (7052): No heartbeat from core client for 30 sec - exiting 20:46:28 (7052): No heartbeat from core client for 30 sec - exiting 20:46:29 (7052): No heartbeat from core client for 30 sec - exiting 20:46:30 (7052): No heartbeat from core client for 30 sec - exiting 20:46:31 (7052): No heartbeat from core client for 30 sec - exiting 20:46:32 (7052): No heartbeat from core client for 30 sec - exiting 20:46:33 (7052): No heartbeat from core client for 30 sec - exiting 20:46:34 (7052): No heartbeat from core client for 30 sec - exiting 20:46:35 (7052): No heartbeat from core client for 30 sec - exiting 20:46:36 (7052): No heartbeat from core client for 30 sec - exiting 20:46:37 (7052): No heartbeat from core client for 30 sec - exiting 20:46:38 (7052): No heartbeat from core client for 30 sec - exiting 20:46:39 (7052): No heartbeat from core client for 30 sec - exiting 20:46:40 (7052): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 17:33:35 (5428): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:33:36 (5428): No heartbeat from core client for 30 sec - exiting 17:33:37 (5428): No heartbeat from core client for 30 sec - exiting 17:33:38 (5428): No heartbeat from core client for 30 sec - exiting 17:33:39 (5428): No heartbeat from core client for 30 sec - exiting 17:33:40 (5428): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 20:48:51 (6220): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3300, iMonCtr=1 Model crash detected, will try to restart... 07:18:12 (6260): No heartbeat from core client for 30 sec - exiting 07:18:13 (6260): No heartbeat from core client for 30 sec - exiting 07:18:14 (6260): No heartbeat from core client for 30 sec - exiting 07:18:15 (6260): No heartbeat from core client for 30 sec - exiting 07:18:17 (6260): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 21:37:58 (6264): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 21:38:00 (6264): No heartbeat from core client for 30 sec - exiting 21:38:01 (6264): No heartbeat from core client for 30 sec - exiting 21:39:11 (1436): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Atmos Hold Restart file rename failed on atmos_restart.hold Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 10:58:25 (3860): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:58:27 (3860): No heartbeat from core client for 30 sec - exiting 10:58:28 (3860): No heartbeat from core client for 30 sec - exiting 10:58:29 (3860): No heartbeat from core client for 30 sec - exiting 10:58:30 (3860): No heartbeat from core client for 30 sec - exiting 10:58:31 (3860): No heartbeat from core client for 30 sec - exiting 10:58:32 (3860): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 10:14:14 (5768): No heartbeat from core client for 30 sec - exiting 10:14:15 (5768): No heartbeat from core client for 30 sec - exiting 10:14:16 (5768): No heartbeat from core client for 30 sec - exiting 10:14:17 (5768): No heartbeat from core client for 30 sec - exiting 10:14:18 (5768): No heartbeat from core client for 30 sec - exiting 10:14:19 (5768): No heartbeat from core client for 30 sec - exiting 10:14:20 (5768): No heartbeat from core client for 30 sec - exiting 10:14:21 (5768): No heartbeat from core client for 30 sec - exiting 10:14:23 (5768): No heartbeat from core client for 30 sec - exiting 10:14:24 (5768): No heartbeat from core client for 30 sec - exiting 10:14:25 (5768): No heartbeat from core client for 30 sec - exiting 10:14:26 (5768): No heartbeat from core client for 30 sec - exiting 10:14:27 (5768): No heartbeat from core client for 30 sec - exiting 10:14:28 (5768): No heartbeat from core client for 30 sec - exiting 10:14:29 (5768): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 12:40:01 (6280): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:40:02 (6280): No heartbeat from core client for 30 sec - exiting 12:40:03 (6280): No heartbeat from core client for 30 sec - exiting 12:40:04 (6280): No heartbeat from core client for 30 sec - exiting 12:40:05 (6280): No heartbeat from core client for 30 sec - exiting 12:40:06 (6280): No heartbeat from core client for 30 sec - exiting 12:40:07 (6280): No heartbeat from core client for 30 sec - exiting 12:40:08 (6280): No heartbeat from core client for 30 sec - exiting 12:40:09 (6280): No heartbeat from core client for 30 sec - exiting 12:40:10 (6280): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
11 Dec 2011 07:31:31	1109605	13663018	hadcm3n_ymk6_1900_40_007513532_4	233,280	240,234	1.0298
09 Dec 2011 23:48:50	1109605	13663018	hadcm3n_ymk6_1900_40_007513532_4	207,360	213,860	1.0313
09 Dec 2011 04:16:35	1109605	13663018	hadcm3n_ymk6_1900_40_007513532_4	181,440	187,154	1.0315
05 Dec 2011 05:50:15	1109605	13663018	hadcm3n_ymk6_1900_40_007513532_4	155,520	160,764	1.0337
04 Dec 2011 10:20:20	1109605	13663018	hadcm3n_ymk6_1900_40_007513532_4	129,600	134,600	1.0386
03 Dec 2011 10:26:39	1109605	13663018	hadcm3n_ymk6_1900_40_007513532_4	103,680	107,653	1.0383
01 Dec 2011 07:05:14	1109605	13663018	hadcm3n_ymk6_1900_40_007513532_4	77,760	80,716	1.0380
29 Nov 2011 10:27:09	1109605	13663018	hadcm3n_ymk6_1900_40_007513532_4	51,840	53,562	1.0332
27 Nov 2011 11:56:33	1109605	13663018	hadcm3n_ymk6_1900_40_007513532_4	25,920	26,849	1.0358