Task 11802918

Name	famous_vhn3_999_200_006708629_4
Workunit	6911882
Created	26 Aug 2010, 17:01:15 UTC
Sent	12 Nov 2010, 0:46:42 UTC
Report deadline	11 Feb 2011, 8:13:53 UTC
Received	12 Nov 2010, 18:44:16 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1093122
Run time	11 hours 15 min 38 sec
CPU time	8 hours 51 min 15 sec
Validate state	Invalid
Credit	216.26
Device peak FLOPS	2.27 GFLOPS
Application version	UK Met Office FAMOUS v6.11 windows_intelx86
Stderr	<core_client_version>6.10.58</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> 01:39:11 (1076): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:00:09 (3948): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:00:46 (3948): No heartbeat from core client for 30 sec - exiting 02:00:49 (3948): No heartbeat from core client for 30 sec - exiting 02:00:50 (3948): No heartbeat from core client for 30 sec - exiting 03:43:47 (8128): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:43:55 (8128): No heartbeat from core client for 30 sec - exiting 03:56:51 (4652): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:56:53 (4652): No heartbeat from core client for 30 sec - exiting 03:56:55 (4652): No heartbeat from core client for 30 sec - exiting 03:57:16 (4652): No heartbeat from core client for 30 sec - exiting 03:57:18 (4652): No heartbeat from core client for 30 sec - exiting 03:57:21 (4652): No heartbeat from core client for 30 sec - exiting 04:54:31 (2204): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:54:32 (2204): No heartbeat from core client for 30 sec - exiting 05:15:10 (468): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:15:20 (468): No heartbeat from core client for 30 sec - exiting 05:15:23 (468): No heartbeat from core client for 30 sec - exiting 05:15:24 (468): No heartbeat from core client for 30 sec - exiting 05:15:28 (468): No heartbeat from core client for 30 sec - exiting 05:15:29 (468): No heartbeat from core client for 30 sec - exiting 05:15:33 (468): No heartbeat from core client for 30 sec - exiting 05:15:34 (468): No heartbeat from core client for 30 sec - exiting 05:17:50 (7320): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:17:52 (7320): No heartbeat from core client for 30 sec - exiting 05:17:53 (7320): No heartbeat from core client for 30 sec - exiting 05:37:37 (4224): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:38:07 (4224): No heartbeat from core client for 30 sec - exiting 05:38:09 (4224): No heartbeat from core client for 30 sec - exiting 05:55:19 (620): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:55:21 (620): No heartbeat from core client for 30 sec - exiting 06:40:14 (7724): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:44:11 (5536): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:56:43 (692): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:56:47 (692): No heartbeat from core client for 30 sec - exiting 06:56:51 (692): No heartbeat from core client for 30 sec - exiting 06:56:52 (692): No heartbeat from core client for 30 sec - exiting 06:56:56 (692): No heartbeat from core client for 30 sec - exiting 06:56:59 (692): No heartbeat from core client for 30 sec - exiting 06:57:00 (692): No heartbeat from core client for 30 sec - exiting 06:57:03 (692): No heartbeat from core client for 30 sec - exiting 06:57:06 (692): No heartbeat from core client for 30 sec - exiting 06:57:07 (692): No heartbeat from core client for 30 sec - exiting 07:12:12 (3700): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:12:33 (3700): No heartbeat from core client for 30 sec - exiting 07:12:54 (3700): No heartbeat from core client for 30 sec - exiting 07:35:36 (7852): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:35:43 (7852): No heartbeat from core client for 30 sec - exiting 08:30:18 (6516): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:30:40 (6516): No heartbeat from core client for 30 sec - exiting 08:30:46 (6516): No heartbeat from core client for 30 sec - exiting Model crashed: TEMPHIST: Write ERROR on history file for namelistNLIHISTO tmp/pipe_dummy 09:13:12 (284): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:13:20 (284): No heartbeat from core client for 30 sec - exiting 09:13:26 (284): No heartbeat from core client for 30 sec - exiting 09:13:27 (284): No heartbeat from core client for 30 sec - exiting 09:13:29 (284): No heartbeat from core client for 30 sec - exiting 09:13:34 (284): No heartbeat from core client for 30 sec - exiting 09:39:05 (6508): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:39:09 (6508): No heartbeat from core client for 30 sec - exiting 09:39:13 (6508): No heartbeat from core client for 30 sec - exiting BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 16 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 16 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 09:53:54 (3024): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:54:00 (3024): No heartbeat from core client for 30 sec - exiting 09:54:05 (3024): No heartbeat from core client for 30 sec - exiting 09:54:06 (3024): No heartbeat from core client for 30 sec - exiting 09:54:15 (3024): No heartbeat from core client for 30 sec - exiting 09:54:21 (3024): No heartbeat from core client for 30 sec - exiting 09:54:25 (3024): No heartbeat from core client for 30 sec - exiting 10:25:04 (4272): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:25:09 (4272): No heartbeat from core client for 30 sec - exiting 10:25:10 (4272): No heartbeat from core client for 30 sec - exiting 10:25:11 (4272): No heartbeat from core client for 30 sec - exiting 10:36:08 (4812): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:37:06 (4812): No heartbeat from core client for 30 sec - exiting 10:37:08 (4812): No heartbeat from core client for 30 sec - exiting 10:37:12 (4812): No heartbeat from core client for 30 sec - exiting 10:37:14 (4812): No heartbeat from core client for 30 sec - exiting 11:17:33 (6976): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:17:56 (6976): No heartbeat from core client for 30 sec - exiting 11:17:57 (6976): No heartbeat from core client for 30 sec - exiting 11:18:08 (6976): No heartbeat from core client for 30 sec - exiting 11:59:00 (8160): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:59:07 (8160): No heartbeat from core client for 30 sec - exiting 11:59:11 (8160): No heartbeat from core client for 30 sec - exiting 11:59:12 (8160): No heartbeat from core client for 30 sec - exiting 12:07:19 (3120): No heartbeat from core client for 30 sec - exiting 12:08:48 (3120): No heartbeat from core client for 30 sec - exiting 12:08:49 (3120): No heartbeat from core client for 30 sec - exiting 12:08:54 (3120): No heartbeat from core client for 30 sec - exiting 12:08:55 (3120): No heartbeat from core client for 30 sec - exiting 12:08:58 (3120): No heartbeat from core client for 30 sec - exiting 12:08:59 (3120): No heartbeat from core client for 30 sec - exiting 12:09:03 (3120): No heartbeat from core client for 30 sec - exiting 12:10:29 (3120): No heartbeat from core client for 30 sec - exiting 12:10:36 (3120): No heartbeat from core client for 30 sec - exiting 12:10:49 (3120): No heartbeat from core client for 30 sec - exiting 12:10:51 (3120): No heartbeat from core client for 30 sec - exiting 12:10:52 (3120): No heartbeat from core client for 30 sec - exiting 12:10:54 (3120): No heartbeat from core client for 30 sec - exiting 12:10:58 (3120): No heartbeat from core client for 30 sec - exiting 12:11:01 (3120): No heartbeat from core client for 30 sec - exiting 12:11:38 (3120): No heartbeat from core client for 30 sec - exiting 12:11:42 (3120): No heartbeat from core client for 30 sec - exiting 12:11:43 (3120): No heartbeat from core client for 30 sec - exiting 12:11:59 (3120): No heartbeat from core client for 30 sec - exiting 12:12:08 (3120): No heartbeat from core client for 30 sec - exiting 12:12:10 (3120): No heartbeat from core client for 30 sec - exiting 12:12:14 (3120): No heartbeat from core client for 30 sec - exiting 12:12:16 (3120): No heartbeat from core client for 30 sec - exiting 12:12:17 (3120): No heartbeat from core client for 30 sec - exiting 12:12:19 (3120): No heartbeat from core client for 30 sec - exiting 12:12:20 (3120): No heartbeat from core client for 30 sec - exiting 12:13:45 (3120): No heartbeat from core client for 30 sec - exiting 12:14:16 (3120): No heartbeat from core client for 30 sec - exiting 12:14:26 (3120): No heartbeat from core client for 30 sec - exiting 12:15:03 (3120): No heartbeat from core client for 30 sec - exiting 12:15:05 (3120): No heartbeat from core client for 30 sec - exiting 12:15:06 (3120): No heartbeat from core client for 30 sec - exiting 12:15:52 (3120): No heartbeat from core client for 30 sec - exiting 12:15:58 (3120): No heartbeat from core client for 30 sec - exiting 12:16:41 (3120): No heartbeat from core client for 30 sec - exiting 12:16:43 (3120): No heartbeat from core client for 30 sec - exiting 12:16:48 (3120): No heartbeat from core client for 30 sec - exiting 12:16:52 (3120): No heartbeat from core client for 30 sec - exiting 12:17:55 (3120): No heartbeat from core client for 30 sec - exiting 12:17:59 (3120): No heartbeat from core client for 30 sec - exiting 12:18:34 (3120): No heartbeat from core client for 30 sec - exiting 12:18:37 (3120): No heartbeat from core client for 30 sec - exiting 12:20:05 (3120): No heartbeat from core client for 30 sec - exiting 12:20:09 (3120): No heartbeat from core client for 30 sec - exiting 12:21:00 (3120): No heartbeat from core client for 30 sec - exiting 12:21:03 (3120): No heartbeat from core client for 30 sec - exiting 12:21:11 (3120): No heartbeat from core client for 30 sec - exiting 12:21:15 (3120): No heartbeat from core client for 30 sec - exiting 12:21:16 (3120): No heartbeat from core client for 30 sec - exiting 12:21:20 (3120): No heartbeat from core client for 30 sec - exiting 12:21:21 (3120): No heartbeat from core client for 30 sec - exiting 12:22:23 (3120): No heartbeat from core client for 30 sec - exiting 12:22:26 (3120): No heartbeat from core client for 30 sec - exiting 12:22:29 (3120): No heartbeat from core client for 30 sec - exiting 12:22:30 (3120): No heartbeat from core client for 30 sec - exiting 12:22:36 (3120): No heartbeat from core client for 30 sec - exiting 12:22:40 (3120): No heartbeat from core client for 30 sec - exiting 12:22:42 (3120): No heartbeat from core client for 30 sec - exiting 12:22:44 (3120): No heartbeat from core client for 30 sec - exiting 12:22:48 (3120): No heartbeat from core client for 30 sec - exiting 12:22:50 (3120): No heartbeat from core client for 30 sec - exiting 12:22:51 (3120): No heartbeat from core client for 30 sec - exiting 12:22:56 (3120): No heartbeat from core client for 30 sec - exiting 12:23:01 (3120): No heartbeat from core client for 30 sec - exiting 12:23:02 (3120): No heartbeat from core client for 30 sec - exiting 12:23:10 (3120): No heartbeat from core client for 30 sec - exiting 12:23:11 (3120): No heartbeat from core client for 30 sec - exiting 12:23:13 (3120): No heartbeat from core client for 30 sec - exiting 12:23:20 (3120): No heartbeat from core client for 30 sec - exiting 12:23:21 (3120): No heartbeat from core client for 30 sec - exiting 12:23:25 (3120): No heartbeat from core client for 30 sec - exiting 12:23:26 (3120): No heartbeat from core client for 30 sec - exiting 12:24:37 (3120): No heartbeat from core client for 30 sec - exiting 12:24:41 (3120): No heartbeat from core client for 30 sec - exiting 12:24:42 (3120): No heartbeat from core client for 30 sec - exiting 12:24:43 (3120): No heartbeat from core client for 30 sec - exiting 12:24:46 (3120): No heartbeat from core client for 30 sec - exiting 12:24:47 (3120): No heartbeat from core client for 30 sec - exiting 12:24:50 (3120): No heartbeat from core client for 30 sec - exiting 12:24:52 (3120): No heartbeat from core client for 30 sec - exiting 12:24:57 (3120): No heartbeat from core client for 30 sec - exiting 12:25:43 (3120): No heartbeat from core client for 30 sec - exiting 12:25:46 (3120): No heartbeat from core client for 30 sec - exiting 12:25:58 (3120): No heartbeat from core client for 30 sec - exiting 12:26:03 (3120): No heartbeat from core client for 30 sec - exiting 12:26:09 (3120): No heartbeat from core client for 30 sec - exiting 12:26:15 (3120): No heartbeat from core client for 30 sec - exiting 12:26:21 (3120): No heartbeat from core client for 30 sec - exiting 12:26:28 (3120): No heartbeat from core client for 30 sec - exiting 12:26:29 (3120): No heartbeat from core client for 30 sec - exiting 12:26:31 (3120): No heartbeat from core client for 30 sec - exiting 12:26:32 (3120): No heartbeat from core client for 30 sec - exiting 12:26:33 (3120): No heartbeat from core client for 30 sec - exiting 12:27:33 (3120): No heartbeat from core client for 30 sec - exiting 12:28:15 (3120): No heartbeat from core client for 30 sec - exiting 12:28:26 (3120): No heartbeat from core client for 30 sec - exiting 12:28:44 (3120): No heartbeat from core client for 30 sec - exiting 12:28:48 (3120): No heartbeat from core client for 30 sec - exiting 12:28:49 (3120): No heartbeat from core client for 30 sec - exiting 12:28:52 (3120): No heartbeat from core client for 30 sec - exiting 12:28:53 (3120): No heartbeat from core client for 30 sec - exiting 12:29:39 (3120): No heartbeat from core client for 30 sec - exiting 12:29:40 (3120): No heartbeat from core client for 30 sec - exiting 12:29:41 (3120): No heartbeat from core client for 30 sec - exiting 12:30:45 (3120): No heartbeat from core client for 30 sec - exiting 12:30:53 (3120): No heartbeat from core client for 30 sec - exiting 12:31:00 (3120): No heartbeat from core client for 30 sec - exiting 12:31:04 (3120): No heartbeat from core client for 30 sec - exiting 12:31:05 (3120): No heartbeat from core client for 30 sec - exiting 12:31:46 (3120): No heartbeat from core client for 30 sec - exiting 12:31:49 (3120): No heartbeat from core client for 30 sec - exiting 12:31:50 (3120): No heartbeat from core client for 30 sec - exiting 12:31:55 (3120): No heartbeat from core client for 30 sec - exiting 12:31:56 (3120): No heartbeat from core client for 30 sec - exiting 12:32:34 (3120): No heartbeat from core client for 30 sec - exiting 12:33:28 (3120): No heartbeat from core client for 30 sec - exiting 12:34:23 (3120): No heartbeat from core client for 30 sec - exiting 12:34:25 (3120): No heartbeat from core client for 30 sec - exiting 12:34:28 (3120): No heartbeat from core client for 30 sec - exiting 12:34:31 (3120): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 16 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 16 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 13:22:05 (7240): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:37:32 (5168): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Model crashed: TEMPHIST: Write ERROR on history file for namelistNLIHISTO tmp/pipe_dummy 16:01:22 (2476): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:36:02 (5988): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:37:33 (5988): No heartbeat from core client for 30 sec - exiting 16:54:53 (7608): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:55:01 (7608): No heartbeat from core client for 30 sec - exiting 16:55:03 (7608): No heartbeat from core client for 30 sec - exiting 16:55:07 (7608): No heartbeat from core client for 30 sec - exiting 16:55:08 (7608): No heartbeat from core client for 30 sec - exiting 16:55:13 (7608): No heartbeat from core client for 30 sec - exiting 16:55:14 (7608): No heartbeat from core client for 30 sec - exiting 117:08:54 (6352): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:09:31 (6352): No heartbeat from core client for 30 sec - exiting 17:38:15 (3424): No heartbeat from core client for 30 sec - exiting 17:38:57 (3424): No heartbeat from core client for 30 sec - exiting 17:39:02 (3424): No heartbeat from core client for 30 sec - exiting 17:39:04 (3424): No heartbeat from core client for 30 sec - exiting 17:39:05 (3424): No heartbeat from core client for 30 sec - exiting 17:39:06 (3424): No heartbeat from core client for 30 sec - exiting 17:39:10 (3424): No heartbeat from core client for 30 sec - exiting 17:39:14 (3424): No heartbeat from core client for 30 sec - exiting 17:39:19 (3424): No heartbeat from core client for 30 sec - exiting 17:39:20 (3424): No heartbeat from core client for 30 sec - exiting 17:39:21 (3424): No heartbeat from core client for 30 sec - exiting 17:39:26 (3424): No heartbeat from core client for 30 sec - exiting 17:39:30 (3424): No heartbeat from core client for 30 sec - exiting 17:39:34 (3424): No heartbeat from core client for 30 sec - exiting 17:39:42 (3424): No heartbeat from core client for 30 sec - exiting 17:40:25 (3424): No heartbeat from core client for 30 sec - exiting 17:40:27 (3424): No heartbeat from core client for 30 sec - exiting 17:40:28 (3424): No heartbeat from core client for 30 sec - exiting 17:40:32 (3424): No heartbeat from core client for 30 sec - exiting 17:40:33 (3424): No heartbeat from core client for 30 sec - exiting 17:40:34 (3424): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:47:42 (2212): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:57:38 (4032): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:59:50 (4032): No heartbeat from core client for 30 sec - exiting 18:13:03 (5104): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:37:01 (3964): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:37:05 (3964): No heartbeat from core client for 30 sec - exiting 18:37:07 (3964): No heartbeat from core client for 30 sec - exiting 18:37:11 (3964): No heartbeat from core client for 30 sec - exiting 18:37:13 (3964): No heartbeat from core client for 30 sec - exiting 18:37:15 (3964): No heartbeat from core client for 30 sec - exiting 18:37:19 (3964): No heartbeat from core client for 30 sec - exiting Model crashed: READHIST: End of file in READ from history file for namelist NLIHISTO tmp/pipe_dummy Model crashed: READHIST: End of file in READ from history file for namelist NLIHISTO tmp/pipe_dummy Model crashed: READHIST: End of file in READ from history file for namelist NLIHISTO tmp/pipe_dummy Model crashed: READHIST: End of file in READ from history file for namelist NLIHISTO tmp/pipe_dummy Model crashed: READHIST: End of file in READ from history file for namelist NLIHISTO tmp/pipe_dummy Model crashed: READHIST: End of file in READ from history file for namelist NLIHISTO tmp/pipe_dummy Sorry, too many model crashes! :-( 18:41:26 (2640): called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
12 Nov 2010 15:33:22	1093122	11802918	famous_vhn3_999_200_006708629_4	65,546	27,896	0.4256
12 Nov 2010 14:16:52	1093122	11802918	famous_vhn3_999_200_006708629_4	56,186	23,904	0.4254
12 Nov 2010 12:03:06	1093122	11802918	famous_vhn3_999_200_006708629_4	46,826	19,931	0.4256
12 Nov 2010 09:45:54	1093122	11802918	famous_vhn3_999_200_006708629_4	37,466	15,945	0.4256
12 Nov 2010 09:25:15	1093122	11802918	famous_vhn3_999_200_006708629_4	28,106	11,986	0.4265
12 Nov 2010 09:25:15	1093122	11802918	famous_vhn3_999_200_006708629_4	18,746	7,993	0.4264
12 Nov 2010 09:25:15	1093122	11802918	famous_vhn3_999_200_006708629_4	9,386	3,988	0.4249