Task 14347663

Name	hadcm3n_ybjt_1980_40_007833533_4
Workunit	7988645
Created	2 Apr 2012, 18:34:14 UTC
Sent	2 Apr 2012, 18:35:19 UTC
Report deadline	3 Jul 2012, 2:02:30 UTC
Received	6 Apr 2012, 16:16:09 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	479314
Run time	2 days 19 hours 24 min 45 sec
CPU time	2 days 11 hours 29 min 32 sec
Validate state	Invalid
Credit	622.08
Device peak FLOPS	1.27 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.3</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> 01:53:08 (1192): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:53:09 (1192): No heartbeat from core client for 30 sec - exiting 01:53:10 (1192): No heartbeat from core client for 30 sec - exiting 01:53:12 (1192): No heartbeat from core client for 30 sec - exiting 18:54:54 (1308): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:54:55 (1308): No heartbeat from core client for 30 sec - exiting 14:58:48 (3428): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:58:50 (3428): No heartbeat from core client for 30 sec - exiting 14:58:51 (3428): No heartbeat from core client for 30 sec - exiting 14:58:52 (3428): No heartbeat from core client for 30 sec - exiting 01:35:52 (3288): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:35:54 (3288): No heartbeat from core client for 30 sec - exiting 01:35:55 (3288): No heartbeat from core client for 30 sec - exiting 01:43:59 (2668): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:44:00 (2668): No heartbeat from core client for 30 sec - exiting 09:04:24 (1364): No heartbeat from core client for 30 sec - exiting 09:04:25 (1364): No heartbeat from core client for 30 sec - exiting 09:04:26 (1364): No heartbeat from core client for 30 sec - exiting 09:04:27 (1364): No heartbeat from core client for 30 sec - exiting 09:04:28 (1364): No heartbeat from core client for 30 sec - exiting 09:04:29 (1364): No heartbeat from core client for 30 sec - exiting 09:04:31 (1364): No heartbeat from core client for 30 sec - exiting 09:04:32 (1364): No heartbeat from core client for 30 sec - exiting 09:04:33 (1364): No heartbeat from core client for 30 sec - exiting 09:04:34 (1364): No heartbeat from core client for 30 sec - exiting 09:04:35 (1364): No heartbeat from core client for 30 sec - exiting 09:04:36 (1364): No heartbeat from core client for 30 sec - exiting 09:04:37 (1364): No heartbeat from core client for 30 sec - exiting 09:04:38 (1364): No heartbeat from core client for 30 sec - exiting 09:04:39 (1364): No heartbeat from core client for 30 sec - exiting 09:04:40 (1364): No heartbeat from core client for 30 sec - exiting 09:04:42 (1364): No heartbeat from core client for 30 sec - exiting 09:04:43 (1364): No heartbeat from core client for 30 sec - exiting 09:04:44 (1364): No heartbeat from core client for 30 sec - exiting 09:04:45 (1364): No heartbeat from core client for 30 sec - exiting 09:04:46 (1364): No heartbeat from core client for 30 sec - exiting 09:04:47 (1364): No heartbeat from core client for 30 sec - exiting 09:04:48 (1364): No heartbeat from core client for 30 sec - exiting 09:04:49 (1364): No heartbeat from core client for 30 sec - exiting 09:04:50 (1364): No heartbeat from core client for 30 sec - exiting 09:04:51 (1364): No heartbeat from core client for 30 sec - exiting 09:04:52 (1364): No heartbeat from core client for 30 sec - exiting 09:04:54 (1364): No heartbeat from core client for 30 sec - exiting 09:04:55 (1364): No heartbeat from core client for 30 sec - exiting 09:04:56 (1364): No heartbeat from core client for 30 sec - exiting 09:04:57 (1364): No heartbeat from core client for 30 sec - exiting 09:05:31 (1364): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:05:32 (1364): No heartbeat from core client for 30 sec - exiting BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/ybjtko.pji2c10 Error converting file to netcdf: dataout/ybjtko.pii2c10 Error converting file to netcdf: dataout/ybjtko.pfi2c10 Error converting file to netcdf: dataout/ybjtka.phi2c10 Error converting file to netcdf: dataout/ybjtka.pgi2c10 Error converting file to netcdf: dataout/ybjtka.pei2c10 Error converting file to netcdf: dataout/ybjtka.pdi2c10 09:11:04 (1336): No heartbeat from core client for 30 sec - exiting 09:11:06 (1336): No heartbeat from core client for 30 sec - exiting 09:11:07 (1336): No heartbeat from core client for 30 sec - exiting 09:11:08 (1336): No heartbeat from core client for 30 sec - exiting 09:11:09 (1336): No heartbeat from core client for 30 sec - exiting 09:11:10 (1336): No heartbeat from core client for 30 sec - exiting 09:11:11 (1336): No heartbeat from core client for 30 sec - exiting 09:11:12 (1336): No heartbeat from core client for 30 sec - exiting 09:11:13 (1336): No heartbeat from core client for 30 sec - exiting 09:11:14 (1336): No heartbeat from core client for 30 sec - exiting 09:11:15 (1336): No heartbeat from core client for 30 sec - exiting 09:11:17 (1336): No heartbeat from core client for 30 sec - exiting 09:11:18 (1336): No heartbeat from core client for 30 sec - exiting 09:11:19 (1336): No heartbeat from core client for 30 sec - exiting 09:11:20 (1336): No heartbeat from core client for 30 sec - exiting 09:11:21 (1336): No heartbeat from core client for 30 sec - exiting 09:11:22 (1336): No heartbeat from core client for 30 sec - exiting 09:11:23 (1336): No heartbeat from core client for 30 sec - exiting 09:11:24 (1336): No heartbeat from core client for 30 sec - exiting 09:11:25 (1336): No heartbeat from core client for 30 sec - exiting 09:11:26 (1336): No heartbeat from core client for 30 sec - exiting 09:11:27 (1336): No heartbeat from core client for 30 sec - exiting 09:11:29 (1336): No heartbeat from core client for 30 sec - exiting 09:11:30 (1336): No heartbeat from core client for 30 sec - exiting 09:11:31 (1336): No heartbeat from core client for 30 sec - exiting 09:11:32 (1336): No heartbeat from core client for 30 sec - exiting 09:11:33 (1336): No heartbeat from core client for 30 sec - exiting 09:11:34 (1336): No heartbeat from core client for 30 sec - exiting 09:12:08 (1336): No heartbeat from core client for 30 sec - exiting 09:12:09 (1336): No heartbeat from core client for 30 sec - exiting 09:12:11 (1336): No heartbeat from core client for 30 sec - exiting 09:12:12 (1336): No heartbeat from core client for 30 sec - exiting 09:12:13 (1336): No heartbeat from core client for 30 sec - exiting 09:12:14 (1336): No heartbeat from core client for 30 sec - exiting 09:12:15 (1336): No heartbeat from core client for 30 sec - exiting 09:12:16 (1336): No heartbeat from core client for 30 sec - exiting 09:12:17 (1336): No heartbeat from core client for 30 sec - exiting 09:12:18 (1336): No heartbeat from core client for 30 sec - exiting 09:12:19 (1336): No heartbeat from core client for 30 sec - exiting 09:12:20 (1336): No heartbeat from core client for 30 sec - exiting 09:12:22 (1336): No heartbeat from core client for 30 sec - exiting 09:12:23 (1336): No heartbeat from core client for 30 sec - exiting 09:12:24 (1336): No heartbeat from core client for 30 sec - exiting 09:12:25 (1336): No heartbeat from core client for 30 sec - exiting 09:12:26 (1336): No heartbeat from core client for 30 sec - exiting 09:12:27 (1336): No heartbeat from core client for 30 sec - exiting 09:12:28 (1336): No heartbeat from core client for 30 sec - exiting 09:12:29 (1336): No heartbeat from core client for 30 sec - exiting 09:12:30 (1336): No heartbeat from core client for 30 sec - exiting 09:12:31 (1336): No heartbeat from core client for 30 sec - exiting 09:12:32 (1336): No heartbeat from core client for 30 sec - exiting 09:12:34 (1336): No heartbeat from core client for 30 sec - exiting 09:12:35 (1336): No heartbeat from core client for 30 sec - exiting 09:12:36 (1336): No heartbeat from core client for 30 sec - exiting 09:12:37 (1336): No heartbeat from core client for 30 sec - exiting 09:12:38 (1336): No heartbeat from core client for 30 sec - exiting 09:12:39 (1336): No heartbeat from core client for 30 sec - exiting 09:12:40 (1336): No heartbeat from core client for 30 sec - exiting 09:12:41 (1336): No heartbeat from core client for 30 sec - exiting 09:12:42 (1336): No heartbeat from core client for 30 sec - exiting 09:12:43 (1336): No heartbeat from core client for 30 sec - exiting 09:13:41 (1336): No heartbeat from core client for 30 sec - exiting 09:13:42 (1336): No heartbeat from core client for 30 sec - exiting 09:13:43 (1336): No heartbeat from core client for 30 sec - exiting 09:13:44 (1336): No heartbeat from core client for 30 sec - exiting 09:13:45 (1336): No heartbeat from core client for 30 sec - exiting 09:13:46 (1336): No heartbeat from core client for 30 sec - exiting 09:13:48 (1336): No heartbeat from core client for 30 sec - exiting 09:13:49 (1336): No heartbeat from core client for 30 sec - exiting 09:13:50 (1336): No heartbeat from core client for 30 sec - exiting 09:13:51 (1336): No heartbeat from core client for 30 sec - exiting 09:13:52 (1336): No heartbeat from core client for 30 sec - exiting 09:13:54 (1336): No heartbeat from core client for 30 sec - exiting 09:13:55 (1336): No heartbeat from core client for 30 sec - exiting 09:15:32 (1336): No heartbeat from core client for 30 sec - exiting 09:15:34 (1336): No heartbeat from core client for 30 sec - exiting 09:15:35 (1336): No heartbeat from core client for 30 sec - exiting 09:15:36 (1336): No heartbeat from core client for 30 sec - exiting 09:15:37 (1336): No heartbeat from core client for 30 sec - exiting 09:15:38 (1336): No heartbeat from core client for 30 sec - exiting 09:15:39 (1336): No heartbeat from core client for 30 sec - exiting 09:15:40 (1336): No heartbeat from core client for 30 sec - exiting 09:15:41 (1336): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 Model crashed: STWORK : I/O error - PP fixed length header tmp/pipe_dummy 2048 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 Model crashed: STWORK : I/O error - PP fixed length header tmp/pipe_dummy 2048 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 Model crashed: STWORK : I/O error - PP fixed length header tmp/pipe_dummy 2048 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 Model crashed: STWORK : I/O error - PP fixed length header tmp/pipe_dummy 2048 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 Model crashed: STWORK : I/O error - PP fixed length header tmp/pipe_dummy 2048 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 Model crashed: STWORK : I/O error - PP fixed length header tmp/pipe_dummy 2048 Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
05 Apr 2012 13:16:09	479314	14347663	hadcm3n_ybjt_1980_40_007833533_4	51,840	213,862	4.1254
04 Apr 2012 04:47:48	479314	14347663	hadcm3n_ybjt_1980_40_007833533_4	25,920	107,039	4.1296