Task 16002256

Name	hadcm3n_o54x_2140_40_008270374_3
Workunit	8425498
Created	3 Sep 2013, 19:19:07 UTC
Sent	3 Sep 2013, 19:27:58 UTC
Report deadline	4 Dec 2013, 2:55:09 UTC
Received	8 Sep 2013, 11:44:58 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1136891
Run time	2 days 2 hours 5 min 46 sec
CPU time	2 days 0 hours 2 min 42 sec
Validate state	Invalid
Credit	933.12
Device peak FLOPS	2.12 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.64</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 19:54:51 (10548): No heartbeat from core client for 30 sec - exiting 19:54:58 (10548): No heartbeat from core client for 30 sec - exiting 19:54:59 (10548): No heartbeat from core client for 30 sec - exiting 19:55:00 (10548): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 15:28:53 (9108): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 19:28:27 (10724): No heartbeat from core client for 30 sec - exiting 19:28:31 (10724): No heartbeat from core client for 30 sec - exiting 19:28:32 (10724): No heartbeat from core client for 30 sec - exiting 19:28:33 (10724): No heartbeat from core client for 30 sec - exiting 19:28:34 (10724): No heartbeat from core client for 30 sec - exiting 19:28:35 (10724): No heartbeat from core client for 30 sec - exiting 19:28:36 (10724): No heartbeat from core client for 30 sec - exiting 19:28:37 (10724): No heartbeat from core client for 30 sec - exiting 19:28:38 (10724): No heartbeat from core client for 30 sec - exiting 19:28:39 (10724): No heartbeat from core client for 30 sec - exiting 19:28:40 (10724): No heartbeat from core client for 30 sec - exiting 19:28:41 (10724): No heartbeat from core client for 30 sec - exiting 19:28:42 (10724): No heartbeat from core client for 30 sec - exiting 19:28:43 (10724): No heartbeat from core client for 30 sec - exiting 19:28:44 (10724): No heartbeat from core client for 30 sec - exiting 19:28:45 (10724): No heartbeat from core client for 30 sec - exiting 19:28:46 (10724): No heartbeat from core client for 30 sec - exiting 19:28:47 (10724): No heartbeat from core client for 30 sec - exiting 19:28:48 (10724): No heartbeat from core client for 30 sec - exiting 19:28:49 (10724): No heartbeat from core client for 30 sec - exiting 19:28:50 (10724): No heartbeat from core client for 30 sec - exiting 19:28:51 (10724): No heartbeat from core client for 30 sec - exiting 19:28:52 (10724): No heartbeat from core client for 30 sec - exiting 19:28:53 (10724): No heartbeat from core client for 30 sec - exiting 19:28:54 (10724): No heartbeat from core client for 30 sec - exiting 19:28:55 (10724): No heartbeat from core client for 30 sec - exiting 19:28:56 (10724): No heartbeat from core client for 30 sec - exiting 19:28:57 (10724): No heartbeat from core client for 30 sec - exiting 19:29:29 (10724): No heartbeat from core client for 30 sec - exiting 19:29:30 (10724): No heartbeat from core client for 30 sec - exiting 19:29:31 (10724): No heartbeat from core client for 30 sec - exiting 19:29:32 (10724): No heartbeat from core client for 30 sec - exiting 19:29:33 (10724): No heartbeat from core client for 30 sec - exiting 19:29:34 (10724): No heartbeat from core client for 30 sec - exiting 19:29:35 (10724): No heartbeat from core client for 30 sec - exiting 19:29:36 (10724): No heartbeat from core client for 30 sec - exiting cpdnmonitor: error reading file D:\BOINC Data/projects/climateprediction.net/hadcm3n_o54x_2140_40_008270374/dataout/ocean_restart.day CPDN Monitor - No 'heartbeat' from BOINC... cpdnmonitor: error reading file D:\BOINC Data/projects/climateprediction.net/hadcm3n_o54x_2140_40_008270374/dataout/ocean_restart.day BUFFIN: C I/O Error ferror - Unit 41 - Return code = 32 Model crashed: READDUMP: BAD BUFFIN OF DATA tmp/pipe_dummy 2048 cpdnmonitor: error reading file D:\BOINC Data/projects/climateprediction.net/hadcm3n_o54x_2140_40_008270374/dataout/ocean_restart.day BUFFIN: C I/O Error ferror - Unit 41 - Return code = 32 Model crashed: READDUMP: BAD BUFFIN OF DATA tmp/pipe_dummy 2048 cpdnmonitor: error reading file D:\BOINC Data/projects/climateprediction.net/hadcm3n_o54x_2140_40_008270374/dataout/ocean_restart.day BUFFIN: C I/O Error ferror - Unit 41 - Return code = 32 Model crashed: READDUMP: BAD BUFFIN OF DATA tmp/pipe_dummy 2048 cpdnmonitor: error reading file D:\BOINC Data/projects/climateprediction.net/hadcm3n_o54x_2140_40_008270374/dataout/ocean_restart.day BUFFIN: C I/O Error ferror - Unit 41 - Return code = 32 Model crashed: READDUMP: BAD BUFFIN OF DATA tmp/pipe_dummy 2048 cpdnmonitor: error reading file D:\BOINC Data/projects/climateprediction.net/hadcm3n_o54x_2140_40_008270374/dataout/ocean_restart.day BUFFIN: C I/O Error ferror - Unit 41 - Return code = 32 Model crashed: READDUMP: BAD BUFFIN OF DATA tmp/pipe_dummy 2048 cpdnmonitor: error reading file D:\BOINC Data/projects/climateprediction.net/hadcm3n_o54x_2140_40_008270374/dataout/ocean_restart.day BUFFIN: C I/O Error ferror - Unit 41 - Return code = 32 Model crashed: READDUMP: BAD BUFFIN OF DATA tmp/pipe_dummy 2048 Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
07 Sep 2013 02:27:26	1136891	16002256	hadcm3n_o54x_2140_40_008270374_3	77,760	131,461	1.6906
05 Sep 2013 18:46:55	1136891	16002256	hadcm3n_o54x_2140_40_008270374_3	51,840	87,810	1.6939
04 Sep 2013 18:04:33	1136891	16002256	hadcm3n_o54x_2140_40_008270374_3	25,920	44,161	1.7037