Task 16051828

Name	hadcm3n_o50b_1980_40_008385627_4
Workunit	8536486
Created	1 Oct 2013, 7:30:51 UTC
Sent	1 Oct 2013, 7:45:49 UTC
Report deadline	31 Dec 2013, 15:13:00 UTC
Received	3 Oct 2013, 12:02:57 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1194844
Run time	1 days 12 hours 41 min 55 sec
CPU time	1 days 11 hours 54 min 55 sec
Validate state	Invalid
Credit	1,555.20
Device peak FLOPS	3.81 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.64</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 05:12:43 (142288): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:16:07 (143308): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:17:38 (140608): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:18:03 (140608): No heartbeat from core client for 30 sec - exiting 05:18:04 (140608): No heartbeat from core client for 30 sec - exiting 05:18:05 (140608): No heartbeat from core client for 30 sec - exiting 05:19:14 (142508): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:20:05 (142508): No heartbeat from core client for 30 sec - exiting 05:20:06 (142508): No heartbeat from core client for 30 sec - exiting 05:20:07 (142508): No heartbeat from core client for 30 sec - exiting 05:20:08 (142508): No heartbeat from core client for 30 sec - exiting 05:20:09 (142508): No heartbeat from core client for 30 sec - exiting 05:20:44 (143176): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:21:21 (143284): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:22:38 (144176): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:23:43 (143156): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:25:44 (143460): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:28:48 (142404): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:29:48 (143708): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:33:19 (143856): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:35:06 (144128): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:37:09 (143540): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:37:21 (143540): No heartbeat from core client for 30 sec - exiting 05:37:22 (143540): No heartbeat from core client for 30 sec - exiting 05:38:05 (145368): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:38:41 (144792): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:38:55 (144792): No heartbeat from core client for 30 sec - exiting 05:38:56 (144792): No heartbeat from core client for 30 sec - exiting 05:39:42 (144504): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:41:32 (144280): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:41:36 (144280): No heartbeat from core client for 30 sec - exiting 05:41:37 (144280): No heartbeat from core client for 30 sec - exiting 05:41:38 (144280): No heartbeat from core client for 30 sec - exiting 05:41:39 (144280): No heartbeat from core client for 30 sec - exiting 05:41:40 (144280): No heartbeat from core client for 30 sec - exiting 05:41:41 (144280): No heartbeat from core client for 30 sec - exiting 05:41:42 (144280): No heartbeat from core client for 30 sec - exiting 05:41:43 (144280): No heartbeat from core client for 30 sec - exiting 05:41:44 (144280): No heartbeat from core client for 30 sec - exiting 05:41:45 (144280): No heartbeat from core client for 30 sec - exiting 05:42:19 (144328): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:42:59 (144328): No heartbeat from core client for 30 sec - exiting 05:43:33 (139664): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:44:45 (144932): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:45:21 (144928): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:45:47 (144928): No heartbeat from core client for 30 sec - exiting 05:46:24 (143592): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:47:14 (143592): No heartbeat from core client for 30 sec - exiting 05:47:15 (143592): No heartbeat from core client for 30 sec - exiting 05:48:32 (145084): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:49:46 (145684): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:50:40 (145268): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:51:05 (145268): No heartbeat from core client for 30 sec - exiting 05:51:43 (144976): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:53:00 (145056): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:53:58 (145056): No heartbeat from core client for 30 sec - exiting 05:54:53 (146192): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:56:19 (145480): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:57:23 (145480): No heartbeat from core client for 30 sec - exiting 05:57:57 (146424): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:59:56 (144468): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:00:44 (144916): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:01:46 (145912): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:02:20 (147248): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:03:35 (146444): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Atmos Hold Restart file rename failed on atmos_restart.hold 06:04:55 (146332): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:04:56 (146332): No heartbeat from core client for 30 sec - exiting 06:04:57 (146332): No heartbeat from core client for 30 sec - exiting 06:04:58 (146332): No heartbeat from core client for 30 sec - exiting 06:04:59 (146332): No heartbeat from core client for 30 sec - exiting 06:05:00 (146332): No heartbeat from core client for 30 sec - exiting 06:05:01 (146332): No heartbeat from core client for 30 sec - exiting 06:05:02 (146332): No heartbeat from core client for 30 sec - exiting 06:05:03 (146332): No heartbeat from core client for 30 sec - exiting 06:05:04 (146332): No heartbeat from core client for 30 sec - exiting 06:05:05 (146332): No heartbeat from core client for 30 sec - exiting Atmos Hold Restart file rename failed on atmos_restart.hold 06:07:53 (146260): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:07:54 (146260): No heartbeat from core client for 30 sec - exiting 06:07:55 (146260): No heartbeat from core client for 30 sec - exiting 06:07:56 (146260): No heartbeat from core client for 30 sec - exiting 06:07:57 (146260): No heartbeat from core client for 30 sec - exiting 06:07:58 (146260): No heartbeat from core client for 30 sec - exiting 06:07:59 (146260): No heartbeat from core client for 30 sec - exiting 06:08:00 (146260): No heartbeat from core client for 30 sec - exiting 06:08:01 (146260): No heartbeat from core client for 30 sec - exiting 06:08:02 (146260): No heartbeat from core client for 30 sec - exiting 06:08:03 (146260): No heartbeat from core client for 30 sec - exiting 06:10:43 (142120): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:10:44 (142120): No heartbeat from core client for 30 sec - exiting 06:10:45 (142120): No heartbeat from core client for 30 sec - exiting 06:10:46 (142120): No heartbeat from core client for 30 sec - exiting 06:10:47 (142120): No heartbeat from core client for 30 sec - exiting 06:10:48 (142120): No heartbeat from core client for 30 sec - exiting 06:10:49 (142120): No heartbeat from core client for 30 sec - exiting 06:10:50 (142120): No heartbeat from core client for 30 sec - exiting 06:10:51 (142120): No heartbeat from core client for 30 sec - exiting 06:10:52 (142120): No heartbeat from core client for 30 sec - exiting 06:10:53 (142120): No heartbeat from core client for 30 sec - exiting 06:12:54 (147628): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:16:32 (147728): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:16:33 (147728): No heartbeat from core client for 30 sec - exiting 06:16:34 (147728): No heartbeat from core client for 30 sec - exiting 06:16:35 (147728): No heartbeat from core client for 30 sec - exiting 06:16:36 (147728): No heartbeat from core client for 30 sec - exiting 06:16:37 (147728): No heartbeat from core client for 30 sec - exiting 06:16:38 (147728): No heartbeat from core client for 30 sec - exiting 06:16:39 (147728): No heartbeat from core client for 30 sec - exiting 06:16:40 (147728): No heartbeat from core client for 30 sec - exiting 06:16:41 (147728): No heartbeat from core client for 30 sec - exiting 06:16:42 (147728): No heartbeat from core client for 30 sec - exiting 06:23:16 (147500): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:23:17 (147500): No heartbeat from core client for 30 sec - exiting 06:23:18 (147500): No heartbeat from core client for 30 sec - exiting 06:23:19 (147500): No heartbeat from core client for 30 sec - exiting 06:23:20 (147500): No heartbeat from core client for 30 sec - exiting 06:23:21 (147500): No heartbeat from core client for 30 sec - exiting 06:23:22 (147500): No heartbeat from core client for 30 sec - exiting 06:23:23 (147500): No heartbeat from core client for 30 sec - exiting 06:23:24 (147500): No heartbeat from core client for 30 sec - exiting 06:23:25 (147500): No heartbeat from core client for 30 sec - exiting 06:23:26 (147500): No heartbeat from core client for 30 sec - exiting 06:28:38 (141484): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:32:56 (146572): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 02:51:05 (152344): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 13:15:09 (153968): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:18:57 (156932): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Atmos Hold Restart file rename failed on atmos_restart.hold 13:21:26 (158416): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:21:27 (158416): No heartbeat from core client for 30 sec - exiting 13:21:28 (158416): No heartbeat from core client for 30 sec - exiting 13:21:29 (158416): No heartbeat from core client for 30 sec - exiting 13:21:30 (158416): No heartbeat from core client for 30 sec - exiting 13:21:31 (158416): No heartbeat from core client for 30 sec - exiting 13:21:32 (158416): No heartbeat from core client for 30 sec - exiting 13:21:33 (158416): No heartbeat from core client for 30 sec - exiting Atmos Hold Restart file rename failed on atmos_restart.hold Model crashed: U_MODEL: Illegal combination of submodels tmp/pipe_dummy 2048 13:23:09 (158224): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:23:10 (158224): No heartbeat from core client for 30 sec - exiting 13:23:11 (158224): No heartbeat from core client for 30 sec - exiting 13:23:12 (158224): No heartbeat from core client for 30 sec - exiting 13:23:13 (158224): No heartbeat from core client for 30 sec - exiting 13:23:14 (158224): No heartbeat from core client for 30 sec - exiting 13:23:15 (158224): No heartbeat from core client for 30 sec - exiting 13:23:16 (158224): No heartbeat from core client for 30 sec - exiting 13:23:17 (158224): No heartbeat from core client for 30 sec - exiting 13:23:18 (158224): No heartbeat from core client for 30 sec - exiting 13:23:19 (158224): No heartbeat from core client for 30 sec - exiting 13:24:25 (157756): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Atmos Hold Restart file rename failed on atmos_restart.hold Model crashed: U_MODEL: Illegal combination of submodels tmp/pipe_dummy 2048 Atmos Hold Restart file rename failed on atmos_restart.hold Model crashed: U_MODEL: Illegal combination of submodels tmp/pipe_dummy 2048 Atmos Hold Restart file rename failed on atmos_restart.hold Model crashed: U_MODEL: Illegal combination of submodels tmp/pipe_dummy 2048 13:27:22 (156272): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:27:23 (156272): No heartbeat from core client for 30 sec - exiting 13:27:24 (156272): No heartbeat from core client for 30 sec - exiting 13:27:25 (156272): No heartbeat from core client for 30 sec - exiting 13:27:26 (156272): No heartbeat from core client for 30 sec - exiting 13:27:27 (156272): No heartbeat from core client for 30 sec - exiting 13:27:28 (156272): No heartbeat from core client for 30 sec - exiting 13:27:29 (156272): No heartbeat from core client for 30 sec - exiting 13:27:30 (156272): No heartbeat from core client for 30 sec - exiting 13:27:31 (156272): No heartbeat from core client for 30 sec - exiting 13:27:32 (156272): No heartbeat from core client for 30 sec - exiting 13:28:45 (157844): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Atmos Hold Restart file rename failed on atmos_restart.hold Model crashed: U_MODEL: Illegal combination of submodels tmp/pipe_dummy 2048 13:33:45 (158560): No heartbeat from core client for 30 sec - exiting 13:33:46 (158560): No heartbeat from core client for 30 sec - exiting 13:34:04 (158560): No heartbeat from core client for 30 sec - exiting 13:34:05 (158560): No heartbeat from core client for 30 sec - exiting 13:34:06 (158560): No heartbeat from core client for 30 sec - exiting 13:34:07 (158560): No heartbeat from core client for 30 sec - exiting 13:34:08 (158560): No heartbeat from core client for 30 sec - exiting 13:34:09 (158560): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Atmos Hold Restart file rename failed on atmos_restart.hold Model crashed: U_MODEL: Illegal combination of submodels tmp/pipe_dummy 2048 Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
03 Oct 2013 11:02:18	1194844	16051828	hadcm3n_o50b_1980_40_008385627_4	129,600	128,375	0.9905
03 Oct 2013 03:33:50	1194844	16051828	hadcm3n_o50b_1980_40_008385627_4	103,680	102,228	0.9860
02 Oct 2013 16:31:54	1194844	16051828	hadcm3n_o50b_1980_40_008385627_4	77,760	76,231	0.9803
02 Oct 2013 06:07:52	1194844	16051828	hadcm3n_o50b_1980_40_008385627_4	51,840	51,909	1.0013
01 Oct 2013 21:25:58	1194844	16051828	hadcm3n_o50b_1980_40_008385627_4	25,920	26,622	1.0271