Task 17229577

Name	hadcm3n_s0ld_1940_40_009093930_0
Workunit	9224266
Created	20 Oct 2014, 16:53:10 UTC
Sent	20 Oct 2014, 18:45:09 UTC
Report deadline	20 Jan 2015, 2:12:20 UTC
Received	7 Nov 2014, 16:25:16 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1307431
Run time	3 days 4 hours 38 min 20 sec
CPU time	2 days 16 hours 4 min 19 sec
Validate state	Invalid
Credit	2,799.36
Device peak FLOPS	3.36 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.2.42</core_client_version> <![CDATA[ <message> Das Gerät erkennt den Befehl nicht. (0x16) - exit code 22 (0x16) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Ocean Restart file copy failed on s0ldko.dae0cr0 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Ocean Restart file copy failed on s0ldko.dae1cg0 Atmos Hold Restart file rename failed on atmos_restart.hold Ocean Restart file copy failed on s0ldko.dae1cs0 Atmos Hold Restart file rename failed on atmos_restart.hold Ocean Restart file copy failed on s0ldko.dae21b0 Atmos Hold Restart file rename failed on atmos_restart.hold CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Ocean Restart file copy failed on s0ldko.dae2aj0 Atmos Hold Restart file rename failed on atmos_restart.hold Atmos Hold Restart file rename failed on atmos_restart.hold CPDN Monitor - Quit request from BOINC... Ocean Restart file copy failed on s0ldko.dae31l0 Atmos Hold Restart file rename failed on atmos_restart.hold CPDN Monitor - Quit request from BOINC... Ocean Restart file copy failed on s0ldko.dae32a0 Atmos Hold Restart file rename failed on atmos_restart.hold Atmos Hold Restart file rename failed on atmos_restart.hold Atmos Hold Restart file rename failed on atmos_restart.hold CPDN Monitor - Quit request from BOINC... Atmos Hold Restart file rename failed on atmos_restart.hold Atmos Hold Restart file rename failed on atmos_restart.hold Ocean Restart file copy failed on s0ldko.dae35j0 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Ocean Restart file copy failed on s0ldko.dae39i0 Ocean Restart file copy failed on s0ldko.dae3bh0 Atmos Hold Restart file rename failed on atmos_restart.hold Atmos Hold Restart file rename failed on atmos_restart.hold Atmos Restart file copy failed on s0ldka.dae3c10 Atmos Hold Restart file rename failed on atmos_restart.hold Atmos Restart file copy failed on s0ldka.dae3c40 CPDN Monitor - Quit request from BOINC... Ocean Restart file copy failed on s0ldko.dae41k0 Ocean Restart file copy failed on s0ldko.dae42d0 Ocean Restart file copy failed on s0ldko.dae42m0 Atmos Hold Restart file rename failed on atmos_restart.hold Atmos Hold Restart file rename failed on atmos_restart.hold Atmos Hold Restart file rename failed on atmos_restart.hold Atmos Hold Restart file rename failed on atmos_restart.hold Atmos Hold Restart file rename failed on atmos_restart.hold Ocean Restart file copy failed on s0ldko.dae43l0 Atmos Hold Restart file rename failed on atmos_restart.hold Atmos Hold Restart file rename failed on atmos_restart.hold Atmos Hold Restart file rename failed on atmos_restart.hold Ocean Restart file copy failed on s0ldko.dae4520 Ocean Restart file copy failed on s0ldko.dae45l0 Atmos Hold Restart file rename failed on atmos_restart.hold Ocean Restart file copy failed on s0ldko.dae4730 Atmos Hold Restart file rename failed on atmos_restart.hold Atmos Hold Restart file rename failed on atmos_restart.hold Atmos Hold Restart file rename failed on atmos_restart.hold Atmos Hold Restart file rename failed on atmos_restart.hold Ocean Restart file copy failed on s0ldko.dae4bf0 Atmos Hold Restart file rename failed on atmos_restart.hold Atmos Hold Restart file rename failed on atmos_restart.hold CPDN Monitor - Quit request from BOINC... Atmos Hold Restart file rename failed on atmos_restart.hold CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Atmos Hold Restart file rename failed on atmos_restart.hold CPDN Monitor - Quit request from BOINC... Atmos Hold Restart file rename failed on atmos_restart.hold Ocean Restart file copy failed on s0ldko.dae55s0 Atmos Hold Restart file rename failed on atmos_restart.hold Atmos Restart file copy failed on s0ldka.dae5610 Atmos Hold Restart file rename failed on atmos_restart.hold Atmos Hold Restart file rename failed on atmos_restart.hold Atmos Hold Restart file rename failed on atmos_restart.hold CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Ocean Restart file copy failed on s0ldko.dae59f0 Ocean Restart file copy failed on s0ldko.dae5a30 Atmos Hold Restart file rename failed on atmos_restart.hold Atmos Hold Restart file rename failed on atmos_restart.hold CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 19:22:28 (2324): No heartbeat from core client for 30 sec - exiting 19:22:29 (2324): No heartbeat from core client for 30 sec - exiting 19:22:30 (2324): No heartbeat from core client for 30 sec - exiting 19:22:32 (2324): No heartbeat from core client for 30 sec - exiting 19:22:33 (2324): No heartbeat from core client for 30 sec - exiting 19:22:34 (2324): No heartbeat from core client for 30 sec - exiting 19:22:35 (2324): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:22:36 (2324): No heartbeat from core client for 30 sec - exiting 19:22:37 (2324): No heartbeat from core client for 30 sec - exiting 19:22:38 (2324): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 13:29:19 (6760): No heartbeat from core client for 30 sec - exiting 13:29:21 (6760): No heartbeat from core client for 30 sec - exiting 13:29:22 (6760): No heartbeat from core client for 30 sec - exiting 13:29:23 (6760): No heartbeat from core client for 30 sec - exiting 13:29:24 (6760): No heartbeat from core client for 30 sec - exiting 13:29:25 (6760): No heartbeat from core client for 30 sec - exiting 13:29:26 (6760): No heartbeat from core client for 30 sec - exiting 13:29:27 (6760): No heartbeat from core client for 30 sec - exiting 13:29:28 (6760): No heartbeat from core client for 30 sec - exiting 13:29:29 (6760): No heartbeat from core client for 30 sec - exiting 13:29:30 (6760): No heartbeat from core client for 30 sec - exiting 13:29:31 (6760): No heartbeat from core client for 30 sec - exiting 13:29:33 (6760): No heartbeat from core client for 30 sec - exiting 13:29:34 (6760): No heartbeat from core client for 30 sec - exiting 13:29:35 (6760): No heartbeat from core client for 30 sec - exiting 13:29:36 (6760): No heartbeat from core client for 30 sec - exiting 13:29:37 (6760): No heartbeat from core client for 30 sec - exiting 13:29:38 (6760): No heartbeat from core client for 30 sec - exiting 13:29:39 (6760): No heartbeat from core client for 30 sec - exiting 13:29:40 (6760): No heartbeat from core client for 30 sec - exiting 13:29:41 (6760): No heartbeat from core client for 30 sec - exiting 13:29:42 (6760): No heartbeat from core client for 30 sec - exiting 13:29:43 (6760): No heartbeat from core client for 30 sec - exiting 13:29:45 (6760): No heartbeat from core client for 30 sec - exiting 13:29:46 (6760): No heartbeat from core client for 30 sec - exiting 13:29:47 (6760): No heartbeat from core client for 30 sec - exiting 13:29:48 (6760): No heartbeat from core client for 30 sec - exiting 13:29:49 (6760): No heartbeat from core client for 30 sec - exiting 13:29:50 (6760): No heartbeat from core client for 30 sec - exiting 13:29:51 (6760): No heartbeat from core client for 30 sec - exiting 13:29:52 (6760): No heartbeat from core client for 30 sec - exiting 13:29:53 (6760): No heartbeat from core client for 30 sec - exiting 13:29:54 (6760): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 13:01:32 (5716): No heartbeat from core client for 30 sec - exiting 13:01:33 (5716): No heartbeat from core client for 30 sec - exiting 13:01:34 (5716): No heartbeat from core client for 30 sec - exiting 13:01:35 (5716): No heartbeat from core client for 30 sec - exiting 13:01:36 (5716): No heartbeat from core client for 30 sec - exiting 13:01:37 (5716): No heartbeat from core client for 30 sec - exiting 13:01:38 (5716): No heartbeat from core client for 30 sec - exiting 13:01:39 (5716): No heartbeat from core client for 30 sec - exiting 13:01:41 (5716): No heartbeat from core client for 30 sec - exiting 13:01:42 (5716): No heartbeat from core client for 30 sec - exiting 13:01:43 (5716): No heartbeat from core client for 30 sec - exiting 13:01:44 (5716): No heartbeat from core client for 30 sec - exiting 13:01:45 (5716): No heartbeat from core client for 30 sec - exiting 13:01:46 (5716): No heartbeat from core client for 30 sec - exiting 13:01:47 (5716): No heartbeat from core client for 30 sec - exiting 13:01:48 (5716): No heartbeat from core client for 30 sec - exiting 13:01:49 (5716): No heartbeat from core client for 30 sec - exiting 13:01:50 (5716): No heartbeat from core client for 30 sec - exiting 13:01:52 (5716): No heartbeat from core client for 30 sec - exiting 13:01:53 (5716): No heartbeat from core client for 30 sec - exiting 13:01:54 (5716): No heartbeat from core client for 30 sec - exiting 13:01:55 (5716): No heartbeat from core client for 30 sec - exiting 13:01:56 (5716): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3548, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3548, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3548, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3548, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4044, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4044, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4044, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
07 Nov 2014 16:30:07	1307431	17229577	hadcm3n_s0ld_1940_40_009093930_0	233,280	211,125	0.9050
06 Nov 2014 13:08:28	1307431	17229577	hadcm3n_s0ld_1940_40_009093930_0	207,360	187,516	0.9043
06 Nov 2014 13:08:28	1307431	17229577	hadcm3n_s0ld_1940_40_009093930_0	181,440	163,946	0.9036
06 Nov 2014 13:08:28	1307431	17229577	hadcm3n_s0ld_1940_40_009093930_0	155,520	141,711	0.9112
01 Nov 2014 22:23:06	1307431	17229577	hadcm3n_s0ld_1940_40_009093930_0	129,600	118,071	0.9110
01 Nov 2014 12:38:12	1307431	17229577	hadcm3n_s0ld_1940_40_009093930_0	103,680	94,195	0.9085
01 Nov 2014 12:38:12	1307431	17229577	hadcm3n_s0ld_1940_40_009093930_0	77,760	70,410	0.9055
29 Oct 2014 18:34:34	1307431	17229577	hadcm3n_s0ld_1940_40_009093930_0	51,840	46,935	0.9054
27 Oct 2014 20:22:14	1307431	17229577	hadcm3n_s0ld_1940_40_009093930_0	25,920	23,260	0.8974