Task 13359853

Name	hadcm3n_o4b6_1940_40_007447732_0
Workunit	7645235
Created	9 Sep 2011, 18:54:45 UTC
Sent	9 Sep 2011, 19:09:47 UTC
Report deadline	10 Dec 2011, 2:36:58 UTC
Received	14 Sep 2011, 7:51:05 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1122757
Run time	2 days 20 hours 17 min 42 sec
CPU time	2 days 19 hours 23 min 38 sec
Validate state	Invalid
Credit	933.12
Device peak FLOPS	1.70 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.12.34</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... 14:27:35 (5596): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:00:43 (6952): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Atmos Hold Restart file rename failed on atmos_restart.hold CPDN Monitor - Quit request from BOINC... Atmos Hold Restart file rename failed on atmos_restart.hold 21:04:34 (6428): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:04:40 (6428): No heartbeat from core client for 30 sec - exiting 21:04:41 (6428): No heartbeat from core client for 30 sec - exiting 21:04:42 (6428): No heartbeat from core client for 30 sec - exiting 21:04:43 (6428): No heartbeat from core client for 30 sec - exiting 21:04:44 (6428): No heartbeat from core client for 30 sec - exiting 21:04:45 (6428): No heartbeat from core client for 30 sec - exiting 21:04:46 (6428): No heartbeat from core client for 30 sec - exiting 21:04:47 (6428): No heartbeat from core client for 30 sec - exiting 21:04:48 (6428): No heartbeat from core client for 30 sec - exiting 21:04:50 (6428): No heartbeat from core client for 30 sec - exiting 21:04:51 (6428): No heartbeat from core client for 30 sec - exiting 21:04:52 (6428): No heartbeat from core client for 30 sec - exiting 21:04:53 (6428): No heartbeat from core client for 30 sec - exiting 21:04:54 (6428): No heartbeat from core client for 30 sec - exiting 21:04:55 (6428): No heartbeat from core client for 30 sec - exiting 21:04:56 (6428): No heartbeat from core client for 30 sec - exiting 21:04:57 (6428): No heartbeat from core client for 30 sec - exiting 21:04:58 (6428): No heartbeat from core client for 30 sec - exiting 21:04:59 (6428): No heartbeat from core client for 30 sec - exiting 21:05:00 (6428): No heartbeat from core client for 30 sec - exiting 21:05:02 (6428): No heartbeat from core client for 30 sec - exiting 21:05:03 (6428): No heartbeat from core client for 30 sec - exiting 21:05:04 (6428): No heartbeat from core client for 30 sec - exiting 23:40:03 (1848): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:44:13 (6660): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:56:54 (4844): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:57:10 (4844): No heartbeat from core client for 30 sec - exiting 23:57:11 (4844): No heartbeat from core client for 30 sec - exiting 23:57:12 (4844): No heartbeat from core client for 30 sec - exiting 23:57:14 (4844): No heartbeat from core client for 30 sec - exiting 23:57:15 (4844): No heartbeat from core client for 30 sec - exiting 23:57:16 (4844): No heartbeat from core client for 30 sec - exiting 23:57:17 (4844): No heartbeat from core client for 30 sec - exiting 23:57:18 (4844): No heartbeat from core client for 30 sec - exiting 23:57:19 (4844): No heartbeat from core client for 30 sec - exiting 23:57:20 (4844): No heartbeat from core client for 30 sec - exiting Atmos Hold Restart file rename failed on atmos_restart.hold 00:07:08 (1660): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:07:09 (1660): No heartbeat from core client for 30 sec - exiting 00:07:10 (1660): No heartbeat from core client for 30 sec - exiting 00:07:11 (1660): No heartbeat from core client for 30 sec - exiting 00:07:13 (1660): No heartbeat from core client for 30 sec - exiting 00:07:14 (1660): No heartbeat from core client for 30 sec - exiting 00:07:15 (1660): No heartbeat from core client for 30 sec - exiting 00:07:16 (1660): No heartbeat from core client for 30 sec - exiting 00:07:17 (1660): No heartbeat from core client for 30 sec - exiting 00:07:18 (1660): No heartbeat from core client for 30 sec - exiting 00:07:19 (1660): No heartbeat from core client for 30 sec - exiting 00:07:55 (968): No heartbeat from core client for 30 sec - exiting 00:08:04 (968): No heartbeat from core client for 30 sec - exiting 00:08:05 (968): No heartbeat from core client for 30 sec - exiting 00:08:07 (968): No heartbeat from core client for 30 sec - exiting 00:08:08 (968): No heartbeat from core client for 30 sec - exiting 00:08:09 (968): No heartbeat from core client for 30 sec - exiting 00:08:10 (968): No heartbeat from core client for 30 sec - exiting 00:08:11 (968): No heartbeat from core client for 30 sec - exiting 00:08:12 (968): No heartbeat from core client for 30 sec - exiting 00:08:13 (968): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Signal 22 received, exiting... Called boinc_finish 00:09:21 (492): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4112, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4112, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4112, iMonCtr=1 Model crash detected, will try to restart... 00:10:28 (4112): No heartbeat from core client for 30 sec - exiting 00:10:29 (4112): No heartbeat from core client for 30 sec - exiting 00:10:30 (4112): No heartbeat from core client for 30 sec - exiting 00:10:31 (4112): No heartbeat from core client for 30 sec - exiting 00:10:32 (4112): No heartbeat from core client for 30 sec - exiting 00:10:33 (4112): No heartbeat from core client for 30 sec - exiting 00:10:34 (4112): No heartbeat from core client for 30 sec - exiting 00:10:35 (4112): No heartbeat from core client for 30 sec - exiting 00:10:36 (4112): No heartbeat from core client for 30 sec - exiting 00:10:38 (4112): No heartbeat from core client for 30 sec - exiting 00:10:39 (4112): No heartbeat from core client for 30 sec - exiting 00:10:40 (4112): No heartbeat from core client for 30 sec - exiting 00:10:41 (4112): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:10:42 (4112): No heartbeat from core client for 30 sec - exiting 00:10:43 (4112): No heartbeat from core client for 30 sec - exiting Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5964, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5964, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5964, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
14 Sep 2011 06:49:13	1122757	13359853	hadcm3n_o4b6_1940_40_007447732_0	77,760	240,823	3.0970
13 Sep 2011 07:27:22	1122757	13359853	hadcm3n_o4b6_1940_40_007447732_0	51,840	160,843	3.1027
12 Sep 2011 09:06:29	1122757	13359853	hadcm3n_o4b6_1940_40_007447732_0	25,920	80,447	3.1037