Task 13619858

Name	hadcm3n_ybz4_1940_40_007539769_2
Workunit	7737001
Created	9 Nov 2011, 4:36:11 UTC
Sent	9 Nov 2011, 4:44:50 UTC
Report deadline	8 Feb 2012, 12:12:01 UTC
Received	15 Nov 2011, 17:33:29 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1176150
Run time	1 days 7 hours 40 min 44 sec
CPU time	1 days 7 hours 40 min 44 sec
Validate state	Invalid
Credit	622.08
Device peak FLOPS	2.47 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.12.34</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> 17:24:53 (672): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:30:14 (1576): No heartbeat from core client for 30 sec - exiting 17:30:17 (1576): No heartbeat from core client for 30 sec - exiting 17:30:18 (1576): No heartbeat from core client for 30 sec - exiting 17:30:19 (1576): No heartbeat from core client for 30 sec - exiting 17:30:20 (1576): No heartbeat from core client for 30 sec - exiting 17:30:25 (1576): No heartbeat from core client for 30 sec - exiting 17:30:26 (1576): No heartbeat from core client for 30 sec - exiting 17:30:27 (1576): No heartbeat from core client for 30 sec - exiting 17:30:28 (1576): No heartbeat from core client for 30 sec - exiting 17:30:29 (1576): No heartbeat from core client for 30 sec - exiting 17:30:30 (1576): No heartbeat from core client for 30 sec - exiting 17:30:31 (1576): No heartbeat from core client for 30 sec - exiting 17:30:32 (1576): No heartbeat from core client for 30 sec - exiting 17:30:33 (1576): No heartbeat from core client for 30 sec - exiting 17:30:34 (1576): No heartbeat from core client for 30 sec - exiting 17:30:36 (1576): No heartbeat from core client for 30 sec - exiting 17:30:37 (1576): No heartbeat from core client for 30 sec - exiting 17:30:38 (1576): No heartbeat from core client for 30 sec - exiting 17:30:39 (1576): No heartbeat from core client for 30 sec - exiting 17:30:40 (1576): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:34:47 (2464): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:39:12 (2176): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:46:18 (2664): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:21:10 (3952): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:26:10 (3728): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:28:40 (552): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:25:12 (2868): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:26:48 (2956): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:25:25 (3756): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:25:53 (2908): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:27:37 (3712): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 04:22:38 (4780): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:24:31 (3004): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:25:30 (1968): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Atmos Hold Restart file rename failed on atmos_restart.hold 04:27:11 (4396): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:30:26 (3880): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:31:11 (2684): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Atmos Hold Restart file rename failed on atmos_restart.hold 04:34:12 (2612): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:39:23 (3500): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:39:24 (3500): No heartbeat from core client for 30 sec - exiting Atmos Hold Restart file rename failed on atmos_restart.hold 04:44:54 (2404): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:44:55 (2404): No heartbeat from core client for 30 sec - exiting 04:44:56 (2404): No heartbeat from core client for 30 sec - exiting 04:44:57 (2404): No heartbeat from core client for 30 sec - exiting 04:49:10 (4764): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:50:46 (3076): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:51:45 (4064): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:52:45 (3268): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:53:43 (3780): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:53:45 (3780): No heartbeat from core client for 30 sec - exiting 04:57:12 (3112): No heartbeat from core client for 30 sec - exiting 04:57:13 (3112): No heartbeat from core client for 30 sec - exiting 04:57:15 (3112): No heartbeat from core client for 30 sec - exiting 04:57:16 (3112): No heartbeat from core client for 30 sec - exiting 04:57:50 (3112): No heartbeat from core client for 30 sec - exiting 04:57:51 (3112): No heartbeat from core client for 30 sec - exiting 04:57:52 (3112): No heartbeat from core client for 30 sec - exiting 04:57:53 (3112): No heartbeat from core client for 30 sec - exiting 04:57:54 (3112): No heartbeat from core client for 30 sec - exiting 04:57:55 (3112): No heartbeat from core client for 30 sec - exiting 04:57:56 (3112): No heartbeat from core client for 30 sec - exiting 04:57:57 (3112): No heartbeat from core client for 30 sec - exiting 04:57:58 (3112): No heartbeat from core client for 30 sec - exiting 04:57:59 (3112): No heartbeat from core client for 30 sec - exiting 04:58:00 (3112): No heartbeat from core client for 30 sec - exiting 04:58:02 (3112): No heartbeat from core client for 30 sec - exiting 04:58:03 (3112): No heartbeat from core client for 30 sec - exiting 04:58:04 (3112): No heartbeat from core client for 30 sec - exiting 04:58:05 (3112): No heartbeat from core client for 30 sec - exiting 04:58:06 (3112): No heartbeat from core client for 30 sec - exiting 04:58:07 (3112): No heartbeat from core client for 30 sec - exiting 04:58:08 (3112): No heartbeat from core client for 30 sec - exiting 04:58:09 (3112): No heartbeat from core client for 30 sec - exiting 04:58:10 (3112): No heartbeat from core client for 30 sec - exiting 04:58:11 (3112): No heartbeat from core client for 30 sec - exiting 04:58:13 (3112): No heartbeat from core client for 30 sec - exiting 04:58:14 (3112): No heartbeat from core client for 30 sec - exiting 04:58:15 (3112): No heartbeat from core client for 30 sec - exiting 04:58:16 (3112): No heartbeat from core client for 30 sec - exiting 04:58:17 (3112): No heartbeat from core client for 30 sec - exiting 04:58:18 (3112): No heartbeat from core client for 30 sec - exiting 04:58:19 (3112): No heartbeat from core client for 30 sec - exiting 04:58:20 (3112): No heartbeat from core client for 30 sec - exiting 04:58:21 (3112): No heartbeat from core client for 30 sec - exiting 04:58:22 (3112): No heartbeat from core client for 30 sec - exiting 04:58:23 (3112): No heartbeat from core client for 30 sec - exiting 04:58:25 (3112): No heartbeat from core client for 30 sec - exiting 04:58:26 (3112): No heartbeat from core client for 30 sec - exiting 04:58:27 (3112): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:58:28 (3112): No heartbeat from core client for 30 sec - exiting 05:01:29 (900): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:02:07 (2472): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:04:50 (1088): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:08:00 (4944): No heartbeat from core client for 30 sec - exiting 05:08:01 (4944): No heartbeat from core client for 30 sec - exiting 05:08:03 (4944): No heartbeat from core client for 30 sec - exiting 05:08:04 (4944): No heartbeat from core client for 30 sec - exiting 05:08:05 (4944): No heartbeat from core client for 30 sec - exiting 05:08:06 (4944): No heartbeat from core client for 30 sec - exiting 05:08:07 (4944): No heartbeat from core client for 30 sec - exiting 05:08:08 (4944): No heartbeat from core client for 30 sec - exiting 05:08:09 (4944): No heartbeat from core client for 30 sec - exiting 05:08:10 (4944): No heartbeat from core client for 30 sec - exiting 05:08:11 (4944): No heartbeat from core client for 30 sec - exiting 05:08:12 (4944): No heartbeat from core client for 30 sec - exiting 05:08:13 (4944): No heartbeat from core client for 30 sec - exiting 05:08:15 (4944): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Atmos Hold Restart file rename failed on atmos_restart.hold 05:12:35 (4384): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:12:36 (4384): No heartbeat from core client for 30 sec - exiting 05:12:37 (4384): No heartbeat from core client for 30 sec - exiting 05:14:54 (864): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:19:49 (4692): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:24:55 (3672): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:26:49 (2312): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:28:15 (4028): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:28:16 (4028): No heartbeat from core client for 30 sec - exiting Atmos Hold Restart file rename failed on atmos_restart.hold 05:29:38 (2868): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Atmos Hold Restart file rename failed on atmos_restart.hold 05:30:27 (3700): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:31:04 (3264): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:32:23 (4584): No heartbeat from core client for 30 sec - exiting tmos Hold Restart file rename failed on atmos_restart.hold CPDN Monitor - No 'heartbeat' from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=980, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=980, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=980, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=980, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=980, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=980, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
15 Nov 2011 18:06:56	1176150	13619858	hadcm3n_ybz4_1940_40_007539769_2	51,840	102,685	1.9808
15 Nov 2011 18:06:56	1176150	13619858	hadcm3n_ybz4_1940_40_007539769_2	25,920	52,528	2.0265