Task 12921236

Name	hadcm3n_o1sc_1940_40_007265035_1
Workunit	7463275
Created	2 Jun 2011, 4:47:07 UTC
Sent	2 Jun 2011, 4:47:14 UTC
Report deadline	1 Sep 2011, 12:14:25 UTC
Received	19 Jun 2011, 21:59:15 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1114071
Run time	4 days 12 hours 35 min 28 sec
CPU time	2 days 17 hours 29 min 40 sec
Validate state	Invalid
Credit	311.04
Device peak FLOPS	1.20 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.10.58</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> 17:49:39 (3852): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:51:19 (5992): No heartbeat from core client for 30 sec - exiting 17:51:20 (5992): No heartbeat from core client for 30 sec - exiting 17:51:21 (5992): No heartbeat from core client for 30 sec - exiting 17:51:22 (5992): No heartbeat from core client for 30 sec - exiting 17:51:24 (5992): No heartbeat from core client for 30 sec - exiting 17:51:25 (5992): No heartbeat from core client for 30 sec - exiting 17:51:26 (5992): No heartbeat from core client for 30 sec - exiting 17:51:27 (5992): No heartbeat from core client for 30 sec - exiting 17:51:28 (5992): No heartbeat from core client for 30 sec - exiting 17:51:29 (5992): No heartbeat from core client for 30 sec - exiting 17:51:30 (5992): No heartbeat from core client for 30 sec - exiting 17:51:31 (5992): No heartbeat from core client for 30 sec - exiting 17:51:32 (5992): No heartbeat from core client for 30 sec - exiting 17:51:33 (5992): No heartbeat from core client for 30 sec - exiting 17:51:34 (5992): No heartbeat from core client for 30 sec - exiting 17:51:35 (5992): No heartbeat from core client for 30 sec - exiting 17:51:36 (5992): No heartbeat from core client for 30 sec - exiting 17:51:37 (5992): No heartbeat from core client for 30 sec - exiting 17:51:38 (5992): No heartbeat from core client for 30 sec - exiting 17:51:39 (5992): No heartbeat from core client for 30 sec - exiting 17:52:10 (5992): No heartbeat from core client for 30 sec - exiting 17:52:11 (5992): No heartbeat from core client for 30 sec - exiting 17:52:12 (5992): No heartbeat from core client for 30 sec - exiting 17:52:13 (5992): No heartbeat from core client for 30 sec - exiting 17:52:14 (5992): No heartbeat from core client for 30 sec - exiting 17:52:15 (5992): No heartbeat from core client for 30 sec - exiting 17:52:16 (5992): No heartbeat from core client for 30 sec - exiting 17:52:17 (5992): No heartbeat from core client for 30 sec - exiting 17:52:18 (5992): No heartbeat from core client for 30 sec - exiting 17:52:19 (5992): No heartbeat from core client for 30 sec - exiting 17:52:20 (5992): No heartbeat from core client for 30 sec - exiting 17:52:21 (5992): No heartbeat from core client for 30 sec - exiting 17:52:22 (5992): No heartbeat from core client for 30 sec - exiting 17:52:23 (5992): No heartbeat from core client for 30 sec - exiting 17:52:24 (5992): No heartbeat from core client for 30 sec - exiting 17:52:25 (5992): No heartbeat from core client for 30 sec - exiting 17:52:26 (5992): No heartbeat from core client for 30 sec - exiting 17:52:27 (5992): No heartbeat from core client for 30 sec - exiting 17:52:28 (5992): No heartbeat from core client for 30 sec - exiting 17:52:29 (5992): No heartbeat from core client for 30 sec - exiting 17:52:30 (5992): No heartbeat from core client for 30 sec - exiting 17:52:31 (5992): No heartbeat from core client for 30 sec - exiting 17:52:32 (5992): No heartbeat from core client for 30 sec - exiting 17:52:33 (5992): No heartbeat from core client for 30 sec - exiting 17:52:34 (5992): No heartbeat from core client for 30 sec - exiting 17:52:35 (5992): No heartbeat from core client for 30 sec - exiting 17:52:36 (5992): No heartbeat from core client for 30 sec - exiting 17:52:37 (5992): No heartbeat from core client for 30 sec - exiting 17:53:46 (5992): No heartbeat from core client for 30 sec - exiting 17:53:48 (5992): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:56:30 (4832): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:56:31 (4832): No heartbeat from core client for 30 sec - exiting 17:56:32 (4832): No heartbeat from core client for 30 sec - exiting 17:56:33 (4832): No heartbeat from core client for 30 sec - exiting 17:56:34 (4832): No heartbeat from core client for 30 sec - exiting 17:56:35 (4832): No heartbeat from core client for 30 sec - exiting 17:56:36 (4832): No heartbeat from core client for 30 sec - exiting 17:56:37 (4832): No heartbeat from core client for 30 sec - exiting 17:56:38 (4832): No heartbeat from core client for 30 sec - exiting 17:56:39 (4832): No heartbeat from core client for 30 sec - exiting 17:56:40 (4832): No heartbeat from core client for 30 sec - exiting 17:58:44 (5296): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:05:12 (2656): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:05:13 (2656): No heartbeat from core client for 30 sec - exiting 18:05:14 (2656): No heartbeat from core client for 30 sec - exiting 18:05:15 (2656): No heartbeat from core client for 30 sec - exiting 18:05:16 (2656): No heartbeat from core client for 30 sec - exiting 18:05:17 (2656): No heartbeat from core client for 30 sec - exiting 18:05:18 (2656): No heartbeat from core client for 30 sec - exiting 18:05:19 (2656): No heartbeat from core client for 30 sec - exiting 18:05:20 (2656): No heartbeat from core client for 30 sec - exiting 18:05:21 (2656): No heartbeat from core client for 30 sec - exiting 18:05:22 (2656): No heartbeat from core client for 30 sec - exiting 18:07:40 (2220): No heartbeat from core client for 30 sec - exiting 18:07:41 (2220): No heartbeat from core client for 30 sec - exiting 18:07:42 (2220): No heartbeat from core client for 30 sec - exiting 18:07:43 (2220): No heartbeat from core client for 30 sec - exiting 18:07:44 (2220): No heartbeat from core client for 30 sec - exiting 18:07:45 (2220): No heartbeat from core client for 30 sec - exiting 18:07:46 (2220): No heartbeat from core client for 30 sec - exiting 18:07:47 (2220): No heartbeat from core client for 30 sec - exiting 18:07:48 (2220): No heartbeat from core client for 30 sec - exiting 18:07:49 (2220): No heartbeat from core client for 30 sec - exiting 18:07:50 (2220): No heartbeat from core client for 30 sec - exiting 18:07:51 (2220): No heartbeat from core client for 30 sec - exiting 18:07:52 (2220): No heartbeat from core client for 30 sec - exiting 18:07:53 (2220): No heartbeat from core client for 30 sec - exiting 18:07:54 (2220): No heartbeat from core client for 30 sec - exiting 18:07:55 (2220): No heartbeat from core client for 30 sec - exiting 18:07:56 (2220): No heartbeat from core client for 30 sec - exiting 18:07:57 (2220): No heartbeat from core client for 30 sec - exiting 18:07:58 (2220): No heartbeat from core client for 30 sec - exiting 18:07:59 (2220): No heartbeat from core client for 30 sec - exiting 18:08:00 (2220): No heartbeat from core client for 30 sec - exiting 18:08:01 (2220): No heartbeat from core client for 30 sec - exiting 18:08:02 (2220): No heartbeat from core client for 30 sec - exiting 18:08:03 (2220): No heartbeat from core client for 30 sec - exiting 18:08:04 (2220): No heartbeat from core client for 30 sec - exiting 18:08:05 (2220): No heartbeat from core client for 30 sec - exiting 18:08:06 (2220): No heartbeat from core client for 30 sec - exiting 18:08:07 (2220): No heartbeat from core client for 30 sec - exiting 18:08:08 (2220): No heartbeat from core client for 30 sec - exiting 18:08:09 (2220): No heartbeat from core client for 30 sec - exiting 18:08:10 (2220): No heartbeat from core client for 30 sec - exiting 18:08:11 (2220): No heartbeat from core client for 30 sec - exiting 18:08:12 (2220): No heartbeat from core client for 30 sec - exiting 18:08:13 (2220): No heartbeat from core client for 30 sec - exiting 18:08:14 (2220): No heartbeat from core client for 30 sec - exiting 18:08:15 (2220): No heartbeat from core client for 30 sec - exiting 18:08:16 (2220): No heartbeat from core client for 30 sec - exiting 18:08:17 (2220): No heartbeat from core client for 30 sec - exiting 18:08:18 (2220): No heartbeat from core client for 30 sec - exiting 18:08:20 (2220): No heartbeat from core client for 30 sec - exiting 18:08:21 (2220): No heartbeat from core client for 30 sec - exiting 18:08:22 (2220): No heartbeat from core client for 30 sec - exiting 18:08:23 (2220): No heartbeat from core client for 30 sec - exiting 18:08:24 (2220): No heartbeat from core client for 30 sec - exiting 18:08:25 (2220): No heartbeat from core client for 30 sec - exiting 18:08:26 (2220): No heartbeat from core client for 30 sec - exiting 18:08:27 (2220): No heartbeat from core client for 30 sec - exiting 18:08:28 (2220): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:08:29 (2220): No heartbeat from core client for 30 sec - exiting 18:08:30 (2220): No heartbeat from core client for 30 sec - exiting 18:08:31 (2220): No heartbeat from core client for 30 sec - exiting 18:08:32 (2220): No heartbeat from core client for 30 sec - exiting 18:08:33 (2220): No heartbeat from core client for 30 sec - exiting 18:08:35 (2220): No heartbeat from core client for 30 sec - exiting 18:08:36 (2220): No heartbeat from core client for 30 sec - exiting 18:08:38 (2220): No heartbeat from core client for 30 sec - exiting 18:11:44 (5292): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:11:45 (5292): No heartbeat from core client for 30 sec - exiting 18:11:46 (5292): No heartbeat from core client for 30 sec - exiting 18:11:47 (5292): No heartbeat from core client for 30 sec - exiting 18:11:48 (5292): No heartbeat from core client for 30 sec - exiting 18:11:49 (5292): No heartbeat from core client for 30 sec - exiting 18:11:50 (5292): No heartbeat from core client for 30 sec - exiting 18:11:51 (5292): No heartbeat from core client for 30 sec - exiting 18:11:52 (5292): No heartbeat from core client for 30 sec - exiting 18:11:53 (5292): No heartbeat from core client for 30 sec - exiting 18:11:54 (5292): No heartbeat from core client for 30 sec - exiting 18:18:01 (4652): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:20:18 (2036): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:29:47 (4336): No heartbeat from core client for 30 sec - exiting 18:29:49 (4336): No heartbeat from core client for 30 sec - exiting 18:29:50 (4336): No heartbeat from core client for 30 sec - exiting 18:29:51 (4336): No heartbeat from core client for 30 sec - exiting 18:29:52 (4336): No heartbeat from core client for 30 sec - exiting 18:29:53 (4336): No heartbeat from core client for 30 sec - exiting 18:29:54 (4336): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:56:41 (5552): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:00:02 (5436): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:05:47 (3532): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:07:15 (4252): No heartbeat from core client for 30 sec - exiting 20:07:16 (4252): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:01:44 (4260): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:00:43 (5536): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:17:54 (6020): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:36:30 (420): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:36:31 (420): No heartbeat from core client for 30 sec - exiting 02:36:32 (420): No heartbeat from core client for 30 sec - exiting 02:36:33 (420): No heartbeat from core client for 30 sec - exiting 02:36:34 (420): No heartbeat from core client for 30 sec - exiting 02:36:35 (420): No heartbeat from core client for 30 sec - exiting 02:36:36 (420): No heartbeat from core client for 30 sec - exiting 02:36:37 (420): No heartbeat from core client for 30 sec - exiting 02:36:39 (420): No heartbeat from core client for 30 sec - exiting 02:36:40 (420): No heartbeat from core client for 30 sec - exiting 02:36:41 (420): No heartbeat from core client for 30 sec - exiting 02:43:49 (5880): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:51:08 (4696): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 03:04:06 (2024): No heartbeat from core client for 30 sec - exiting 03:04:07 (2024): No heartbeat from core client for 30 sec - exiting 03:04:08 (2024): No heartbeat from core client for 30 sec - exiting 03:04:09 (2024): No heartbeat from core client for 30 sec - exiting 03:04:10 (2024): No heartbeat from core client for 30 sec - exiting 03:04:12 (2024): No heartbeat from core client for 30 sec - exiting 03:04:13 (2024): No heartbeat from core client for 30 sec - exiting 03:04:14 (2024): No heartbeat from core client for 30 sec - exiting 03:04:15 (2024): No heartbeat from core client for 30 sec - exiting 03:04:16 (2024): No heartbeat from core client for 30 sec - exiting 03:04:18 (2024): No heartbeat from core client for 30 sec - exiting 03:04:19 (2024): No heartbeat from core client for 30 sec - exiting 03:04:20 (2024): No heartbeat from core client for 30 sec - exiting 03:04:21 (2024): No heartbeat from core client for 30 sec - exiting 03:04:23 (2024): No heartbeat from core client for 30 sec - exiting 03:04:24 (2024): No heartbeat from core client for 30 sec - exiting 03:04:25 (2024): No heartbeat from core client for 30 sec - exiting 03:04:26 (2024): No heartbeat from core client for 30 sec - exiting 03:04:27 (2024): No heartbeat from core client for 30 sec - exiting 03:04:29 (2024): No heartbeat from core client for 30 sec - exiting 03:04:30 (2024): No heartbeat from core client for 30 sec - exiting 03:04:31 (2024): No heartbeat from core client for 30 sec - exiting 03:04:32 (2024): No heartbeat from core client for 30 sec - exiting 03:04:33 (2024): No heartbeat from core client for 30 sec - exiting 03:04:34 (2024): No heartbeat from core client for 30 sec - exiting 03:04:35 (2024): No heartbeat from core client for 30 sec - exiting 03:04:36 (2024): No heartbeat from core client for 30 sec - exiting 03:04:38 (2024): No heartbeat from core client for 30 sec - exiting 03:04:39 (2024): No heartbeat from core client for 30 sec - exiting 03:04:40 (2024): No heartbeat from core client for 30 sec - exiting 03:04:41 (2024): No heartbeat from core client for 30 sec - exiting 03:04:42 (2024): No heartbeat from core client for 30 sec - exiting 03:04:43 (2024): No heartbeat from core client for 30 sec - exiting 03:04:45 (2024): No heartbeat from core client for 30 sec - exiting 03:04:46 (2024): No heartbeat from core client for 30 sec - exiting 03:04:47 (2024): No heartbeat from core client for 30 sec - exiting 03:04:48 (2024): No heartbeat from core client for 30 sec - exiting 03:04:49 (2024): No heartbeat from core client for 30 sec - exiting 03:04:50 (2024): No heartbeat from core client for 30 sec - exiting 03:04:51 (2024): No heartbeat from core client for 30 sec - exiting 03:04:52 (2024): No heartbeat from core client for 30 sec - exiting 03:04:54 (2024): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:13:00 (3772): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:24:13 (3620): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:55:41 (3916): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:09:06 (5652): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:09:07 (5652): No heartbeat from core client for 30 sec - exiting 06:09:08 (5652): No heartbeat from core client for 30 sec - exiting 06:09:09 (5652): No heartbeat from core client for 30 sec - exiting 06:09:10 (5652): No heartbeat from core client for 30 sec - exiting 06:09:11 (5652): No heartbeat from core client for 30 sec - exiting 06:09:12 (5652): No heartbeat from core client for 30 sec - exiting 06:09:13 (5652): No heartbeat from core client for 30 sec - exiting 06:09:14 (5652): No heartbeat from core client for 30 sec - exiting 06:09:15 (5652): No heartbeat from core client for 30 sec - exiting 06:09:16 (5652): No heartbeat from core client for 30 sec - exiting 06:50:01 (2600): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:18:06 (4440): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:41:47 (5000): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4836, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4836, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4836, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4836, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4836, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4836, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
15 Jun 2011 20:10:10	1114071	12921236	hadcm3n_o1sc_1940_40_007265035_1	25,920	141,309	5.4517