Task 12815322

Name	hadcm3n_p039_1900_40_007217701_0
Workunit	7415941
Created	26 Apr 2011, 15:13:05 UTC
Sent	6 May 2011, 2:57:29 UTC
Report deadline	5 Aug 2011, 10:24:40 UTC
Received	18 May 2011, 0:01:38 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	302859
Run time	3 days 22 hours 49 min 10 sec
CPU time	2 days 22 hours 16 min 58 sec
Validate state	Invalid
Credit	311.04
Device peak FLOPS	0.98 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.10.18</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> 09:01:15 (3384): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:01:27 (3384): No heartbeat from core client for 30 sec - exiting 09:01:28 (3384): No heartbeat from core client for 30 sec - exiting 09:01:29 (3384): No heartbeat from core client for 30 sec - exiting 09:01:30 (3384): No heartbeat from core client for 30 sec - exiting 09:01:31 (3384): No heartbeat from core client for 30 sec - exiting 09:01:32 (3384): No heartbeat from core client for 30 sec - exiting 09:01:33 (3384): No heartbeat from core client for 30 sec - exiting 09:01:34 (3384): No heartbeat from core client for 30 sec - exiting 09:01:35 (3384): No heartbeat from core client for 30 sec - exiting 09:01:36 (3384): No heartbeat from core client for 30 sec - exiting 09:01:37 (3384): No heartbeat from core client for 30 sec - exiting 09:02:42 (3384): No heartbeat from core client for 30 sec - exiting 09:02:43 (3384): No heartbeat from core client for 30 sec - exiting 09:02:44 (3384): No heartbeat from core client for 30 sec - exiting 09:02:45 (3384): No heartbeat from core client for 30 sec - exiting 09:02:46 (3384): No heartbeat from core client for 30 sec - exiting 09:02:47 (3384): No heartbeat from core client for 30 sec - exiting 09:02:48 (3384): No heartbeat from core client for 30 sec - exiting 09:02:49 (3384): No heartbeat from core client for 30 sec - exiting 09:02:50 (3384): No heartbeat from core client for 30 sec - exiting 09:02:51 (3384): No heartbeat from core client for 30 sec - exiting 09:02:52 (3384): No heartbeat from core client for 30 sec - exiting 18:41:41 (3620): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 19:17:35 (2512): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:18:32 (2512): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 20:44:45 (2308): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:46:16 (2308): No heartbeat from core client for 30 sec - exiting 20:46:17 (2308): No heartbeat from core client for 30 sec - exiting 20:46:18 (2308): No heartbeat from core client for 30 sec - exiting 20:46:19 (2308): No heartbeat from core client for 30 sec - exiting 20:46:20 (2308): No heartbeat from core client for 30 sec - exiting 20:46:21 (2308): No heartbeat from core client for 30 sec - exiting 20:46:22 (2308): No heartbeat from core client for 30 sec - exiting 20:46:23 (2308): No heartbeat from core client for 30 sec - exiting 20:46:24 (2308): No heartbeat from core client for 30 sec - exiting 20:46:25 (2308): No heartbeat from core client for 30 sec - exiting 20:46:26 (2308): No heartbeat from core client for 30 sec - exiting 20:46:27 (2308): No heartbeat from core client for 30 sec - exiting 20:46:28 (2308): No heartbeat from core client for 30 sec - exiting 20:46:29 (2308): No heartbeat from core client for 30 sec - exiting 20:46:30 (2308): No heartbeat from core client for 30 sec - exiting 20:46:31 (2308): No heartbeat from core client for 30 sec - exiting 20:46:32 (2308): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 22:12:40 (3068): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 00:24:50 (2388): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:45:38 (3508): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:45:55 (3508): No heartbeat from core client for 30 sec - exiting 00:45:56 (3508): No heartbeat from core client for 30 sec - exiting 00:48:49 (3804): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:50:31 (3876): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:50:42 (3876): No heartbeat from core client for 30 sec - exiting 00:50:43 (3876): No heartbeat from core client for 30 sec - exiting 00:50:44 (3876): No heartbeat from core client for 30 sec - exiting 00:50:45 (3876): No heartbeat from core client for 30 sec - exiting 00:50:46 (3876): No heartbeat from core client for 30 sec - exiting 00:56:04 (4004): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:56:11 (4004): No heartbeat from core client for 30 sec - exiting 00:56:12 (4004): No heartbeat from core client for 30 sec - exiting 01:05:03 (4072): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:05:16 (4072): No heartbeat from core client for 30 sec - exiting 01:05:17 (4072): No heartbeat from core client for 30 sec - exiting 01:05:18 (4072): No heartbeat from core client for 30 sec - exiting 01:05:19 (4072): No heartbeat from core client for 30 sec - exiting 01:05:20 (4072): No heartbeat from core client for 30 sec - exiting 01:05:21 (4072): No heartbeat from core client for 30 sec - exiting 01:05:22 (4072): No heartbeat from core client for 30 sec - exiting 01:23:57 (348): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:24:04 (348): No heartbeat from core client for 30 sec - exiting 01:24:05 (348): No heartbeat from core client for 30 sec - exiting 01:24:06 (348): No heartbeat from core client for 30 sec - exiting 01:39:11 (1604): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:39:26 (1604): No heartbeat from core client for 30 sec - exiting 01:41:31 (1492): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:41:41 (1492): No heartbeat from core client for 30 sec - exiting 01:41:42 (1492): No heartbeat from core client for 30 sec - exiting 01:43:59 (1148): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:44:00 (1148): No heartbeat from core client for 30 sec - exiting 01:55:43 (2484): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:12:12 (868): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:12:22 (868): No heartbeat from core client for 30 sec - exiting 02:12:23 (868): No heartbeat from core client for 30 sec - exiting 04:19:51 (2544): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:25:15 (2008): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:25:25 (2008): No heartbeat from core client for 30 sec - exiting 06:25:27 (2008): No heartbeat from core client for 30 sec - exiting 06:25:28 (2008): No heartbeat from core client for 30 sec - exiting 06:25:29 (2008): No heartbeat from core client for 30 sec - exiting 06:25:30 (2008): No heartbeat from core client for 30 sec - exiting 06:25:31 (2008): No heartbeat from core client for 30 sec - exiting 06:25:32 (2008): No heartbeat from core client for 30 sec - exiting 06:50:18 (204): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:50:28 (204): No heartbeat from core client for 30 sec - exiting 06:50:29 (204): No heartbeat from core client for 30 sec - exiting 07:58:02 (804): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:45:29 (584): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:48:45 (2384): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:51:08 (1148): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:56:07 (2444): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:56:17 (2444): No heartbeat from core client for 30 sec - exiting 08:56:18 (2444): No heartbeat from core client for 30 sec - exiting 08:56:19 (2444): No heartbeat from core client for 30 sec - exiting 08:56:20 (2444): No heartbeat from core client for 30 sec - exiting 08:56:21 (2444): No heartbeat from core client for 30 sec - exiting 08:56:22 (2444): No heartbeat from core client for 30 sec - exiting 08:56:23 (2444): No heartbeat from core client for 30 sec - exiting 08:56:25 (2444): No heartbeat from core client for 30 sec - exiting 09:01:31 (504): No heartbeat from core client for 30 sec - exiting 09:01:32 (504): No heartbeat from core client for 30 sec - exiting 09:01:33 (504): No heartbeat from core client for 30 sec - exiting 09:01:41 (504): No heartbeat from core client for 30 sec - exiting 09:01:42 (504): No heartbeat from core client for 30 sec - exiting 09:01:43 (504): No heartbeat from core client for 30 sec - exiting 09:01:44 (504): No heartbeat from core client for 30 sec - exiting 09:01:45 (504): No heartbeat from core client for 30 sec - exiting 09:01:46 (504): No heartbeat from core client for 30 sec - exiting 09:01:47 (504): No heartbeat from core client for 30 sec - exiting 09:01:49 (504): No heartbeat from core client for 30 sec - exiting 09:01:50 (504): No heartbeat from core client for 30 sec - exiting 09:01:51 (504): No heartbeat from core client for 30 sec - exiting 09:01:52 (504): No heartbeat from core client for 30 sec - exiting 09:01:53 (504): No heartbeat from core client for 30 sec - exiting 09:01:54 (504): No heartbeat from core client for 30 sec - exiting 09:01:55 (504): No heartbeat from core client for 30 sec - exiting 09:01:56 (504): No heartbeat from core client for 30 sec - exiting 09:01:57 (504): No heartbeat from core client for 30 sec - exiting 09:01:58 (504): No heartbeat from core client for 30 sec - exiting 09:02:00 (504): No heartbeat from core client for 30 sec - exiting 09:02:01 (504): No heartbeat from core client for 30 sec - exiting 09:02:02 (504): No heartbeat from core client for 30 sec - exiting 09:02:03 (504): No heartbeat from core client for 30 sec - exiting 09:02:04 (504): No heartbeat from core client for 30 sec - exiting 09:02:05 (504): No heartbeat from core client for 30 sec - exiting 09:02:06 (504): No heartbeat from core client for 30 sec - exiting 09:02:15 (504): No heartbeat from core client for 30 sec - exiting 09:02:16 (504): No heartbeat from core client for 30 sec - exiting 09:02:17 (504): No heartbeat from core client for 30 sec - exiting 09:02:18 (504): No heartbeat from core client for 30 sec - exiting 09:02:20 (504): No heartbeat from core client for 30 sec - exiting 09:02:21 (504): No heartbeat from core client for 30 sec - exiting 09:02:22 (504): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2376, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2376, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2376, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2376, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... 03:12:29 (2376): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 03:12:30 (2376): No heartbeat from core client for 30 sec - exiting Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1260, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1260, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
13 May 2011 02:39:06	302859	12815322	hadcm3n_p039_1900_40_007217701_0	25,920	148,651	5.7350