Task 15636194

Name	hadcm3n_3iia_1980_40_008320195_1
Workunit	8471330
Created	24 Feb 2013, 17:47:46 UTC
Sent	24 Feb 2013, 17:49:46 UTC
Report deadline	27 May 2013, 1:16:57 UTC
Received	28 Mar 2013, 9:58:48 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1267596
Run time	2 days 10 hours 41 min 28 sec
CPU time	1 days 19 hours 32 min 57 sec
Validate state	Invalid
Credit	622.08
Device peak FLOPS	2.74 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.8.44</core_client_version> <![CDATA[ <message> The device does not recognise the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 19:24:53 (5204): No heartbeat from core client for 30 sec - exiting 19:24:54 (5204): No heartbeat from core client for 30 sec - exiting 19:24:55 (5204): No heartbeat from core client for 30 sec - exiting 19:24:56 (5204): No heartbeat from core client for 30 sec - exiting 19:24:57 (5204): No heartbeat from core client for 30 sec - exiting 19:24:58 (5204): No heartbeat from core client for 30 sec - exiting 19:24:59 (5204): No heartbeat from core client for 30 sec - exiting 19:25:00 (5204): No heartbeat from core client for 30 sec - exiting 19:25:01 (5204): No heartbeat from core client for 30 sec - exiting 19:25:02 (5204): No heartbeat from core client for 30 sec - exiting 19:25:03 (5204): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 18:58:29 (7248): No heartbeat from core client for 30 sec - exiting 18:58:30 (7248): No heartbeat from core client for 30 sec - exiting 18:58:31 (7248): No heartbeat from core client for 30 sec - exiting 18:58:32 (7248): No heartbeat from core client for 30 sec - exiting 18:58:33 (7248): No heartbeat from core client for 30 sec - exiting 18:58:34 (7248): No heartbeat from core client for 30 sec - exiting 18:58:35 (7248): No heartbeat from core client for 30 sec - exiting 18:58:36 (7248): No heartbeat from core client for 30 sec - exiting 18:58:37 (7248): No heartbeat from core client for 30 sec - exiting 18:58:38 (7248): No heartbeat from core client for 30 sec - exiting 18:58:39 (7248): No heartbeat from core client for 30 sec - exiting 18:58:40 (7248): No heartbeat from core client for 30 sec - exiting 18:58:41 (7248): No heartbeat from core client for 30 sec - exiting 18:58:42 (7248): No heartbeat from core client for 30 sec - exiting 18:58:43 (7248): No heartbeat from core client for 30 sec - exiting 18:58:44 (7248): No heartbeat from core client for 30 sec - exiting 18:58:45 (7248): No heartbeat from core client for 30 sec - exiting 18:58:46 (7248): No heartbeat from core client for 30 sec - exiting 18:58:47 (7248): No heartbeat from core client for 30 sec - exiting 18:58:48 (7248): No heartbeat from core client for 30 sec - exiting 18:58:49 (7248): No heartbeat from core client for 30 sec - exiting 18:58:50 (7248): No heartbeat from core client for 30 sec - exiting 18:58:51 (7248): No heartbeat from core client for 30 sec - exiting 18:58:52 (7248): No heartbeat from core client for 30 sec - exiting 18:58:53 (7248): No heartbeat from core client for 30 sec - exiting 18:58:54 (7248): No heartbeat from core client for 30 sec - exiting 18:58:55 (7248): No heartbeat from core client for 30 sec - exiting 18:58:56 (7248): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 14:40:31 (9912): No heartbeat from core client for 30 sec - exiting 14:40:32 (9912): No heartbeat from core client for 30 sec - exiting 14:40:33 (9912): No heartbeat from core client for 30 sec - exiting 14:40:34 (9912): No heartbeat from core client for 30 sec - exiting 14:40:35 (9912): No heartbeat from core client for 30 sec - exiting 14:40:36 (9912): No heartbeat from core client for 30 sec - exiting 14:40:37 (9912): No heartbeat from core client for 30 sec - exiting 14:40:38 (9912): No heartbeat from core client for 30 sec - exiting 14:40:39 (9912): No heartbeat from core client for 30 sec - exiting 14:40:40 (9912): No heartbeat from core client for 30 sec - exiting 14:40:41 (9912): No heartbeat from core client for 30 sec - exiting 14:40:42 (9912): No heartbeat from core client for 30 sec - exiting 14:40:43 (9912): No heartbeat from core client for 30 sec - exiting 14:40:44 (9912): No heartbeat from core client for 30 sec - exiting 14:40:45 (9912): No heartbeat from core client for 30 sec - exiting 14:40:46 (9912): No heartbeat from core client for 30 sec - exiting 14:40:47 (9912): No heartbeat from core client for 30 sec - exiting 14:40:48 (9912): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 15:59:23 (5184): No heartbeat from core client for 30 sec - exiting 15:59:24 (5184): No heartbeat from core client for 30 sec - exiting 15:59:25 (5184): No heartbeat from core client for 30 sec - exiting 15:59:26 (5184): No heartbeat from core client for 30 sec - exiting 15:59:27 (5184): No heartbeat from core client for 30 sec - exiting 15:59:28 (5184): No heartbeat from core client for 30 sec - exiting 15:59:29 (5184): No heartbeat from core client for 30 sec - exiting 15:59:30 (5184): No heartbeat from core client for 30 sec - exiting 15:59:31 (5184): No heartbeat from core client for 30 sec - exiting 15:59:32 (5184): No heartbeat from core client for 30 sec - exiting 15:59:33 (5184): No heartbeat from core client for 30 sec - exiting 15:59:35 (5184): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:59:36 (5184): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 08:19:14 (6460): No heartbeat from core client for 30 sec - exiting 08:19:15 (6460): No heartbeat from core client for 30 sec - exiting 08:19:16 (6460): No heartbeat from core client for 30 sec - exiting 08:19:17 (6460): No heartbeat from core client for 30 sec - exiting 08:19:18 (6460): No heartbeat from core client for 30 sec - exiting 08:19:19 (6460): No heartbeat from core client for 30 sec - exiting 08:19:20 (6460): No heartbeat from core client for 30 sec - exiting 08:19:21 (6460): No heartbeat from core client for 30 sec - exiting 08:19:22 (6460): No heartbeat from core client for 30 sec - exiting 08:19:23 (6460): No heartbeat from core client for 30 sec - exiting 08:19:24 (6460): No heartbeat from core client for 30 sec - exiting 08:19:25 (6460): No heartbeat from core client for 30 sec - exiting 08:19:26 (6460): No heartbeat from core client for 30 sec - exiting 08:19:27 (6460): No heartbeat from core client for 30 sec - exiting 08:19:28 (6460): No heartbeat from core client for 30 sec - exiting 08:19:29 (6460): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2600, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2600, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2600, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2600, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2600, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2600, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
21 Mar 2013 22:10:10	1267596	15636194	hadcm3n_3iia_1980_40_008320195_1	51,840	108,971	2.1021
20 Mar 2013 04:51:06	1267596	15636194	hadcm3n_3iia_1980_40_008320195_1	25,920	55,748	2.1508