Task 14694035

Name	hadcm3n_104h_1940_40_007957287_4
Workunit	8112399
Created	16 May 2012, 23:00:45 UTC
Sent	16 May 2012, 23:01:00 UTC
Report deadline	16 Aug 2012, 6:28:11 UTC
Received	22 Jun 2012, 17:52:54 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1166636
Run time	9 days 12 hours 12 min 30 sec
CPU time	8 days 10 hours 14 min 39 sec
Validate state	Invalid
Credit	8,087.04
Device peak FLOPS	4.37 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.25</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 07:35:04 (4832): No heartbeat from core client for 30 sec - exiting 07:35:05 (4832): No heartbeat from core client for 30 sec - exiting 07:35:06 (4832): No heartbeat from core client for 30 sec - exiting 07:35:07 (4832): No heartbeat from core client for 30 sec - exiting 07:35:08 (4832): No heartbeat from core client for 30 sec - exiting 07:35:09 (4832): No heartbeat from core client for 30 sec - exiting 07:35:10 (4832): No heartbeat from core client for 30 sec - exiting 07:35:11 (4832): No heartbeat from core client for 30 sec - exiting 07:35:12 (4832): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 00:01:06 (3472): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:01:08 (3472): No heartbeat from core client for 30 sec - exiting 00:01:09 (3472): No heartbeat from core client for 30 sec - exiting 00:01:10 (3472): No heartbeat from core client for 30 sec - exiting 00:01:11 (3472): No heartbeat from core client for 30 sec - exiting 00:01:12 (3472): No heartbeat from core client for 30 sec - exiting 00:01:13 (3472): No heartbeat from core client for 30 sec - exiting 00:01:14 (3472): No heartbeat from core client for 30 sec - exiting 00:01:15 (3472): No heartbeat from core client for 30 sec - exiting 00:01:16 (3472): No heartbeat from core client for 30 sec - exiting 00:01:17 (3472): No heartbeat from core client for 30 sec - exiting 00:02:46 (1172): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:04:25 (3564): No heartbeat from core client for 30 sec - exiting Ctmos Hold Restart file rename failed on atmos_restart.hold PDN Monitor - No 'heartbeat' from BOINC... 00:06:28 (6136): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:07:54 (6092): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5236, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5236, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2772, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2772, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2772, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2772, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
20 Jun 2012 23:16:28	1166636	14694035	hadcm3n_104h_1940_40_007957287_4	673,920	707,800	1.0503
20 Jun 2012 23:16:28	1166636	14694035	hadcm3n_104h_1940_40_007957287_4	648,000	683,957	1.0555
20 Jun 2012 04:18:56	1166636	14694035	hadcm3n_104h_1940_40_007957287_4	622,080	660,065	1.0611
19 Jun 2012 23:03:48	1166636	14694035	hadcm3n_104h_1940_40_007957287_4	596,160	636,217	1.0672
19 Jun 2012 23:03:48	1166636	14694035	hadcm3n_104h_1940_40_007957287_4	570,240	612,372	1.0739
16 Jun 2012 23:01:31	1166636	14694035	hadcm3n_104h_1940_40_007957287_4	544,320	586,262	1.0771
16 Jun 2012 23:01:31	1166636	14694035	hadcm3n_104h_1940_40_007957287_4	518,400	558,682	1.0777
16 Jun 2012 23:01:31	1166636	14694035	hadcm3n_104h_1940_40_007957287_4	492,480	530,794	1.0778
15 Jun 2012 02:10:13	1166636	14694035	hadcm3n_104h_1940_40_007957287_4	466,560	502,960	1.0780
14 Jun 2012 23:05:33	1166636	14694035	hadcm3n_104h_1940_40_007957287_4	440,640	475,214	1.0785
14 Jun 2012 23:05:33	1166636	14694035	hadcm3n_104h_1940_40_007957287_4	414,720	447,430	1.0789
14 Jun 2012 02:41:03	1166636	14694035	hadcm3n_104h_1940_40_007957287_4	388,800	421,455	1.0840
13 Jun 2012 23:06:40	1166636	14694035	hadcm3n_104h_1940_40_007957287_4	362,880	397,602	1.0957
13 Jun 2012 23:06:40	1166636	14694035	hadcm3n_104h_1940_40_007957287_4	336,960	373,825	1.1094
08 Jun 2012 23:05:22	1166636	14694035	hadcm3n_104h_1940_40_007957287_4	311,040	348,753	1.1212
08 Jun 2012 23:05:22	1166636	14694035	hadcm3n_104h_1940_40_007957287_4	285,120	320,095	1.1227
08 Jun 2012 06:26:29	1166636	14694035	hadcm3n_104h_1940_40_007957287_4	259,200	291,222	1.1235
07 Jun 2012 02:32:22	1166636	14694035	hadcm3n_104h_1940_40_007957287_4	233,280	262,386	1.1248
01 Jun 2012 23:05:19	1166636	14694035	hadcm3n_104h_1940_40_007957287_4	207,360	232,916	1.1232
31 May 2012 23:08:38	1166636	14694035	hadcm3n_104h_1940_40_007957287_4	181,440	202,828	1.1179