Task 15675583

Name	hadcm3n_3l1m_1940_40_008260017_1
Workunit	8415141
Created	22 Mar 2013, 0:13:06 UTC
Sent	22 Mar 2013, 0:13:20 UTC
Report deadline	21 Jun 2013, 7:40:31 UTC
Received	3 Apr 2013, 21:09:51 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1167468
Run time	11 days 22 hours 30 min 21 sec
CPU time	11 days 19 hours 46 min 12 sec
Validate state	Invalid
Credit	9,020.16
Device peak FLOPS	3.15 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.40</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 12:31:28 (3776): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3740, iMonCtr=1 Model crash detected, will try to restart... 14:01:07 (3740): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5852, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5852, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5852, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5852, iMonCtr=1 Model crash detected, will try to restart... 14:01:44 (5852): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:03:17 (1456): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4824, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish 14:07:06 (4824): No heartbeat from core client for 30 sec - exiting 14:07:07 (4824): No heartbeat from core client for 30 sec - exiting </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
03 Apr 2013 16:41:23	1167468	15675583	hadcm3n_3l1m_1940_40_008260017_1	751,680	1,007,642	1.3405
03 Apr 2013 07:14:51	1167468	15675583	hadcm3n_3l1m_1940_40_008260017_1	725,760	974,537	1.3428
02 Apr 2013 21:00:15	1167468	15675583	hadcm3n_3l1m_1940_40_008260017_1	699,840	940,337	1.3436
02 Apr 2013 10:59:29	1167468	15675583	hadcm3n_3l1m_1940_40_008260017_1	673,920	906,023	1.3444
02 Apr 2013 01:05:45	1167468	15675583	hadcm3n_3l1m_1940_40_008260017_1	648,000	871,373	1.3447
01 Apr 2013 15:30:49	1167468	15675583	hadcm3n_3l1m_1940_40_008260017_1	622,080	837,106	1.3457
01 Apr 2013 05:28:21	1167468	15675583	hadcm3n_3l1m_1940_40_008260017_1	596,160	802,938	1.3468
31 Mar 2013 19:53:17	1167468	15675583	hadcm3n_3l1m_1940_40_008260017_1	570,240	769,199	1.3489
31 Mar 2013 09:29:19	1167468	15675583	hadcm3n_3l1m_1940_40_008260017_1	544,320	733,552	1.3476
30 Mar 2013 21:50:48	1167468	15675583	hadcm3n_3l1m_1940_40_008260017_1	518,400	698,910	1.3482
30 Mar 2013 04:12:58	1167468	15675583	hadcm3n_3l1m_1940_40_008260017_1	492,480	662,883	1.3460
29 Mar 2013 18:31:00	1167468	15675583	hadcm3n_3l1m_1940_40_008260017_1	466,560	627,991	1.3460
29 Mar 2013 08:40:52	1167468	15675583	hadcm3n_3l1m_1940_40_008260017_1	440,640	593,010	1.3458
28 Mar 2013 22:20:26	1167468	15675583	hadcm3n_3l1m_1940_40_008260017_1	414,720	556,359	1.3415
28 Mar 2013 09:43:58	1167468	15675583	hadcm3n_3l1m_1940_40_008260017_1	388,800	520,444	1.3386
27 Mar 2013 23:15:11	1167468	15675583	hadcm3n_3l1m_1940_40_008260017_1	362,880	483,962	1.3337
27 Mar 2013 13:19:08	1167468	15675583	hadcm3n_3l1m_1940_40_008260017_1	336,960	448,880	1.3321
27 Mar 2013 03:25:25	1167468	15675583	hadcm3n_3l1m_1940_40_008260017_1	311,040	414,128	1.3314
26 Mar 2013 17:47:15	1167468	15675583	hadcm3n_3l1m_1940_40_008260017_1	285,120	380,212	1.3335
26 Mar 2013 08:15:57	1167468	15675583	hadcm3n_3l1m_1940_40_008260017_1	259,200	345,223	1.3319