Task 15895245

Name	hadcm3n_o0o3_1940_40_008381549_4
Workunit	8532408
Created	18 Jul 2013, 12:35:20 UTC
Sent	18 Jul 2013, 17:44:04 UTC
Report deadline	18 Oct 2013, 1:11:15 UTC
Received	14 Aug 2013, 15:39:01 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1272463
Run time	24 days 2 hours 23 min 32 sec
CPU time	19 days 0 hours 42 min 51 sec
Validate state	Invalid
Credit	8,398.08
Device peak FLOPS	2.07 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.28</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> 09:31:07 (4188): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:30:05 (5440): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 19:23:26 (6060): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:39:34 (5120): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 20:38:33 (7164): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3184, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3184, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - CPD23:07:03 (5496): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:06:02 (1920): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:05:01 (1488): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:04:00 (4648): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 16:02:58 (528): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:01:54 (6412): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:34:49 (6320): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:01:00 (4440): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:59:58 (7260): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
14 Aug 2013 15:47:50	1272463	15895245	hadcm3n_o0o3_1940_40_008381549_4	699,840	1,692,709	2.4187
14 Aug 2013 15:47:50	1272463	15895245	hadcm3n_o0o3_1940_40_008381549_4	673,920	1,621,588	2.4062
14 Aug 2013 15:47:50	1272463	15895245	hadcm3n_o0o3_1940_40_008381549_4	648,000	1,550,629	2.3929
14 Aug 2013 15:47:50	1272463	15895245	hadcm3n_o0o3_1940_40_008381549_4	622,080	1,480,102	2.3793
14 Aug 2013 15:47:50	1272463	15895245	hadcm3n_o0o3_1940_40_008381549_4	596,160	1,409,527	2.3643
14 Aug 2013 15:47:50	1272463	15895245	hadcm3n_o0o3_1940_40_008381549_4	570,240	1,338,100	2.3466
14 Aug 2013 15:47:50	1272463	15895245	hadcm3n_o0o3_1940_40_008381549_4	544,320	1,267,418	2.3284
14 Aug 2013 15:47:50	1272463	15895245	hadcm3n_o0o3_1940_40_008381549_4	518,400	1,196,821	2.3087
14 Aug 2013 15:47:50	1272463	15895245	hadcm3n_o0o3_1940_40_008381549_4	492,480	1,126,206	2.2868
14 Aug 2013 15:47:50	1272463	15895245	hadcm3n_o0o3_1940_40_008381549_4	466,560	1,053,133	2.2572
14 Aug 2013 15:47:50	1272463	15895245	hadcm3n_o0o3_1940_40_008381549_4	440,640	982,193	2.2290
14 Aug 2013 15:47:50	1272463	15895245	hadcm3n_o0o3_1940_40_008381549_4	414,720	910,562	2.1956
14 Aug 2013 15:47:50	1272463	15895245	hadcm3n_o0o3_1940_40_008381549_4	388,800	840,582	2.1620
14 Aug 2013 15:47:50	1272463	15895245	hadcm3n_o0o3_1940_40_008381549_4	362,880	770,771	2.1240
30 Jul 2013 10:26:38	1272463	15895245	hadcm3n_o0o3_1940_40_008381549_4	336,960	699,711	2.0765
29 Jul 2013 12:55:15	1272463	15895245	hadcm3n_o0o3_1940_40_008381549_4	311,040	841,964	2.7069
29 Jul 2013 12:55:14	1272463	15895245	hadcm3n_o0o3_1940_40_008381549_4	285,120	769,797	2.6999
29 Jul 2013 12:55:14	1272463	15895245	hadcm3n_o0o3_1940_40_008381549_4	259,200	698,591	2.6952
26 Jul 2013 12:39:59	1272463	15895245	hadcm3n_o0o3_1940_40_008381549_4	233,280	628,862	2.6957
25 Jul 2013 16:23:41	1272463	15895245	hadcm3n_o0o3_1940_40_008381549_4	207,360	559,217	2.6968