Task 15666556

Name	hadcm3n_3l06_1980_40_008325793_2
Workunit	8476928
Created	13 Mar 2013, 20:00:23 UTC
Sent	13 Mar 2013, 20:01:11 UTC
Report deadline	13 Jun 2013, 3:28:22 UTC
Received	1 Apr 2013, 4:25:26 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1261981
Run time	17 days 20 hours 34 min 40 sec
CPU time	18 hours 27 min 44 sec
Validate state	Invalid
Credit	8,087.04
Device peak FLOPS	1.70 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-pc-linux-gnu
Stderr	<core_client_version>6.10.58</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Suspended CPDN Monitor - Suspend request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Suspended CPDN Monitor - Suspend request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 20:44:14 (26971): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:29:24 (4080): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:35:42 (4126): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:35:43 (4126): No heartbeat from core client for 30 sec - exiting 21:35:44 (4126): No heartbeat from core client for 30 sec - exiting 07:51:59 (4161): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:01:25 (4468): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
01 Apr 2013 00:26:21	1261981	15666556	hadcm3n_3l06_1980_40_008325793_2	673,920	371,334	0.5510
31 Mar 2013 07:43:50	1261981	15666556	hadcm3n_3l06_1980_40_008325793_2	648,000	312,388	0.4821
30 Mar 2013 14:55:30	1261981	15666556	hadcm3n_3l06_1980_40_008325793_2	622,080	253,139	0.4069
29 Mar 2013 22:05:27	1261981	15666556	hadcm3n_3l06_1980_40_008325793_2	596,160	193,780	0.3250
29 Mar 2013 05:19:23	1261981	15666556	hadcm3n_3l06_1980_40_008325793_2	570,240	134,505	0.2359
28 Mar 2013 18:43:49	1261981	15666556	hadcm3n_3l06_1980_40_008325793_2	544,320	74,928	0.1377
28 Mar 2013 18:43:50	1261981	15666556	hadcm3n_3l06_1980_40_008325793_2	518,400	15,678	0.0302
27 Mar 2013 02:40:05	1261981	15666556	hadcm3n_3l06_1980_40_008325793_2	492,480	587,324	1.1926
26 Mar 2013 10:01:37	1261981	15666556	hadcm3n_3l06_1980_40_008325793_2	466,560	528,325	1.1324
25 Mar 2013 18:03:07	1261981	15666556	hadcm3n_3l06_1980_40_008325793_2	440,640	469,737	1.0660
25 Mar 2013 01:05:19	1261981	15666556	hadcm3n_3l06_1980_40_008325793_2	414,720	410,328	0.9894
24 Mar 2013 08:52:08	1261981	15666556	hadcm3n_3l06_1980_40_008325793_2	388,800	350,763	0.9022
23 Mar 2013 15:00:39	1261981	15666556	hadcm3n_3l06_1980_40_008325793_2	362,880	291,015	0.8020
22 Mar 2013 22:07:52	1261981	15666556	hadcm3n_3l06_1980_40_008325793_2	336,960	231,264	0.6863
22 Mar 2013 05:22:54	1261981	15666556	hadcm3n_3l06_1980_40_008325793_2	311,040	171,599	0.5517
21 Mar 2013 12:35:11	1261981	15666556	hadcm3n_3l06_1980_40_008325793_2	285,120	112,268	0.3938
20 Mar 2013 20:18:27	1261981	15666556	hadcm3n_3l06_1980_40_008325793_2	259,200	52,541	0.2027
20 Mar 2013 03:00:42	1261981	15666556	hadcm3n_3l06_1980_40_008325793_2	233,280	21,798	0.0934
19 Mar 2013 10:29:29	1261981	15666556	hadcm3n_3l06_1980_40_008325793_2	207,360	387,323	1.8679
18 Mar 2013 18:25:54	1261981	15666556	hadcm3n_3l06_1980_40_008325793_2	181,440	328,819	1.8123