Task 15604209

Name	hadcm3n_4i7k_1940_40_008303257_2
Workunit	8454392
Created	10 Feb 2013, 22:28:05 UTC
Sent	10 Feb 2013, 22:30:43 UTC
Report deadline	13 May 2013, 5:57:54 UTC
Received	19 Feb 2013, 19:36:34 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1245310
Run time	7 days 1 hours 54 min 51 sec
CPU time	4 days 15 hours 58 min 31 sec
Validate state	Invalid
Credit	9,953.28
Device peak FLOPS	2.17 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-pc-linux-gnu
Stderr	<core_client_version>7.0.28</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Signal 15 received, exiting... Called boinc_finish Signal 15 received, exiting... Called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... Called boinc_finish Signal 15 received, exiting... Called boinc_finish Signal 15 received, exiting... Signal 15 received, exiting... Called boinc_finish Signal 15 received, exiting... Called boinc_finish Signal 15 received, exiting... Called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... Called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... Called boinc_finish Signal 15 received, exiting... Called boinc_finish Signal 15 received, exiting... Called boinc_finish Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... Called boinc_finish Signal 15 received, exiting... Called boinc_finish Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
19 Feb 2013 17:36:36	1245310	15604209	hadcm3n_4i7k_1940_40_008303257_2	829,440	488,050	0.5884
19 Feb 2013 12:19:01	1245310	15604209	hadcm3n_4i7k_1940_40_008303257_2	803,520	469,501	0.5843
19 Feb 2013 07:02:54	1245310	15604209	hadcm3n_4i7k_1940_40_008303257_2	777,600	451,023	0.5800
19 Feb 2013 01:46:43	1245310	15604209	hadcm3n_4i7k_1940_40_008303257_2	751,680	432,509	0.5754
18 Feb 2013 20:33:55	1245310	15604209	hadcm3n_4i7k_1940_40_008303257_2	725,760	413,862	0.5702
18 Feb 2013 15:30:38	1245310	15604209	hadcm3n_4i7k_1940_40_008303257_2	699,840	427,600	0.6110
18 Feb 2013 10:13:37	1245310	15604209	hadcm3n_4i7k_1940_40_008303257_2	673,920	409,045	0.6070
18 Feb 2013 05:22:34	1245310	15604209	hadcm3n_4i7k_1940_40_008303257_2	648,000	390,317	0.6023
17 Feb 2013 23:16:07	1245310	15604209	hadcm3n_4i7k_1940_40_008303257_2	622,080	383,740	0.6169
17 Feb 2013 12:27:43	1245310	15604209	hadcm3n_4i7k_1940_40_008303257_2	596,160	365,102	0.6124
17 Feb 2013 07:26:43	1245310	15604209	hadcm3n_4i7k_1940_40_008303257_2	570,240	346,561	0.6077
16 Feb 2013 10:04:14	1245310	15604209	hadcm3n_4i7k_1940_40_008303257_2	544,320	327,805	0.6022
16 Feb 2013 05:03:10	1245310	15604209	hadcm3n_4i7k_1940_40_008303257_2	518,400	309,233	0.5965
15 Feb 2013 10:42:30	1245310	15604209	hadcm3n_4i7k_1940_40_008303257_2	492,480	290,621	0.5901
15 Feb 2013 02:25:52	1245310	15604209	hadcm3n_4i7k_1940_40_008303257_2	466,560	271,961	0.5829
14 Feb 2013 21:03:44	1245310	15604209	hadcm3n_4i7k_1940_40_008303257_2	440,640	253,434	0.5751
14 Feb 2013 15:45:41	1245310	15604209	hadcm3n_4i7k_1940_40_008303257_2	414,720	234,969	0.5666
14 Feb 2013 10:31:04	1245310	15604209	hadcm3n_4i7k_1940_40_008303257_2	388,800	216,437	0.5567
14 Feb 2013 05:14:59	1245310	15604209	hadcm3n_4i7k_1940_40_008303257_2	362,880	197,953	0.5455
14 Feb 2013 00:03:42	1245310	15604209	hadcm3n_4i7k_1940_40_008303257_2	336,960	179,450	0.5326