Task 16162797

Name	hadcm3n_oevp_1900_40_008474408_1
Workunit	8625247
Created	28 Dec 2013, 11:04:02 UTC
Sent	28 Dec 2013, 11:04:03 UTC
Report deadline	29 Mar 2014, 18:31:14 UTC
Received	9 Feb 2014, 11:03:31 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1299850
Run time	9 days 18 hours 50 min 33 sec
CPU time	9 days 14 hours 39 min 37 sec
Validate state	Invalid
Credit	7,776.00
Device peak FLOPS	2.93 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-pc-linux-gnu
Stderr	<core_client_version>7.2.33</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... Signal 15 received, exiting... Called boinc_finish CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 15 received, exiting... Called boinc_finish CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 15 received, exiting... Called boinc_finish Signal 15 received, exiting... Called boinc_finish CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 15 received, exiting... Signal 15 received, exiting... Called boinc_finish CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 15 received, exiting... Called boinc_finish CPDN Monitor - Quit request from BOINC... Signal 15 received, exiting... Called boinc_finish Signal 15 received, exiting... Called boinc_finish CPDN Monitor - Quit request from BOINC... Signal 15 received, exiting... Called boinc_finish CPDN Monitor - Quit request from BOINC... Signal 15 received, exiting... Called boinc_finish Signal 15 received, exiting... Called boinc_finish CPDN Monitor - Quit request from BOINC... Signal 15 received, exiting... Called boinc_finish Atmos Hold Restart file rename failed on atmos_restart.hold Signal 15 received, exiting... Called boinc_finish CPDN Monitor - Quit request from BOINC... Signal 15 received, exiting... Called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received, exiting... Called boinc_finish Signal 15 received, exiting... Called boinc_finish Signal 15 received, exiting... Signal 15 received, exiting... Called boinc_finish CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 15 received, exiting... Called boinc_finish Signal 15 received, exiting... Called boinc_finish CPDN Monitor - Quit request from BOINC... Signal 15 received, exiting... Called boinc_finish Signal 15 received, exiting... Called boinc_finish Signal 15 received, exiting... Signal 15 received, exiting... Called boinc_finish CPDN Monitor - Quit request from BOINC... Signal 15 received, exiting... Called boinc_finish CPDN Monitor - Quit request from BOINC... Signal 15 received, exiting... Called boinc_finish Signal 15 received, exiting... Called boinc_finish CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 15 received, exiting... Called boinc_finish CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 15 received, exiting... Called boinc_finish CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 15 received, exiting... Called boinc_finish CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 15 received, exiting... Called boinc_finish CPDN Monitor - Quit request from BOINC... Signal 15 received, exiting... Called boinc_finish Signal 15 received, exiting... Called boinc_finish CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 15 received, exiting... Called boinc_finish CPDN Monitor - Quit request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 15 received, exiting... Called boinc_finish CPDN Monitor - Quit request from BOINC... Signal 15 received, exiting... Called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 15 received, exiting... Called boinc_finish Signal 15 received, exiting... Called boinc_finish CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 15 received, exiting... Called boinc_finish CPDN Monitor - Quit request from BOINC... Signal 15 received, exiting... Called boinc_finish CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
06 Feb 2014 15:42:21	1299850	16162797	hadcm3n_oevp_1900_40_008474408_1	648,000	803,097	1.2393
05 Feb 2014 20:42:59	1299850	16162797	hadcm3n_oevp_1900_40_008474408_1	622,080	774,027	1.2443
03 Feb 2014 16:39:58	1299850	16162797	hadcm3n_oevp_1900_40_008474408_1	596,160	740,994	1.2429
30 Jan 2014 20:44:35	1299850	16162797	hadcm3n_oevp_1900_40_008474408_1	570,240	709,160	1.2436
30 Jan 2014 20:44:35	1299850	16162797	hadcm3n_oevp_1900_40_008474408_1	544,320	676,439	1.2427
30 Jan 2014 20:44:35	1299850	16162797	hadcm3n_oevp_1900_40_008474408_1	518,400	646,782	1.2477
27 Jan 2014 19:38:32	1299850	16162797	hadcm3n_oevp_1900_40_008474408_1	492,480	614,407	1.2476
24 Jan 2014 21:29:54	1299850	16162797	hadcm3n_oevp_1900_40_008474408_1	466,560	581,832	1.2471
24 Jan 2014 16:07:27	1299850	16162797	hadcm3n_oevp_1900_40_008474408_1	440,640	553,093	1.2552
24 Jan 2014 16:07:27	1299850	16162797	hadcm3n_oevp_1900_40_008474408_1	414,720	520,277	1.2545
20 Jan 2014 16:45:24	1299850	16162797	hadcm3n_oevp_1900_40_008474408_1	388,800	487,469	1.2538
17 Jan 2014 21:19:22	1299850	16162797	hadcm3n_oevp_1900_40_008474408_1	362,880	454,419	1.2523
16 Jan 2014 17:12:17	1299850	16162797	hadcm3n_oevp_1900_40_008474408_1	336,960	425,468	1.2627
14 Jan 2014 13:24:27	1299850	16162797	hadcm3n_oevp_1900_40_008474408_1	311,040	394,782	1.2692
10 Jan 2014 17:42:00	1299850	16162797	hadcm3n_oevp_1900_40_008474408_1	285,120	363,631	1.2754
10 Jan 2014 17:42:00	1299850	16162797	hadcm3n_oevp_1900_40_008474408_1	259,200	334,583	1.2908
08 Jan 2014 19:58:18	1299850	16162797	hadcm3n_oevp_1900_40_008474408_1	233,280	304,660	1.3060
08 Jan 2014 19:58:18	1299850	16162797	hadcm3n_oevp_1900_40_008474408_1	207,360	272,408	1.3137
07 Jan 2014 16:14:11	1299850	16162797	hadcm3n_oevp_1900_40_008474408_1	181,440	238,037	1.3119
05 Jan 2014 14:36:28	1299850	16162797	hadcm3n_oevp_1900_40_008474408_1	155,520	202,853	1.3044