Task 15628495

Name	hadcm3n_o15m_2140_40_008269470_2
Workunit	8424594
Created	23 Feb 2013, 14:50:07 UTC
Sent	23 Feb 2013, 14:50:13 UTC
Report deadline	25 May 2013, 22:17:24 UTC
Received	20 Mar 2013, 19:20:35 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1115193
Run time	13 days 11 hours 28 min 22 sec
CPU time	12 days 11 hours 22 min 6 sec
Validate state	Invalid
Credit	9,331.20
Device peak FLOPS	2.86 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-pc-linux-gnu
Stderr	<core_client_version>6.10.58</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... Signal 15 received, exiting... Called boinc_finish CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 15 received, exiting... Called boinc_finish SIGSEGV: segmentation violation Stack trace (8 frames): /home/ggriffin/boinc/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7772400] /lib/libc.so.6(cfree+0x26)[0xf75f4f76] /home/ggriffin/boinc/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83f6a51] /home/ggriffin/boinc/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83f6a7d] /home/ggriffin/boinc/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839df27] /home/ggriffin/boinc/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83a5fb2] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7772400] Exiting... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 15 received, exiting... Called boinc_finish Signal 15 received, exiting... Called boinc_finish CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 15 received, exiting... Called boinc_finish MainError: 02:12:43 AM No files match the supplied pattern. MainError: 02:12:44 AM No files match the supplied pattern. MainError: 12:49:30 AM No files match the supplied pattern. MainError: 12:49:30 AM No files match the supplied pattern. Signal 15 received, exiting... Called boinc_finish MainError: 05:08:34 AM No files match the supplied pattern. MainError: 05:08:34 AM No files match the supplied pattern. MainError: 03:57:42 PM No files match the supplied pattern. MainError: 03:57:42 PM No files match the supplied pattern. Signal 15 received, exiting... Called boinc_finish MainError: 01:33:20 PM No files match the supplied pattern. MainError: 01:33:20 PM No files match the supplied pattern. MainError: 12:44:09 AM No files match the supplied pattern. MainError: 12:44:09 AM No files match the supplied pattern. CPDN Monitor - Quit request from BOINC... MainError: 11:58:38 AM No files match the supplied pattern. MainError: 11:58:38 AM No files match the supplied pattern. Signal 1 received, exiting... Called boinc_finish Signal 15 received, exiting... Called boinc_finish Signal 15 received, exiting... Called boinc_finish Signal 1 received, exiting... Called boinc_finish Signal 15 received, exiting... Called boinc_finish CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 15 received, exiting... Called boinc_finish MainError: 06:57:37 AM No files match the supplied pattern. MainError: 06:57:37 AM No files match the supplied pattern. MainError: 05:29:47 PM No files match the supplied pattern. MainError: 05:29:47 PM No files match the supplied pattern. CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... MainError: 05:11:06 AM No files match the supplied pattern. MainError: 05:11:06 AM No files match the supplied pattern. Error converting file to netcdf: dataout/o15mka.ph11c10 Error converting file to netcdf: dataout/o15mka.pg11c10 Error converting file to netcdf: dataout/o15mka.pe11c10 MainError: 03:49:40 PM No files match the supplied pattern. MainError: 03:49:40 PM No files match the supplied pattern. BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 67 - Return code = 1 Model crashed: STWORK : I/O error - PP fixed length header tmp/pipe_dummy 2048 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 64 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 66 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 67 - Return code = 1 BUFFIN: Read Failed: Numerical result out of range BUFFIN: C I/O Error feof - Unit 67 - Return code = 1 Model crashed: STWORK : I/O error - PP fixed length header tmp/pipe_dummy 2048 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 64 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 66 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 67 - Return code = 1 BUFFIN: Read Failed: Numerical result out of range BUFFIN: C I/O Error feof - Unit 67 - Return code = 1 Model crashed: STWORK : I/O error - PP fixed length header tmp/pipe_dummy 2048 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 64 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 66 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 67 - Return code = 1 BUFFIN: Read Failed: Numerical result out of range BUFFIN: C I/O Error feof - Unit 67 - Return code = 1 Model crashed: STWORK : I/O error - PP fixed length header tmp/pipe_dummy 2048 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 64 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 66 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 67 - Return code = 1 BUFFIN: Read Failed: Numerical result out of range BUFFIN: C I/O Error feof - Unit 67 - Return code = 1 Model crashed: STWORK : I/O error - PP fixed length header tmp/pipe_dummy 2048 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 64 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 66 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 67 - Return code = 1 BUFFIN: Read Failed: Numerical result out of range BUFFIN: C I/O Error feof - Unit 67 - Return code = 1 Model crashed: STWORK : I/O error - PP fixed length header tmp/pipe_dummy 2048 Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
20 Mar 2013 15:50:34	1115193	15628495	hadcm3n_o15m_2140_40_008269470_2	777,600	1,134,488	1.4590
20 Mar 2013 06:01:20	1115193	15628495	hadcm3n_o15m_2140_40_008269470_2	751,680	1,096,892	1.4593
19 Mar 2013 17:33:47	1115193	15628495	hadcm3n_o15m_2140_40_008269470_2	725,760	1,059,218	1.4595
19 Mar 2013 07:48:48	1115193	15628495	hadcm3n_o15m_2140_40_008269470_2	699,840	1,021,901	1.4602
11 Mar 2013 11:59:17	1115193	15628495	hadcm3n_o15m_2140_40_008269470_2	673,920	984,467	1.4608
11 Mar 2013 01:30:20	1115193	15628495	hadcm3n_o15m_2140_40_008269470_2	648,000	945,919	1.4598
10 Mar 2013 13:35:10	1115193	15628495	hadcm3n_o15m_2140_40_008269470_2	622,080	907,914	1.4595
09 Mar 2013 16:00:32	1115193	15628495	hadcm3n_o15m_2140_40_008269470_2	596,160	870,542	1.4602
09 Mar 2013 06:01:44	1115193	15628495	hadcm3n_o15m_2140_40_008269470_2	570,240	832,244	1.4595
08 Mar 2013 12:52:45	1115193	15628495	hadcm3n_o15m_2140_40_008269470_2	544,320	794,473	1.4596
08 Mar 2013 02:14:27	1115193	15628495	hadcm3n_o15m_2140_40_008269470_2	518,400	756,918	1.4601
07 Mar 2013 15:40:37	1115193	15628495	hadcm3n_o15m_2140_40_008269470_2	492,480	719,605	1.4612
07 Mar 2013 04:59:22	1115193	15628495	hadcm3n_o15m_2140_40_008269470_2	466,560	681,748	1.4612
06 Mar 2013 18:22:26	1115193	15628495	hadcm3n_o15m_2140_40_008269470_2	440,640	643,722	1.4609
06 Mar 2013 07:30:31	1115193	15628495	hadcm3n_o15m_2140_40_008269470_2	414,720	605,669	1.4604
05 Mar 2013 19:45:28	1115193	15628495	hadcm3n_o15m_2140_40_008269470_2	388,800	567,978	1.4608
05 Mar 2013 08:35:50	1115193	15628495	hadcm3n_o15m_2140_40_008269470_2	362,880	530,762	1.4626
04 Mar 2013 21:57:21	1115193	15628495	hadcm3n_o15m_2140_40_008269470_2	336,960	493,548	1.4647
04 Mar 2013 10:28:52	1115193	15628495	hadcm3n_o15m_2140_40_008269470_2	311,040	456,423	1.4674
03 Mar 2013 23:00:29	1115193	15628495	hadcm3n_o15m_2140_40_008269470_2	285,120	419,362	1.4708