Task 15456360

Name	hadcm3n_zmj9_1880_40_008252119_0
Workunit	8407243
Created	23 Nov 2012, 11:32:02 UTC
Sent	23 Nov 2012, 11:32:12 UTC
Report deadline	22 Feb 2013, 18:59:23 UTC
Received	3 Dec 2012, 21:30:06 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1227722
Run time	6 days 7 hours 24 min 2 sec
CPU time	5 days 23 hours 27 min 45 sec
Validate state	Invalid
Credit	4,043.52
Device peak FLOPS	1.71 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-pc-linux-gnu
Stderr	<core_client_version>7.0.25</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> Signal 2 received, exiting... Called boinc_finish Signal 2 received, exiting... Called boinc_finish Signal 2 received, exiting... Called boinc_finish Signal 2 received, exiting... Called boinc_finish Signal 2 received, exiting... Called boinc_finish Signal 2 received, exiting... Called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... Signal 2 received, exiting... Called boinc_finish Signal 2 received, exiting... Called boinc_finish CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 2 received, exiting... Called boinc_finish Signal 2 received, exiting... Called boinc_finish CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=28896, iMonCtr=1 Model crash detected, will try to restart... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 SIGSEGV: segmentation violation Stack trace (11 frames): /data/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf77b4400] /lib32/libm.so.6(sin+0x4a1)[0xf773f781] /data/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x822a4f8] /data/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x818a767] /data/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x818d749] /data/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8391957] /data/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f8b7] /data/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8] /lib32/libc.so.6(__libc_start_main+0xe6)[0xf75a1826] /data/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x804cb11] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=28896, iMonCtr=1 Model crash detected, will try to restart... SIGSEGV: segmentation violation Stack trace (11 frames): /data/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7712400] /lib32/libm.so.6(sin+0xa82)[0xf769dd62] /data/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x822a4f8] /data/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x818a767] /data/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x818d749] /data/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8391957] /data/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f8b7] /data/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8] /lib32/libc.so.6(__libc_start_main+0xe6)[0xf74ff826] /data/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x804cb11] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=28896, iMonCtr=1 Model crash detected, will try to restart... SIGSEGV: segmentation violation Stack trace (11 frames): /data/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf775a400] /lib32/libm.so.6(sin+0xa82)[0xf76e5d62] /data/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x822a4f8] /data/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x818a767] /data/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x818d749] /data/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8391957] /data/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f8b7] /data/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8] /lib32/libc.so.6(__libc_start_main+0xe6)[0xf7547826] /data/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x804cb11] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=28896, iMonCtr=1 Model crash detected, will try to restart... SIGSEGV: segmentation violation Stack trace (11 frames): /data/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf77de400] /lib32/libm.so.6(sin+0xa82)[0xf7769d62] /data/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x822a4f8] /data/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x818a767] /data/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x818d749] /data/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8391957] /data/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f8b7] /data/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8] /lib32/libc.so.6(__libc_start_main+0xe6)[0xf75cb826] /data/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x804cb11] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=28896, iMonCtr=1 Model crash detected, will try to restart... SIGSEGV: segmentation violation Stack trace (11 frames): /data/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7757400] /lib32/libm.so.6(sin+0xa82)[0xf76e2d62] /data/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x822a4f8] /data/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x818a767] /data/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x818d749] /data/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8391957] /data/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f8b7] /data/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8] /lib32/libc.so.6(__libc_start_main+0xe6)[0xf7544826] /data/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x804cb11] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=28896, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
02 Dec 2012 20:46:11	1227722	15456360	hadcm3n_zmj9_1880_40_008252119_0	336,960	483,388	1.4346
02 Dec 2012 10:03:51	1227722	15456360	hadcm3n_zmj9_1880_40_008252119_0	311,040	446,353	1.4350
01 Dec 2012 19:58:34	1227722	15456360	hadcm3n_zmj9_1880_40_008252119_0	285,120	409,069	1.4347
28 Nov 2012 08:01:40	1227722	15456360	hadcm3n_zmj9_1880_40_008252119_0	259,200	371,721	1.4341
27 Nov 2012 19:31:46	1227722	15456360	hadcm3n_zmj9_1880_40_008252119_0	233,280	334,229	1.4327
27 Nov 2012 08:23:34	1227722	15456360	hadcm3n_zmj9_1880_40_008252119_0	207,360	296,725	1.4310
26 Nov 2012 21:25:46	1227722	15456360	hadcm3n_zmj9_1880_40_008252119_0	181,440	259,252	1.4289
26 Nov 2012 10:25:30	1227722	15456360	hadcm3n_zmj9_1880_40_008252119_0	155,520	221,961	1.4272
25 Nov 2012 23:43:10	1227722	15456360	hadcm3n_zmj9_1880_40_008252119_0	129,600	185,239	1.4293
25 Nov 2012 12:51:18	1227722	15456360	hadcm3n_zmj9_1880_40_008252119_0	103,680	148,382	1.4312
24 Nov 2012 22:13:00	1227722	15456360	hadcm3n_zmj9_1880_40_008252119_0	77,760	111,165	1.4296
24 Nov 2012 11:03:58	1227722	15456360	hadcm3n_zmj9_1880_40_008252119_0	51,840	73,944	1.4264
23 Nov 2012 22:20:49	1227722	15456360	hadcm3n_zmj9_1880_40_008252119_0	25,920	36,894	1.4234