climateprediction.net home page
Task 15593886

Task 15593886

Name hadcm3n_4dp8_1940_40_008307357_0
Workunit 8458492
Created 7 Feb 2013, 13:23:26 UTC
Sent 7 Feb 2013, 13:31:11 UTC
Report deadline 9 May 2013, 20:58:22 UTC
Received 12 Mar 2013, 18:57:46 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 22 (0x00000016) Unknown error code
Computer ID 1421091
Run time 32 days 13 hours 32 min 20 sec
CPU time 31 days 23 hours 19 min 41 sec
Validate state Invalid
Credit 10,575.36
Device peak FLOPS 1.41 GFLOPS
Application version UK Met Office Coupled Model Full Resolution Ocean v6.07
i686-pc-linux-gnu
Stderr 
<core_client_version>7.0.27</core_client_version>
<![CDATA[
<message>
process exited with code 22 (0x16, -234)
</message>
<stderr_txt>
03:06:47 (10002): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
03:08:39 (4496): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
03:09:30 (4505): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
03:12:42 (4514): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
03:16:05 (4524): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
03:19:12 (4533): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
03:22:15 (4545): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
03:25:07 (4554): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
03:28:19 (4564): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
03:31:31 (4573): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
03:34:29 (4583): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
03:34:30 (4583): No heartbeat from core client for 30 sec - exiting
03:34:31 (4583): No heartbeat from core client for 30 sec - exiting
03:34:32 (4583): No heartbeat from core client for 30 sec - exiting
03:37:21 (4591): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
03:40:08 (4602): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
03:43:15 (4610): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
03:46:13 (4620): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
03:46:14 (4620): No heartbeat from core client for 30 sec - exiting
03:46:15 (4620): No heartbeat from core client for 30 sec - exiting
03:46:16 (4620): No heartbeat from core client for 30 sec - exiting
03:49:19 (4628): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
03:52:42 (4638): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
03:55:35 (4647): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
03:58:27 (4657): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
04:01:38 (4665): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
04:04:42 (4675): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
04:04:43 (4675): No heartbeat from core client for 30 sec - exiting
04:07:53 (4684): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
04:10:56 (4694): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
04:14:03 (4702): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
04:16:45 (4712): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
04:19:43 (4724): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
04:22:45 (4734): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
04:25:27 (4742): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
04:28:39 (4752): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
04:31:42 (4761): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
04:34:53 (4771): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
04:37:21 (4779): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
04:40:03 (4789): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
04:42:36 (4797): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
04:45:47 (4808): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
04:48:29 (4816): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
04:51:41 (4826): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
04:54:34 (4834): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
04:57:46 (4845): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
05:00:49 (4853): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
SIGABRT: abort called
Stack trace (9 frames):
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f]
[0xf77c3400]
[0xf77c3425]
/lib/i386-linux-gnu/libc.so.6(gsignal+0x4f)[0xf75dc1df]
/lib/i386-linux-gnu/libc.so.6(abort+0x175)[0xf75df825]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8]
/lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0xf75c74d3]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4863, iMonCtr=1
Model crash detected, will try to restart...
SIGABRT: abort called
Stack trace (9 frames):
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f]
[0xf77ab400]
[0xf77ab425]
/lib/i386-linux-gnu/libc.so.6(gsignal+0x4f)[0xf75c41df]
/lib/i386-linux-gnu/libc.so.6(abort+0x175)[0xf75c7825]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8]
/lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0xf75af4d3]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4863, iMonCtr=1
Model crash detected, will try to restart...
SIGABRT: abort called
Stack trace (9 frames):
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f]
[0xf76fa400]
[0xf76fa425]
/lib/i386-linux-gnu/libc.so.6(gsignal+0x4f)[0xf75131df]
/lib/i386-linux-gnu/libc.so.6(abort+0x175)[0xf7516825]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8]
/lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0xf74fe4d3]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4863, iMonCtr=1
Model crash detected, will try to restart...
SIGABRT: abort called
Stack trace (9 frames):
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f]
[0xf778d400]
[0xf778d425]
/lib/i386-linux-gnu/libc.so.6(gsignal+0x4f)[0xf75a61df]
/lib/i386-linux-gnu/libc.so.6(abort+0x175)[0xf75a9825]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8]
/lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0xf75914d3]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4863, iMonCtr=1
Model crash detected, will try to restart...
SIGABRT: abort called
Stack trace (9 frames):
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f]
[0xf772d400]
[0xf772d425]
/lib/i386-linux-gnu/libc.so.6(gsignal+0x4f)[0xf75461df]
/lib/i386-linux-gnu/libc.so.6(abort+0x175)[0xf7549825]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8]
/lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0xf75314d3]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4863, iMonCtr=1
Model crash detected, will try to restart...
SIGABRT: abort called
Stack trace (9 frames):
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f]
[0xf77b2400]
[0xf77b2425]
/lib/i386-linux-gnu/libc.so.6(gsignal+0x4f)[0xf75cb1df]
/lib/i386-linux-gnu/libc.so.6(abort+0x175)[0xf75ce825]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8]
/lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0xf75b64d3]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4863, iMonCtr=1
Model crash detected, will try to restart...
Sorry, too many model crashes! :-(
Called boinc_finish

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
11 Mar 2013 17:31:16 1261273 15593886 hadcm3n_4dp8_1940_40_008307357_0 881,280 2,728,462 3.0960
10 Mar 2013 18:44:12 1261273 15593886 hadcm3n_4dp8_1940_40_008307357_0 855,360 2,647,978 3.0957
09 Mar 2013 20:01:22 1261273 15593886 hadcm3n_4dp8_1940_40_008307357_0 829,440 2,567,619 3.0956
08 Mar 2013 21:18:41 1261273 15593886 hadcm3n_4dp8_1940_40_008307357_0 803,520 2,487,315 3.0955
07 Mar 2013 22:32:56 1261273 15593886 hadcm3n_4dp8_1940_40_008307357_0 777,600 2,406,866 3.0952
06 Mar 2013 23:51:56 1261273 15593886 hadcm3n_4dp8_1940_40_008307357_0 751,680 2,326,613 3.0952
06 Mar 2013 01:04:02 1261273 15593886 hadcm3n_4dp8_1940_40_008307357_0 725,760 2,246,090 3.0948
05 Mar 2013 02:19:04 1261273 15593886 hadcm3n_4dp8_1940_40_008307357_0 699,840 2,165,768 3.0947
04 Mar 2013 03:37:02 1261273 15593886 hadcm3n_4dp8_1940_40_008307357_0 673,920 2,085,479 3.0945
03 Mar 2013 04:55:00 1261273 15593886 hadcm3n_4dp8_1940_40_008307357_0 648,000 2,005,112 3.0943
02 Mar 2013 06:07:09 1261273 15593886 hadcm3n_4dp8_1940_40_008307357_0 622,080 1,924,754 3.0941
01 Mar 2013 07:22:08 1261273 15593886 hadcm3n_4dp8_1940_40_008307357_0 596,160 1,844,302 3.0936
28 Feb 2013 08:42:44 1261273 15593886 hadcm3n_4dp8_1940_40_008307357_0 570,240 1,764,062 3.0935
27 Feb 2013 10:02:05 1261273 15593886 hadcm3n_4dp8_1940_40_008307357_0 544,320 1,683,778 3.0934
26 Feb 2013 11:11:03 1261273 15593886 hadcm3n_4dp8_1940_40_008307357_0 518,400 1,603,300 3.0928
25 Feb 2013 12:26:01 1261273 15593886 hadcm3n_4dp8_1940_40_008307357_0 492,480 1,522,965 3.0924
24 Feb 2013 13:45:30 1261273 15593886 hadcm3n_4dp8_1940_40_008307357_0 466,560 1,442,675 3.0922
23 Feb 2013 15:05:36 1261273 15593886 hadcm3n_4dp8_1940_40_008307357_0 440,640 1,362,406 3.0919
22 Feb 2013 16:20:44 1261273 15593886 hadcm3n_4dp8_1940_40_008307357_0 414,720 1,282,154 3.0916
21 Feb 2013 17:41:27 1261273 15593886 hadcm3n_4dp8_1940_40_008307357_0 388,800 1,201,952 3.0914


©2024 climateprediction.net