climateprediction.net home page
Task 15781145

Task 15781145

Name hadcm3n_n2od_1920_40_008367753_0
Workunit 8518612
Created 13 May 2013, 7:15:28 UTC
Sent 13 May 2013, 7:15:38 UTC
Report deadline 12 Aug 2013, 14:42:49 UTC
Received 30 May 2013, 16:49:30 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 22 (0x00000016) Unknown error code
Computer ID 1282401
Run time 17 days 2 hours 50 min 11 sec
CPU time 16 days 13 hours 7 min 7 sec
Validate state Invalid
Credit 5,909.76
Device peak FLOPS 2.01 GFLOPS
Application version UK Met Office Coupled Model Full Resolution Ocean v6.07
i686-pc-linux-gnu
Stderr 
<core_client_version>7.0.27</core_client_version>
<![CDATA[
<message>
process exited with code 22 (0x16, -234)
</message>
<stderr_txt>
15:42:45 (21270): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
17:25:14 (2326): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
17:25:15 (2326): No heartbeat from core client for 30 sec - exiting
17:25:16 (2326): No heartbeat from core client for 30 sec - exiting
09:35:20 (2951): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:35:21 (2951): No heartbeat from core client for 30 sec - exiting
09:35:22 (2951): No heartbeat from core client for 30 sec - exiting
09:35:23 (2951): No heartbeat from core client for 30 sec - exiting
09:35:24 (2951): No heartbeat from core client for 30 sec - exiting
09:35:25 (2951): No heartbeat from core client for 30 sec - exiting
09:35:26 (2951): No heartbeat from core client for 30 sec - exiting
09:35:27 (2951): No heartbeat from core client for 30 sec - exiting
09:35:28 (2951): No heartbeat from core client for 30 sec - exiting
09:35:29 (2951): No heartbeat from core client for 30 sec - exiting
09:35:30 (2951): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - Suspend request from BOINC...
06:40:34 (13111): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
10:36:02 (38865): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
10:36:03 (38865): No heartbeat from core client for 30 sec - exiting
10:40:28 (41659): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:03:47 (41874): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Atmos Hold Restart file rename failed on atmos_restart.hold
14:13:25 (43952): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:13:26 (43952): No heartbeat from core client for 30 sec - exiting
14:13:27 (43952): No heartbeat from core client for 30 sec - exiting
14:13:28 (43952): No heartbeat from core client for 30 sec - exiting
14:13:29 (43952): No heartbeat from core client for 30 sec - exiting
14:13:30 (43952): No heartbeat from core client for 30 sec - exiting
14:51:27 (44323): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
15:05:47 (44836): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
15:47:25 (45253): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
15:47:26 (45253): No heartbeat from core client for 30 sec - exiting
15:47:27 (45253): No heartbeat from core client for 30 sec - exiting
Atmos Hold Restart file rename failed on atmos_restart.hold
16:26:39 (45852): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
16:31:37 (46353): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
16:31:38 (46353): No heartbeat from core client for 30 sec - exiting
16:31:39 (46353): No heartbeat from core client for 30 sec - exiting
Atmos Hold Restart file rename failed on atmos_restart.hold
17:29:13 (46558): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
17:29:14 (46558): No heartbeat from core client for 30 sec - exiting
17:29:15 (46558): No heartbeat from core client for 30 sec - exiting
17:29:16 (46558): No heartbeat from core client for 30 sec - exiting
17:29:17 (46558): No heartbeat from core client for 30 sec - exiting
17:29:18 (46558): No heartbeat from core client for 30 sec - exiting
17:29:19 (46558): No heartbeat from core client for 30 sec - exiting
17:29:20 (46558): No heartbeat from core client for 30 sec - exiting
17:29:21 (46558): No heartbeat from core client for 30 sec - exiting
17:29:22 (46558): No heartbeat from core client for 30 sec - exiting
17:29:23 (46558): No heartbeat from core client for 30 sec - exiting
17:29:24 (46558): No heartbeat from core client for 30 sec - exiting
SIGABRT: abort called
Stack trace (9 frames):
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f]
[0xf770d400]
[0xf770d425]
/lib/i386-linux-gnu/libc.so.6(gsignal+0x4f)[0xf752a1df]
/lib/i386-linux-gnu/libc.so.6(abort+0x175)[0xf752d825]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8]
/lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0xf75154d3]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=47424, iMonCtr=1
Model crash detected, will try to restart...
SIGABRT: abort called
Stack trace (9 frames):
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f]
[0xf7789400]
[0xf7789425]
/lib/i386-linux-gnu/libc.so.6(gsignal+0x4f)[0xf75a61df]
/lib/i386-linux-gnu/libc.so.6(abort+0x175)[0xf75a9825]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8]
/lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0xf75914d3]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=47424, iMonCtr=1
Model crash detected, will try to restart...
SIGABRT: abort called
Stack trace (9 frames):
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f]
[0xf778e400]
[0xf778e425]
/lib/i386-linux-gnu/libc.so.6(gsignal+0x4f)[0xf75ab1df]
/lib/i386-linux-gnu/libc.so.6(abort+0x175)[0xf75ae825]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8]
/lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0xf75964d3]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=47424, iMonCtr=1
Model crash detected, will try to restart...
SIGABRT: abort called
Stack trace (9 frames):
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f]
[0xf772a400]
[0xf772a425]
/lib/i386-linux-gnu/libc.so.6(gsignal+0x4f)[0xf75471df]
/lib/i386-linux-gnu/libc.so.6(abort+0x175)[0xf754a825]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8]
/lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0xf75324d3]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=47424, iMonCtr=1
Model crash detected, will try to restart...
SIGABRT: abort called
Stack trace (9 frames):
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f]
[0xf7703400]
[0xf7703425]
/lib/i386-linux-gnu/libc.so.6(gsignal+0x4f)[0xf75201df]
/lib/i386-linux-gnu/libc.so.6(abort+0x175)[0xf7523825]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8]
/lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0xf750b4d3]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=47424, iMonCtr=1
Model crash detected, will try to restart...
SIGABRT: abort called
Stack trace (9 frames):
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f]
[0xf774a400]
[0xf774a425]
/lib/i386-linux-gnu/libc.so.6(gsignal+0x4f)[0xf75671df]
/lib/i386-linux-gnu/libc.so.6(abort+0x175)[0xf756a825]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8]
/lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0xf75524d3]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=47424, iMonCtr=1
Model crash detected, will try to restart...
Sorry, too many model crashes! :-(
Called boinc_finish

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
29 May 2013 17:56:40 1282401 15781145 hadcm3n_n2od_1920_40_008367753_0 492,480 1,356,948 2.7553
28 May 2013 20:11:33 1282401 15781145 hadcm3n_n2od_1920_40_008367753_0 466,560 1,278,908 2.7411
27 May 2013 22:29:32 1282401 15781145 hadcm3n_n2od_1920_40_008367753_0 440,640 1,205,547 2.7359
27 May 2013 01:31:25 1282401 15781145 hadcm3n_n2od_1920_40_008367753_0 414,720 1,133,615 2.7334
26 May 2013 04:59:03 1282401 15781145 hadcm3n_n2od_1920_40_008367753_0 388,800 1,061,575 2.7304
25 May 2013 06:26:19 1282401 15781145 hadcm3n_n2od_1920_40_008367753_0 362,880 983,269 2.7096
24 May 2013 08:52:56 1282401 15781145 hadcm3n_n2od_1920_40_008367753_0 336,960 905,782 2.6881
23 May 2013 10:38:43 1282401 15781145 hadcm3n_n2od_1920_40_008367753_0 311,040 830,039 2.6686
22 May 2013 14:10:24 1282401 15781145 hadcm3n_n2od_1920_40_008367753_0 285,120 762,498 2.6743
21 May 2013 19:56:46 1282401 15781145 hadcm3n_n2od_1920_40_008367753_0 259,200 698,003 2.6929
21 May 2013 01:40:24 1282401 15781145 hadcm3n_n2od_1920_40_008367753_0 233,280 631,853 2.7086
20 May 2013 06:55:15 1282401 15781145 hadcm3n_n2od_1920_40_008367753_0 207,360 566,010 2.7296
19 May 2013 12:18:23 1282401 15781145 hadcm3n_n2od_1920_40_008367753_0 181,440 500,065 2.7561
18 May 2013 17:07:20 1282401 15781145 hadcm3n_n2od_1920_40_008367753_0 155,520 433,396 2.7868
17 May 2013 20:21:52 1282401 15781145 hadcm3n_n2od_1920_40_008367753_0 129,600 362,183 2.7946
16 May 2013 23:31:36 1282401 15781145 hadcm3n_n2od_1920_40_008367753_0 103,680 286,813 2.7663
16 May 2013 01:19:33 1282401 15781145 hadcm3n_n2od_1920_40_008367753_0 77,760 210,063 2.7014
15 May 2013 03:08:33 1282401 15781145 hadcm3n_n2od_1920_40_008367753_0 51,840 133,927 2.5835
14 May 2013 06:58:04 1282401 15781145 hadcm3n_n2od_1920_40_008367753_0 25,920 62,382 2.4067


©2024 climateprediction.net