climateprediction.net home page
Task 15832987

Task 15832987

Name hadcm3n_o1v1_2140_40_008269390_4
Workunit 8424514
Created 6 Jun 2013, 14:23:30 UTC
Sent 6 Jun 2013, 14:30:38 UTC
Report deadline 5 Sep 2013, 21:57:49 UTC
Received 8 Jun 2013, 11:05:32 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 22 (0x00000016) Unknown error code
Computer ID 1282401
Run time 1 days 11 hours 6 min 11 sec
CPU time 1 days 10 hours 13 min 52 sec
Validate state Invalid
Credit 311.04
Device peak FLOPS 1.99 GFLOPS
Application version UK Met Office Coupled Model Full Resolution Ocean v6.07
i686-pc-linux-gnu
Stderr
<core_client_version>7.0.27</core_client_version>
<![CDATA[
<message>
process exited with code 22 (0x16, -234)
</message>
<stderr_txt>
16:04:08 (33593): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
16:08:45 (33708): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...

Model crashed: CHK_LOOK: Consistency check                                                                                                                                                                                                                                     tmp/pipe_dummy                                                                  2048    
16:50:39 (33800): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
16:54:33 (34255): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
16:59:02 (34402): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
17:04:09 (34552): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
17:08:24 (34703): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
17:08:25 (34703): No heartbeat from core client for 30 sec - exiting
17:12:44 (34854): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
18:30:03 (34976): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
18:34:22 (35718): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
18:34:30 (35718): No heartbeat from core client for 30 sec - exiting
18:39:09 (35863): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
18:48:37 (36010): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
18:48:38 (36010): No heartbeat from core client for 30 sec - exiting
18:48:39 (36010): No heartbeat from core client for 30 sec - exiting
18:52:38 (36094): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
18:57:02 (36200): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
19:01:53 (36328): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
19:01:59 (36328): No heartbeat from core client for 30 sec - exiting
19:06:20 (36466): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
19:10:45 (36630): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
19:10:46 (36630): No heartbeat from core client for 30 sec - exiting
19:10:47 (36630): No heartbeat from core client for 30 sec - exiting
19:10:48 (36630): No heartbeat from core client for 30 sec - exiting
19:10:49 (36630): No heartbeat from core client for 30 sec - exiting
19:10:50 (36630): No heartbeat from core client for 30 sec - exiting
19:10:51 (36630): No heartbeat from core client for 30 sec - exiting
19:10:52 (36630): No heartbeat from core client for 30 sec - exiting
23:35:10 (36773): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
23:39:16 (39052): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
23:44:03 (39210): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
23:44:04 (39210): No heartbeat from core client for 30 sec - exiting
23:44:05 (39210): No heartbeat from core client for 30 sec - exiting
23:44:06 (39210): No heartbeat from core client for 30 sec - exiting
23:48:34 (39360): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
23:48:35 (39360): No heartbeat from core client for 30 sec - exiting
23:48:36 (39360): No heartbeat from core client for 30 sec - exiting
23:48:37 (39360): No heartbeat from core client for 30 sec - exiting
23:48:38 (39360): No heartbeat from core client for 30 sec - exiting
23:48:39 (39360): No heartbeat from core client for 30 sec - exiting
23:48:40 (39360): No heartbeat from core client for 30 sec - exiting
23:48:41 (39360): No heartbeat from core client for 30 sec - exiting
23:48:42 (39360): No heartbeat from core client for 30 sec - exiting
23:53:21 (39513): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
23:53:22 (39513): No heartbeat from core client for 30 sec - exiting
23:53:23 (39513): No heartbeat from core client for 30 sec - exiting
23:53:24 (39513): No heartbeat from core client for 30 sec - exiting
23:53:25 (39513): No heartbeat from core client for 30 sec - exiting
23:53:26 (39513): No heartbeat from core client for 30 sec - exiting
23:53:27 (39513): No heartbeat from core client for 30 sec - exiting
23:53:28 (39513): No heartbeat from core client for 30 sec - exiting
23:53:29 (39513): No heartbeat from core client for 30 sec - exiting
01:32:51 (39661): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
01:36:32 (40612): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
01:40:22 (40768): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
01:55:34 (40910): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
01:59:52 (41147): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
02:08:30 (41301): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
02:32:11 (41530): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
02:32:12 (41530): No heartbeat from core client for 30 sec - exiting
02:32:13 (41530): No heartbeat from core client for 30 sec - exiting
02:32:14 (41530): No heartbeat from core client for 30 sec - exiting
02:32:15 (41530): No heartbeat from core client for 30 sec - exiting
02:32:16 (41530): No heartbeat from core client for 30 sec - exiting
02:32:17 (41530): No heartbeat from core client for 30 sec - exiting
02:32:18 (41530): No heartbeat from core client for 30 sec - exiting
02:32:19 (41530): No heartbeat from core client for 30 sec - exiting
02:32:20 (41530): No heartbeat from core client for 30 sec - exiting
02:32:21 (41530): No heartbeat from core client for 30 sec - exiting
02:40:59 (41872): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
02:41:00 (41872): No heartbeat from core client for 30 sec - exiting
02:41:01 (41872): No heartbeat from core client for 30 sec - exiting
02:41:02 (41872): No heartbeat from core client for 30 sec - exiting
02:41:03 (41872): No heartbeat from core client for 30 sec - exiting
02:41:04 (41872): No heartbeat from core client for 30 sec - exiting
02:41:05 (41872): No heartbeat from core client for 30 sec - exiting
02:41:06 (41872): No heartbeat from core client for 30 sec - exiting
02:41:07 (41872): No heartbeat from core client for 30 sec - exiting
02:41:08 (41872): No heartbeat from core client for 30 sec - exiting
02:45:12 (42072): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
02:45:13 (42072): No heartbeat from core client for 30 sec - exiting
02:45:14 (42072): No heartbeat from core client for 30 sec - exiting
02:45:15 (42072): No heartbeat from core client for 30 sec - exiting
02:45:16 (42072): No heartbeat from core client for 30 sec - exiting
02:45:17 (42072): No heartbeat from core client for 30 sec - exiting
02:45:18 (42072): No heartbeat from core client for 30 sec - exiting
02:45:19 (42072): No heartbeat from core client for 30 sec - exiting
02:45:20 (42072): No heartbeat from core client for 30 sec - exiting
02:50:05 (42233): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
02:54:51 (42391): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
02:59:05 (42570): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
03:03:37 (42719): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
03:08:03 (42880): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
05:59:36 (43098): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
11:56:02 (44678): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
11:56:03 (44678): No heartbeat from core client for 30 sec - exiting
11:56:04 (44678): No heartbeat from core client for 30 sec - exiting
11:59:42 (48161): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
15:46:57 (2718): No heartbeat from core client for 30 sec - exiting
15:47:27 (2718): No heartbeat from core client for 30 sec - exiting
15:47:28 (2718): No heartbeat from core client for 30 sec - exiting
15:47:29 (2718): No heartbeat from core client for 30 sec - exiting
15:47:30 (2718): No heartbeat from core client for 30 sec - exiting
15:47:31 (2718): No heartbeat from core client for 30 sec - exiting
15:47:33 (2718): No heartbeat from core client for 30 sec - exiting
15:47:34 (2718): No heartbeat from core client for 30 sec - exiting
15:47:35 (2718): No heartbeat from core client for 30 sec - exiting
15:47:36 (2718): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
19:21:40 (3044): No heartbeat from core client for 30 sec - exiting
19:21:41 (3044): No heartbeat from core client for 30 sec - exiting
19:21:42 (3044): No heartbeat from core client for 30 sec - exiting
19:21:43 (3044): No heartbeat from core client for 30 sec - exiting
19:21:44 (3044): No heartbeat from core client for 30 sec - exiting
19:21:45 (3044): No heartbeat from core client for 30 sec - exiting
19:21:46 (3044): No heartbeat from core client for 30 sec - exiting
19:21:47 (3044): No heartbeat from core client for 30 sec - exiting
19:21:48 (3044): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
19:56:28 (5064): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
19:56:29 (5064): No heartbeat from core client for 30 sec - exiting
19:56:30 (5064): No heartbeat from core client for 30 sec - exiting
19:56:31 (5064): No heartbeat from core client for 30 sec - exiting
19:56:32 (5064): No heartbeat from core client for 30 sec - exiting
20:01:17 (5519): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:06:33 (5729): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:06:34 (5729): No heartbeat from core client for 30 sec - exiting
20:06:35 (5729): No heartbeat from core client for 30 sec - exiting
20:06:36 (5729): No heartbeat from core client for 30 sec - exiting
20:06:37 (5729): No heartbeat from core client for 30 sec - exiting
20:06:38 (5729): No heartbeat from core client for 30 sec - exiting
20:06:39 (5729): No heartbeat from core client for 30 sec - exiting
20:06:40 (5729): No heartbeat from core client for 30 sec - exiting
20:06:41 (5729): No heartbeat from core client for 30 sec - exiting
20:11:20 (5935): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:20:12 (6137): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:24:17 (6349): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:13:37 (6527): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:18:39 (7097): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:18:40 (7097): No heartbeat from core client for 30 sec - exiting
21:18:41 (7097): No heartbeat from core client for 30 sec - exiting
21:18:42 (7097): No heartbeat from core client for 30 sec - exiting
22:01:15 (7305): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
22:05:13 (7833): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:13:15 (8004): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:17:00 (13873): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:43:36 (14062): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:48:13 (14428): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:51:59 (14618): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:52:00 (14618): No heartbeat from core client for 30 sec - exiting
08:52:01 (14618): No heartbeat from core client for 30 sec - exiting
08:52:02 (14618): No heartbeat from core client for 30 sec - exiting
08:52:03 (14618): No heartbeat from core client for 30 sec - exiting
08:52:04 (14618): No heartbeat from core client for 30 sec - exiting
08:52:05 (14618): No heartbeat from core client for 30 sec - exiting
08:52:06 (14618): No heartbeat from core client for 30 sec - exiting
08:52:07 (14618): No heartbeat from core client for 30 sec - exiting
08:52:08 (14618): No heartbeat from core client for 30 sec - exiting
08:52:09 (14618): No heartbeat from core client for 30 sec - exiting
08:52:10 (14618): No heartbeat from core client for 30 sec - exiting
08:52:11 (14618): No heartbeat from core client for 30 sec - exiting
08:52:12 (14618): No heartbeat from core client for 30 sec - exiting
08:52:13 (14618): No heartbeat from core client for 30 sec - exiting
08:52:14 (14618): No heartbeat from core client for 30 sec - exiting
Atmos Hold Restart file rename failed on atmos_restart.hold
09:43:32 (14801): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:48:15 (15380): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:52:58 (15577): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
11:50:57 (15785): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
SIGABRT: abort called
Stack trace (9 frames):
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f]
[0xf772c400]
[0xf772c425]
/lib/i386-linux-gnu/libc.so.6(gsignal+0x4f)[0xf75491df]
/lib/i386-linux-gnu/libc.so.6(abort+0x175)[0xf754c825]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8]
/lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0xf75344d3]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=16959, iMonCtr=1
Model crash detected, will try to restart...
SIGABRT: abort called
Stack trace (9 frames):
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f]
[0xf771c400]
[0xf771c425]
/lib/i386-linux-gnu/libc.so.6(gsignal+0x4f)[0xf75391df]
/lib/i386-linux-gnu/libc.so.6(abort+0x175)[0xf753c825]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8]
/lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0xf75244d3]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=16959, iMonCtr=1
Model crash detected, will try to restart...
SIGABRT: abort called
Stack trace (9 frames):
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f]
[0xf770f400]
[0xf770f425]
/lib/i386-linux-gnu/libc.so.6(gsignal+0x4f)[0xf752c1df]
/lib/i386-linux-gnu/libc.so.6(abort+0x175)[0xf752f825]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8]
/lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0xf75174d3]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=16959, iMonCtr=1
Model crash detected, will try to restart...
SIGABRT: abort called
Stack trace (9 frames):
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f]
[0xf771e400]
[0xf771e425]
/lib/i386-linux-gnu/libc.so.6(gsignal+0x4f)[0xf753b1df]
/lib/i386-linux-gnu/libc.so.6(abort+0x175)[0xf753e825]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8]
/lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0xf75264d3]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=16959, iMonCtr=1
Model crash detected, will try to restart...
SIGABRT: abort called
Stack trace (9 frames):
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f]
[0xf773d400]
[0xf773d425]
/lib/i386-linux-gnu/libc.so.6(gsignal+0x4f)[0xf755a1df]
/lib/i386-linux-gnu/libc.so.6(abort+0x175)[0xf755d825]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8]
/lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0xf75454d3]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=16959, iMonCtr=1
Model crash detected, will try to restart...
SIGABRT: abort called
Stack trace (9 frames):
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f]
[0xf77b0400]
[0xf77b0425]
/lib/i386-linux-gnu/libc.so.6(gsignal+0x4f)[0xf75cd1df]
/lib/i386-linux-gnu/libc.so.6(abort+0x175)[0xf75d0825]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8]
/lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0xf75b84d3]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=16959, iMonCtr=1
Model crash detected, will try to restart...
Sorry, too many model crashes! :-(
Called boinc_finish

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
07 Jun 2013 15:51:12 1282401 15832987 hadcm3n_o1v1_2140_40_008269390_4 25,920 60,696 2.3417


©2024 climateprediction.net