climateprediction.net home page
Task 15780192

Task 15780192

Name hadcm3n_39aj_1980_40_008367030_0
Workunit 8517889
Created 13 May 2013, 4:13:14 UTC
Sent 13 May 2013, 4:13:29 UTC
Report deadline 12 Aug 2013, 11:40:40 UTC
Received 31 May 2013, 12:37:24 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 22 (0x00000016) Unknown error code
Computer ID 1282401
Run time 17 days 21 hours 45 min 49 sec
CPU time 17 days 6 hours 39 min 46 sec
Validate state Invalid
Credit 6,531.84
Device peak FLOPS 2.01 GFLOPS
Application version UK Met Office Coupled Model Full Resolution Ocean v6.07
i686-pc-linux-gnu
Stderr 
<core_client_version>7.0.27</core_client_version>
<![CDATA[
<message>
process exited with code 22 (0x16, -234)
</message>
<stderr_txt>
15:42:45 (20935): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
17:25:14 (2319): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
17:25:15 (2319): No heartbeat from core client for 30 sec - exiting
17:25:16 (2319): No heartbeat from core client for 30 sec - exiting
09:35:20 (2921): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
02:02:27 (13083): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
06:40:35 (9886): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
06:40:36 (9886): No heartbeat from core client for 30 sec - exiting
06:40:37 (9886): No heartbeat from core client for 30 sec - exiting
06:40:38 (9886): No heartbeat from core client for 30 sec - exiting
06:40:39 (9886): No heartbeat from core client for 30 sec - exiting
06:40:40 (9886): No heartbeat from core client for 30 sec - exiting
06:40:41 (9886): No heartbeat from core client for 30 sec - exiting
06:40:42 (9886): No heartbeat from core client for 30 sec - exiting
06:40:43 (9886): No heartbeat from core client for 30 sec - exiting
06:40:44 (9886): No heartbeat from core client for 30 sec - exiting
06:40:45 (9886): No heartbeat from core client for 30 sec - exiting
06:40:46 (9886): No heartbeat from core client for 30 sec - exiting
06:40:47 (9886): No heartbeat from core client for 30 sec - exiting
06:40:48 (9886): No heartbeat from core client for 30 sec - exiting
06:40:49 (9886): No heartbeat from core client for 30 sec - exiting
06:40:50 (9886): No heartbeat from core client for 30 sec - exiting
06:40:51 (9886): No heartbeat from core client for 30 sec - exiting
06:40:52 (9886): No heartbeat from core client for 30 sec - exiting
06:40:53 (9886): No heartbeat from core client for 30 sec - exiting
06:40:54 (9886): No heartbeat from core client for 30 sec - exiting
06:40:55 (9886): No heartbeat from core client for 30 sec - exiting
06:40:56 (9886): No heartbeat from core client for 30 sec - exiting
06:40:57 (9886): No heartbeat from core client for 30 sec - exiting
10:36:02 (38878): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
10:40:27 (41671): No heartbeat from core client for 30 sec - exiting
10:40:28 (41671): No heartbeat from core client for 30 sec - exiting
10:40:29 (41671): No heartbeat from core client for 30 sec - exiting
10:40:30 (41671): No heartbeat from core client for 30 sec - exiting
10:40:31 (41671): No heartbeat from core client for 30 sec - exiting
10:40:32 (41671): No heartbeat from core client for 30 sec - exiting
10:40:33 (41671): No heartbeat from core client for 30 sec - exiting
10:40:34 (41671): No heartbeat from core client for 30 sec - exiting
10:40:35 (41671): No heartbeat from core client for 30 sec - exiting
10:40:36 (41671): No heartbeat from core client for 30 sec - exiting
10:40:37 (41671): No heartbeat from core client for 30 sec - exiting
10:40:38 (41671): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:03:47 (41886): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:13:25 (43960): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:51:26 (44334): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:51:27 (44334): No heartbeat from core client for 30 sec - exiting
14:51:28 (44334): No heartbeat from core client for 30 sec - exiting
Atmos Hold Restart file rename failed on atmos_restart.hold
15:05:47 (44845): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
15:42:26 (45260): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
15:47:25 (45731): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
15:47:26 (45731): No heartbeat from core client for 30 sec - exiting
16:26:39 (45942): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
16:31:38 (46414): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
16:31:39 (46414): No heartbeat from core client for 30 sec - exiting
16:31:40 (46414): No heartbeat from core client for 30 sec - exiting
17:29:13 (46629): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
17:38:37 (47449): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
18:23:56 (47690): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
18:28:22 (48413): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
18:28:23 (48413): No heartbeat from core client for 30 sec - exiting
18:28:24 (48413): No heartbeat from core client for 30 sec - exiting
18:28:25 (48413): No heartbeat from core client for 30 sec - exiting
19:51:14 (48604): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
19:51:15 (48604): No heartbeat from core client for 30 sec - exiting
19:56:00 (49562): No heartbeat from core client for 30 sec - exiting
19:56:01 (49562): No heartbeat from core client for 30 sec - exiting
19:56:02 (49562): No heartbeat from core client for 30 sec - exiting
19:56:03 (49562): No heartbeat from core client for 30 sec - exiting
19:56:04 (49562): No heartbeat from core client for 30 sec - exiting
19:56:05 (49562): No heartbeat from core client for 30 sec - exiting
19:56:06 (49562): No heartbeat from core client for 30 sec - exiting
19:56:07 (49562): No heartbeat from core client for 30 sec - exiting
19:56:08 (49562): No heartbeat from core client for 30 sec - exiting
19:56:09 (49562): No heartbeat from core client for 30 sec - exiting
19:56:10 (49562): No heartbeat from core client for 30 sec - exiting
19:56:11 (49562): No heartbeat from core client for 30 sec - exiting
19:56:12 (49562): No heartbeat from core client for 30 sec - exiting
19:56:13 (49562): No heartbeat from core client for 30 sec - exiting
19:56:14 (49562): No heartbeat from core client for 30 sec - exiting
19:56:15 (49562): No heartbeat from core client for 30 sec - exiting
19:56:16 (49562): No heartbeat from core client for 30 sec - exiting
19:56:17 (49562): No heartbeat from core client for 30 sec - exiting
19:56:18 (49562): No heartbeat from core client for 30 sec - exiting
19:56:19 (49562): No heartbeat from core client for 30 sec - exiting
19:56:20 (49562): No heartbeat from core client for 30 sec - exiting
19:56:21 (49562): No heartbeat from core client for 30 sec - exiting
19:56:22 (49562): No heartbeat from core client for 30 sec - exiting
19:56:23 (49562): No heartbeat from core client for 30 sec - exiting
19:56:24 (49562): No heartbeat from core client for 30 sec - exiting
19:56:25 (49562): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:00:34 (49752): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:12:14 (49945): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:17:03 (50714): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:17:04 (50714): No heartbeat from core client for 30 sec - exiting
21:17:05 (50714): No heartbeat from core client for 30 sec - exiting
21:17:06 (50714): No heartbeat from core client for 30 sec - exiting
21:17:07 (50714): No heartbeat from core client for 30 sec - exiting
21:17:08 (50714): No heartbeat from core client for 30 sec - exiting
21:17:09 (50714): No heartbeat from core client for 30 sec - exiting
21:17:10 (50714): No heartbeat from core client for 30 sec - exiting
21:17:11 (50714): No heartbeat from core client for 30 sec - exiting
21:17:12 (50714): No heartbeat from core client for 30 sec - exiting
21:17:13 (50714): No heartbeat from core client for 30 sec - exiting
21:17:14 (50714): No heartbeat from core client for 30 sec - exiting
21:17:15 (50714): No heartbeat from core client for 30 sec - exiting
21:17:16 (50714): No heartbeat from core client for 30 sec - exiting
21:17:17 (50714): No heartbeat from core client for 30 sec - exiting
21:17:18 (50714): No heartbeat from core client for 30 sec - exiting
03:14:02 (50899): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
03:18:27 (54317): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
03:18:28 (54317): No heartbeat from core client for 30 sec - exiting
03:18:29 (54317): No heartbeat from core client for 30 sec - exiting
03:37:01 (54500): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
03:58:24 (54879): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
04:03:00 (55290): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
04:03:01 (55290): No heartbeat from core client for 30 sec - exiting
04:03:02 (55290): No heartbeat from core client for 30 sec - exiting
04:03:03 (55290): No heartbeat from core client for 30 sec - exiting
04:07:30 (55477): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
04:07:31 (55477): No heartbeat from core client for 30 sec - exiting
04:07:32 (55477): No heartbeat from core client for 30 sec - exiting
04:07:33 (55477): No heartbeat from core client for 30 sec - exiting
04:07:34 (55477): No heartbeat from core client for 30 sec - exiting
04:07:35 (55477): No heartbeat from core client for 30 sec - exiting
04:07:36 (55477): No heartbeat from core client for 30 sec - exiting
04:07:37 (55477): No heartbeat from core client for 30 sec - exiting
04:11:40 (55657): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
04:11:41 (55657): No heartbeat from core client for 30 sec - exiting
04:11:42 (55657): No heartbeat from core client for 30 sec - exiting
04:11:43 (55657): No heartbeat from core client for 30 sec - exiting
04:11:44 (55657): No heartbeat from core client for 30 sec - exiting
06:42:20 (55837): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
06:45:51 (57314): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:12:06 (57481): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
11:29:14 (59264): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
11:33:43 (60663): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
11:38:10 (60828): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
11:38:11 (60828): No heartbeat from core client for 30 sec - exiting
11:38:12 (60828): No heartbeat from core client for 30 sec - exiting
11:38:13 (60828): No heartbeat from core client for 30 sec - exiting
11:38:14 (60828): No heartbeat from core client for 30 sec - exiting
11:38:15 (60828): No heartbeat from core client for 30 sec - exiting
11:38:16 (60828): No heartbeat from core client for 30 sec - exiting
11:38:17 (60828): No heartbeat from core client for 30 sec - exiting
11:38:18 (60828): No heartbeat from core client for 30 sec - exiting
11:38:19 (60828): No heartbeat from core client for 30 sec - exiting
11:38:20 (60828): No heartbeat from core client for 30 sec - exiting
11:38:21 (60828): No heartbeat from core client for 30 sec - exiting
11:38:22 (60828): No heartbeat from core client for 30 sec - exiting
11:38:23 (60828): No heartbeat from core client for 30 sec - exiting
11:38:24 (60828): No heartbeat from core client for 30 sec - exiting
11:38:25 (60828): No heartbeat from core client for 30 sec - exiting
11:38:26 (60828): No heartbeat from core client for 30 sec - exiting
SIGABRT: abort called
Stack trace (9 frames):
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f]
[0xf775d400]
[0xf775d425]
/lib/i386-linux-gnu/libc.so.6(gsignal+0x4f)[0xf757a1df]
/lib/i386-linux-gnu/libc.so.6(abort+0x175)[0xf757d825]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8]
/lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0xf75654d3]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=61002, iMonCtr=1
Model crash detected, will try to restart...
SIGABRT: abort called
Stack trace (9 frames):
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f]
[0xf77d3400]
[0xf77d3425]
/lib/i386-linux-gnu/libc.so.6(gsignal+0x4f)[0xf75f01df]
/lib/i386-linux-gnu/libc.so.6(abort+0x175)[0xf75f3825]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8]
/lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0xf75db4d3]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=61002, iMonCtr=1
Model crash detected, will try to restart...
SIGABRT: abort called
Stack trace (9 frames):
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f]
[0xf775c400]
[0xf775c425]
/lib/i386-linux-gnu/libc.so.6(gsignal+0x4f)[0xf75791df]
/lib/i386-linux-gnu/libc.so.6(abort+0x175)[0xf757c825]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8]
/lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0xf75644d3]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=61002, iMonCtr=1
Model crash detected, will try to restart...
SIGABRT: abort called
Stack trace (9 frames):
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f]
[0xf7723400]
[0xf7723425]
/lib/i386-linux-gnu/libc.so.6(gsignal+0x4f)[0xf75401df]
/lib/i386-linux-gnu/libc.so.6(abort+0x175)[0xf7543825]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8]
/lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0xf752b4d3]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=61002, iMonCtr=1
Model crash detected, will try to restart...
SIGABRT: abort called
Stack trace (9 frames):
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f]
[0xf7793400]
[0xf7793425]
/lib/i386-linux-gnu/libc.so.6(gsignal+0x4f)[0xf75b01df]
/lib/i386-linux-gnu/libc.so.6(abort+0x175)[0xf75b3825]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8]
/lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0xf759b4d3]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=61002, iMonCtr=1
Model crash detected, will try to restart...
SIGABRT: abort called
Stack trace (9 frames):
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f]
[0xf7755400]
[0xf7755425]
/lib/i386-linux-gnu/libc.so.6(gsignal+0x4f)[0xf75721df]
/lib/i386-linux-gnu/libc.so.6(abort+0x175)[0xf7575825]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8]
/lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0xf755d4d3]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=61002, iMonCtr=1
Model crash detected, will try to restart...
Sorry, too many model crashes! :-(
Called boinc_finish

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
31 May 2013 02:28:49 1282401 15780192 hadcm3n_39aj_1980_40_008367030_0 544,320 1,466,547 2.6943
30 May 2013 04:57:43 1282401 15780192 hadcm3n_39aj_1980_40_008367030_0 518,400 1,401,351 2.7032
29 May 2013 09:38:13 1282401 15780192 hadcm3n_39aj_1980_40_008367030_0 492,480 1,334,268 2.7093
28 May 2013 15:01:01 1282401 15780192 hadcm3n_39aj_1980_40_008367030_0 466,560 1,267,961 2.7177
27 May 2013 19:06:13 1282401 15780192 hadcm3n_39aj_1980_40_008367030_0 440,640 1,200,201 2.7238
26 May 2013 21:54:45 1282401 15780192 hadcm3n_39aj_1980_40_008367030_0 414,720 1,129,731 2.7241
26 May 2013 01:43:05 1282401 15780192 hadcm3n_39aj_1980_40_008367030_0 388,800 1,057,020 2.7187
25 May 2013 03:20:24 1282401 15780192 hadcm3n_39aj_1980_40_008367030_0 362,880 979,935 2.7004
24 May 2013 04:51:33 1282401 15780192 hadcm3n_39aj_1980_40_008367030_0 336,960 903,091 2.6801
23 May 2013 07:37:39 1282401 15780192 hadcm3n_39aj_1980_40_008367030_0 311,040 829,294 2.6662
22 May 2013 11:09:11 1282401 15780192 hadcm3n_39aj_1980_40_008367030_0 285,120 759,263 2.6630
21 May 2013 15:50:55 1282401 15780192 hadcm3n_39aj_1980_40_008367030_0 259,200 689,087 2.6585
20 May 2013 18:39:26 1282401 15780192 hadcm3n_39aj_1980_40_008367030_0 233,280 617,132 2.6455
19 May 2013 22:47:04 1282401 15780192 hadcm3n_39aj_1980_40_008367030_0 207,360 545,628 2.6313
19 May 2013 03:15:13 1282401 15780192 hadcm3n_39aj_1980_40_008367030_0 181,440 477,420 2.6313
18 May 2013 07:53:15 1282401 15780192 hadcm3n_39aj_1980_40_008367030_0 155,520 409,736 2.6346
17 May 2013 12:16:41 1282401 15780192 hadcm3n_39aj_1980_40_008367030_0 129,600 341,768 2.6371
16 May 2013 16:52:02 1282401 15780192 hadcm3n_39aj_1980_40_008367030_0 103,680 272,872 2.6319
15 May 2013 20:16:12 1282401 15780192 hadcm3n_39aj_1980_40_008367030_0 77,760 200,164 2.5741
14 May 2013 23:01:53 1282401 15780192 hadcm3n_39aj_1980_40_008367030_0 51,840 127,610 2.4616


©2024 climateprediction.net