climateprediction.net home page
Task 13012832

Task 13012832

Name hadcm3n_t16x_1940_40_007311095_1
Workunit 7508525
Created 27 Jun 2011, 21:44:16 UTC
Sent 27 Jun 2011, 21:44:31 UTC
Report deadline 27 Sep 2011, 5:11:42 UTC
Received 3 Aug 2011, 18:57:14 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 22 (0x00000016) Unknown error code
Computer ID 1137646
Run time 20 days 9 hours 43 min 2 sec
CPU time 19 days 8 hours 21 min 10 sec
Validate state Invalid
Credit 9,953.28
Device peak FLOPS 2.43 GFLOPS
Application version UK Met Office Coupled Model Full Resolution Ocean v6.07
i686-pc-linux-gnu
Stderr 
<core_client_version>6.10.17</core_client_version>
<![CDATA[
<message>
process exited with code 22 (0x16, -234)
</message>
<stderr_txt>
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
21:12:01 (1978): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
22:22:36 (471): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
22:57:50 (10063): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
00:08:42 (15041): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
05:24:25 (24909): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:12:08 (6565): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
12:29:44 (31059): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
13:11:55 (3806): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:25:01 (9710): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
16:25:05 (20390): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
17:05:08 (5182): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
18:25:48 (10990): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
19:53:50 (22521): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:35:41 (2643): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:20:26 (8576): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
22:03:38 (14906): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
22:25:12 (21034): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
23:31:10 (24092): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
00:13:28 (1127): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
00:59:32 (6915): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
01:46:14 (13545): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
02:27:19 (19921): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
03:11:39 (25517): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
03:37:17 (31760): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
04:37:15 (3005): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
05:27:59 (11275): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
05:37:19 (18086): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
07:04:31 (19378): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
07:04:47 (19378): No heartbeat from core client for 30 sec - exiting
07:04:48 (19378): No heartbeat from core client for 30 sec - exiting
07:04:49 (19378): No heartbeat from core client for 30 sec - exiting
07:54:17 (30481): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
07:54:55 (4736): No heartbeat from core client for 30 sec - exiting
07:54:56 (4736): No heartbeat from core client for 30 sec - exiting
07:54:57 (4736): No heartbeat from core client for 30 sec - exiting
07:54:59 (4736): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:19:18 (5301): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:27:12 (7997): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=9015, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=9015, iMonCtr=1
Model crash detected, will try to restart...
08:49:14 (9015): No heartbeat from core client for 30 sec - exiting
08:49:47 (9015): No heartbeat from core client for 30 sec - exiting
08:49:48 (9015): No heartbeat from core client for 30 sec - exiting
08:49:49 (9015): No heartbeat from core client for 30 sec - exiting
08:49:50 (9015): No heartbeat from core client for 30 sec - exiting
08:49:51 (9015): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:49:52 (9015): No heartbeat from core client for 30 sec - exiting
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=11898, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=11898, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=11898, iMonCtr=1
Model crash detected, will try to restart...
09:41:34 (11898): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
12:40:35 (17778): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
13:47:11 (11574): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
14:21:50 (21400): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
16:13:04 (26424): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
17:27:21 (10029): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
18:02:51 (20886): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
19:17:41 (25798): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
19:53:42 (4278): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
SIGABRT: abort called
Stack trace (10 frames):
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f]
[0xf771f400]
[0xf771f430]
/lib32/libc.so.6(gsignal+0x51)[0xf7586921]
/lib32/libc.so.6(abort+0x182)[0xf7589d52]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8]
/lib32/libc.so.6(__libc_start_main+0xe6)[0xf7572bd6]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x804cb11]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=9462, iMonCtr=1
Model crash detected, will try to restart...
SIGABRT: abort called
Stack trace (10 frames):
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f]
[0xf7773400]
[0xf7773430]
/lib32/libc.so.6(gsignal+0x51)[0xf75da921]
/lib32/libc.so.6(abort+0x182)[0xf75ddd52]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8]
/lib32/libc.so.6(__libc_start_main+0xe6)[0xf75c6bd6]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x804cb11]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=9462, iMonCtr=1
Model crash detected, will try to restart...
SIGABRT: abort called
Stack trace (10 frames):
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f]
[0xf7720400]
[0xf7720430]
/lib32/libc.so.6(gsignal+0x51)[0xf7587921]
/lib32/libc.so.6(abort+0x182)[0xf758ad52]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8]
/lib32/libc.so.6(__libc_start_main+0xe6)[0xf7573bd6]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x804cb11]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=9462, iMonCtr=1
Model crash detected, will try to restart...
SIGABRT: abort called
Stack trace (10 frames):
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f]
[0xf77b1400]
[0xf77b1430]
/lib32/libc.so.6(gsignal+0x51)[0xf7618921]
/lib32/libc.so.6(abort+0x182)[0xf761bd52]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8]
/lib32/libc.so.6(__libc_start_main+0xe6)[0xf7604bd6]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x804cb11]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=9462, iMonCtr=1
Model crash detected, will try to restart...
SIGABRT: abort called
Stack trace (10 frames):
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f]
[0xf7753400]
[0xf7753430]
/lib32/libc.so.6(gsignal+0x51)[0xf75ba921]
/lib32/libc.so.6(abort+0x182)[0xf75bdd52]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8]
/lib32/libc.so.6(__libc_start_main+0xe6)[0xf75a6bd6]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x804cb11]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=9462, iMonCtr=1
Model crash detected, will try to restart...
SIGABRT: abort called
Stack trace (10 frames):
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f]
[0xf7748400]
[0xf7748430]
/lib32/libc.so.6(gsignal+0x51)[0xf75af921]
/lib32/libc.so.6(abort+0x182)[0xf75b2d52]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8]
/lib32/libc.so.6(__libc_start_main+0xe6)[0xf759bbd6]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x804cb11]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=9462, iMonCtr=1
Model crash detected, will try to restart...
Sorry, too many model crashes! :-(
Called boinc_finish

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
02 Aug 2011 22:37:58 1137646 13012832 hadcm3n_t16x_1940_40_007311095_1 829,440 1,636,116 1.9726
02 Aug 2011 00:19:08 1137646 13012832 hadcm3n_t16x_1940_40_007311095_1 803,520 1,585,048 1.9726
01 Aug 2011 07:38:31 1137646 13012832 hadcm3n_t16x_1940_40_007311095_1 777,600 1,533,124 1.9716
31 Jul 2011 17:03:19 1137646 13012832 hadcm3n_t16x_1940_40_007311095_1 751,680 1,480,609 1.9697
31 Jul 2011 01:44:48 1137646 13012832 hadcm3n_t16x_1940_40_007311095_1 725,760 1,428,283 1.9680
30 Jul 2011 10:54:17 1137646 13012832 hadcm3n_t16x_1940_40_007311095_1 699,840 1,375,742 1.9658
29 Jul 2011 20:02:32 1137646 13012832 hadcm3n_t16x_1940_40_007311095_1 673,920 1,323,370 1.9637
25 Jul 2011 19:01:17 1137646 13012832 hadcm3n_t16x_1940_40_007311095_1 648,000 1,271,040 1.9615
25 Jul 2011 19:01:17 1137646 13012832 hadcm3n_t16x_1940_40_007311095_1 622,080 1,219,976 1.9611
25 Jul 2011 19:01:17 1137646 13012832 hadcm3n_t16x_1940_40_007311095_1 596,160 1,169,299 1.9614
11 Jul 2011 02:15:53 1137646 13012832 hadcm3n_t16x_1940_40_007311095_1 570,240 1,118,603 1.9616
10 Jul 2011 11:53:02 1137646 13012832 hadcm3n_t16x_1940_40_007311095_1 544,320 1,067,840 1.9618
09 Jul 2011 21:32:21 1137646 13012832 hadcm3n_t16x_1940_40_007311095_1 518,400 1,017,125 1.9620
09 Jul 2011 07:14:53 1137646 13012832 hadcm3n_t16x_1940_40_007311095_1 492,480 966,383 1.9623
08 Jul 2011 17:51:48 1137646 13012832 hadcm3n_t16x_1940_40_007311095_1 466,560 915,610 1.9625
08 Jul 2011 02:16:25 1137646 13012832 hadcm3n_t16x_1940_40_007311095_1 440,640 864,254 1.9614
07 Jul 2011 17:56:13 1137646 13012832 hadcm3n_t16x_1940_40_007311095_1 414,720 813,080 1.9606
07 Jul 2011 15:39:16 1137646 13012832 hadcm3n_t16x_1940_40_007311095_1 388,800 762,058 1.9600
07 Jul 2011 15:37:33 1137646 13012832 hadcm3n_t16x_1940_40_007311095_1 362,880 711,264 1.9601
05 Jul 2011 17:17:25 1137646 13012832 hadcm3n_t16x_1940_40_007311095_1 336,960 660,522 1.9602


©2024 cpdn.org