climateprediction.net home page
Task 15775097

Task 15775097

Name hadcm3n_48d5_1980_40_008365286_0
Workunit 8516145
Created 11 May 2013, 0:23:00 UTC
Sent 11 May 2013, 0:25:15 UTC
Report deadline 10 Aug 2013, 7:52:26 UTC
Received 29 May 2013, 15:47:23 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 193 (0x000000C1) EXIT_SIGNAL
Computer ID 1179592
Run time 18 days 10 hours 33 min 52 sec
CPU time 17 days 5 hours 48 min 28 sec
Validate state Invalid
Credit 6,220.80
Device peak FLOPS 1.77 GFLOPS
Application version UK Met Office Coupled Model Full Resolution Ocean v6.07
i686-pc-linux-gnu
Stderr 
<core_client_version>6.10.58</core_client_version>
<![CDATA[
<message>
process exited with code 193 (0xc1, -63)
</message>
<stderr_txt>
Suspended CPDN Monitor - Suspend request from BOINC...
20:08:12 (27611): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:39:57 (29946): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
22:13:32 (33221): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
22:14:01 (33221): No heartbeat from core client for 30 sec - exiting
22:14:02 (33221): No heartbeat from core client for 30 sec - exiting
22:14:03 (33221): No heartbeat from core client for 30 sec - exiting
22:14:04 (33221): No heartbeat from core client for 30 sec - exiting
22:14:05 (33221): No heartbeat from core client for 30 sec - exiting
22:14:06 (33221): No heartbeat from core client for 30 sec - exiting
22:14:07 (33221): No heartbeat from core client for 30 sec - exiting
23:00:23 (38129): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
00:57:38 (38381): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
02:02:27 (38681): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
03:03:15 (38938): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
06:03:27 (39188): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
07:06:18 (39547): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
10:10:56 (39803): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
10:21:29 (40302): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
11:11:43 (40512): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:13:47 (40754): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
17:51:46 (41151): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
18:16:39 (41622): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:18:35 (41852): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:18:54 (42210): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
04:18:20 (42485): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
04:25:47 (43088): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
05:13:26 (43289): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
05:26:28 (43525): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
05:52:26 (43736): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
06:28:05 (43954): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:39:26 (44188): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:47:46 (44615): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:30:21 (44819): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
10:31:00 (45055): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
11:31:41 (45307): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
13:06:44 (45577): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
13:33:05 (45888): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:33:46 (46119): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:39:16 (46391): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
03:43:48 (46596): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
23:16:29 (47652): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
10:26:21 (2598): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
20:16:55 (3329): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
02:42:39 (6253): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
06:52:43 (6581): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:47:30 (6849): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:52:38 (7213): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:47:51 (7416): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
19:01:34 (7646): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
19:49:15 (8183): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
19:55:26 (8416): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:02:26 (8619): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:08:22 (8819): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:03:04 (9019): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
22:04:01 (9252): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
23:38:53 (9484): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
23:45:29 (9725): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
01:40:27 (9923): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
07:21:28 (10156): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
07:45:48 (10460): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
07:56:08 (10702): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:46:30 (10988): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:47:32 (11185): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:47:12 (11383): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
10:47:58 (11597): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
10:52:36 (11831): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
11:49:23 (12038): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
13:50:17 (12272): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
15:52:27 (12549): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
*** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x093b1a10 ***
======= Backtrace: =========
/lib32/libc.so.6(+0x6bff1)[0xf755fff1]
/lib32/libc.so.6(+0x6d880)[0xf7561880]
/lib32/libc.so.6(cfree+0x6d)[0xf756492d]
/usr/lib32/libstdc++.so.6(_ZdlPv+0x21)[0xf773b7b1]
/usr/lib32/libstdc++.so.6(_ZdaPv+0x1d)[0xf773b80d]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8053e8e]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8057bc4]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x804f232]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8050491]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a]
/lib32/libc.so.6(__libc_start_main+0xe7)[0xf750ace7]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51]
======= Memory map: ========
08048000-080e3000 r-xp 00000000 08:02 3514599                            /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu
080e3000-080e4000 rw-p 0009b000 08:02 3514599                            /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu
080e4000-0813b000 rw-p 00000000 00:00 0 
0935c000-093c1000 rw-p 00000000 00:00 0                                  [heap]
f6f00000-f6f21000 rw-p 00000000 00:00 0 
f6f21000-f7000000 ---p 00000000 00:00 0 
f7076000-f74f1000 rw-s 00000000 08:02 6316145                            /var/lib/boinc-client/slots/37/132235
f74f1000-f74f4000 rw-p 00000000 00:00 0 
f74f4000-f7648000 r-xp 00000000 08:02 9879703                            /lib32/libc-2.12.1.so
f7648000-f7649000 ---p 00154000 08:02 9879703                            /lib32/libc-2.12.1.so
f7649000-f764b000 r--p 00154000 08:02 9879703                            /lib32/libc-2.12.1.so
f764b000-f764c000 rw-p 00156000 08:02 9879703                            /lib32/libc-2.12.1.so
f764c000-f764f000 rw-p 00000000 00:00 0 
f764f000-f7669000 r-xp 00000000 08:02 13436533                           /usr/lib32/libgcc_s.so.1
f7669000-f766a000 r--p 00019000 08:02 13436533                           /usr/lib32/libgcc_s.so.1
f766a000-f766b000 rw-p 0001a000 08:02 13436533                           /usr/lib32/libgcc_s.so.1
f766b000-f768f000 r-xp 00000000 08:02 9879707                            /lib32/libm-2.12.1.so
f768f000-f7690000 r--p 00023000 08:02 9879707                            /lib32/libm-2.12.1.so
f7690000-f7691000 rw-p 00024000 08:02 9879707                            /lib32/libm-2.12.1.so
f7691000-f7770000 r-xp 00000000 08:02 13436540                           /usr/lib32/libstdc++.so.6.0.14
f7770000-f7774000 r--p 000de000 08:02 13436540                           /usr/lib32/libstdc++.so.6.0.14
f7774000-f7775000 rw-p 000e2000 08:02 13436540                           /usr/lib32/libstdc++.so.6.0.14
f7775000-f777c000 rw-p 00000000 00:00 0 
f777c000-f777e000 r-xp 00000000 08:02 9879706                            /lib32/libdl-2.12.1.so
f777e000-f777f000 r--p 00001000 08:02 9879706                            /lib32/libdl-2.12.1.so
f777f000-f7780000 rw-p 00002000 08:02 9879706                            /lib32/libdl-2.12.1.so
f7780000-f7781000 rw-p 00000000 00:00 0 
f7781000-f7796000 r-xp 00000000 08:02 9879717                            /lib32/libpthread-2.12.1.so
f7796000-f7797000 r--p 00014000 08:02 9879717                            /lib32/libpthread-2.12.1.so
f7797000-f7798000 rw-p 00015000 08:02 9879717                            /lib32/libpthread-2.12.1.so
f7798000-f779a000 rw-p 00000000 00:00 0 
f77ab000-f77ac000 rw-p 00000000 00:00 0 
f77ac000-f77ad000 ---p 00000000 00:00 0 
f77ad000-f77b0000 rw-p 00000000 00:00 0 
f77b0000-f77b2000 rw-s 00000000 08:02 6316054                            /var/lib/boinc-client/slots/37/boinc_mmap_file
f77b2000-f77b4000 rw-p 00000000 00:00 0 
f77b4000-f77b5000 r-xp 00000000 00:00 0                                  [vdso]
f77b5000-f77d1000 r-xp 00000000 08:02 9879700                            /lib32/ld-2.12.1.so
f77d1000-f77d2000 r--p 0001b000 08:02 9879700                            /lib32/ld-2.12.1.so
f77d2000-f77d3000 rw-p 0001c000 08:02 9879700                            /lib32/ld-2.12.1.so
ff94d000-ff9bd000 rw-p 00000000 00:00 0                                  [stack]
SIGABRT: abort called
Stack trace (19 frames):
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x80b80df]
[0xf77b4400]
[0xf77b4425]
/lib32/libc.so.6(gsignal+0x51)[0xf751ea01]
/lib32/libc.so.6(abort+0x182)[0xf7521e42]
/lib32/libc.so.6(+0x61f15)[0xf7555f15]
/lib32/libc.so.6(+0x6bff1)[0xf755fff1]
/lib32/libc.so.6(+0x6d880)[0xf7561880]
/lib32/libc.so.6(cfree+0x6d)[0xf756492d]
/usr/lib32/libstdc++.so.6(_ZdlPv+0x21)[0xf773b7b1]
/usr/lib32/libstdc++.so.6(_ZdaPv+0x1d)[0xf773b80d]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8053e8e]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8057bc4]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x804f232]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8050491]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a]
/lib32/libc.so.6(__libc_start_main+0xe7)[0xf750ace7]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51]

Exiting...

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
29 May 2013 15:50:24 1179592 15775097 hadcm3n_48d5_1980_40_008365286_0 518,400 1,489,004 2.8723
28 May 2013 16:57:09 1179592 15775097 hadcm3n_48d5_1980_40_008365286_0 492,480 1,411,604 2.8663
27 May 2013 18:16:22 1179592 15775097 hadcm3n_48d5_1980_40_008365286_0 466,560 1,334,072 2.8594
26 May 2013 23:00:34 1179592 15775097 hadcm3n_48d5_1980_40_008365286_0 440,640 1,262,349 2.8648
26 May 2013 02:43:23 1179592 15775097 hadcm3n_48d5_1980_40_008365286_0 414,720 1,190,744 2.8712
25 May 2013 06:46:25 1179592 15775097 hadcm3n_48d5_1980_40_008365286_0 388,800 1,119,892 2.8804
24 May 2013 10:23:25 1179592 15775097 hadcm3n_48d5_1980_40_008365286_0 362,880 1,048,619 2.8897
23 May 2013 16:06:19 1179592 15775097 hadcm3n_48d5_1980_40_008365286_0 336,960 983,059 2.9174
22 May 2013 21:53:37 1179592 15775097 hadcm3n_48d5_1980_40_008365286_0 311,040 915,002 2.9418
22 May 2013 02:41:10 1179592 15775097 hadcm3n_48d5_1980_40_008365286_0 285,120 847,702 2.9731
20 May 2013 22:18:50 1179592 15775097 hadcm3n_48d5_1980_40_008365286_0 259,200 749,725 2.8925
20 May 2013 01:58:32 1179592 15775097 hadcm3n_48d5_1980_40_008365286_0 233,280 678,689 2.9093
19 May 2013 06:11:19 1179592 15775097 hadcm3n_48d5_1980_40_008365286_0 207,360 605,721 2.9211
18 May 2013 09:39:01 1179592 15775097 hadcm3n_48d5_1980_40_008365286_0 181,440 532,687 2.9359
17 May 2013 12:51:59 1179592 15775097 hadcm3n_48d5_1980_40_008365286_0 155,520 460,402 2.9604
16 May 2013 16:36:50 1179592 15775097 hadcm3n_48d5_1980_40_008365286_0 129,600 386,620 2.9832
15 May 2013 16:33:25 1179592 15775097 hadcm3n_48d5_1980_40_008365286_0 103,680 302,765 2.9202
14 May 2013 17:17:30 1179592 15775097 hadcm3n_48d5_1980_40_008365286_0 77,760 217,835 2.8014
13 May 2013 18:04:01 1179592 15775097 hadcm3n_48d5_1980_40_008365286_0 51,840 133,316 2.5717
12 May 2013 22:43:45 1179592 15775097 hadcm3n_48d5_1980_40_008365286_0 25,920 66,139 2.5517


©2024 climateprediction.net