climateprediction.net home page
Task 14739363

Task 14739363

Name hadcm3n_o5a3_1980_40_007998526_2
Workunit 8153640
Created 24 May 2012, 22:57:11 UTC
Sent 24 May 2012, 23:06:13 UTC
Report deadline 24 Aug 2012, 6:33:24 UTC
Received 7 Jun 2012, 4:06:05 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 22 (0x00000016) Unknown error code
Computer ID 1050454
Run time 12 days 22 hours 3 min 30 sec
CPU time 3 days 7 hours 12 min 5 sec
Validate state Invalid
Credit 7,153.92
Device peak FLOPS 2.56 GFLOPS
Application version UK Met Office Coupled Model Full Resolution Ocean v6.07
i686-pc-linux-gnu
Stderr 
<core_client_version>6.10.58</core_client_version>
<![CDATA[
<message>
process exited with code 22 (0x16, -234)
</message>
<stderr_txt>

Model crashed: ATM_DYN : INVALID THETA DETECTED.                                                                                                                                                                                                                               tmp/pipe_dummy                                                                  2048    
Suspended CPDN Monitor - Suspend request from BOINC...
00:11:33 (23867): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
18:31:40 (14683): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
18:31:41 (14683): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - Suspend request from BOINC...
SIGSEGV: segmentation violation
Stack trace (13 frames):
/y1/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f]
[0xffffe400]
/y1/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8155092]
/y1/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8159b01]
/y1/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x81518bf]
/y1/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x815a74a]
/y1/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8153291]
/y1/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x807f0e3]
/y1/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x837e9f4]
/y1/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839982e]
/y1/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f8b7]
/y1/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8]
/lib/libc.so.6(__libc_start_main+0xfe)[0xf7603c0e]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4605, iMonCtr=1
Model crash detected, will try to restart...
SIGSEGV: segmentation violation
Stack trace (13 frames):
/y1/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f]
[0xffffe400]
/y1/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8155092]
/y1/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8159b01]
/y1/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x81518bf]
/y1/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x815a74a]
/y1/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8153291]
/y1/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x807f0e3]
/y1/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x837e9f4]
/y1/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839982e]
/y1/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f8b7]
/y1/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8]
/lib/libc.so.6(__libc_start_main+0xfe)[0xf7570c0e]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4605, iMonCtr=1
Model crash detected, will try to restart...
SIGSEGV: segmentation violation
Stack trace (13 frames):
/y1/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f]
[0xffffe400]
/y1/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8155092]
/y1/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8159b01]
/y1/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x81518bf]
/y1/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x815a74a]
/y1/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8153291]
/y1/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x807f0e3]
/y1/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x837e9f4]
/y1/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839982e]
/y1/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f8b7]
/y1/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8]
/lib/libc.so.6(__libc_start_main+0xfe)[0xf756cc0e]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4605, iMonCtr=1
Model crash detected, will try to restart...
SIGSEGV: segmentation violation
Stack trace (13 frames):
/y1/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f]
[0xffffe400]
/y1/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8155092]
/y1/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8159b01]
/y1/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x81518bf]
/y1/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x815a74a]
/y1/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8153291]
/y1/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x807f0e3]
/y1/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x837e9f4]
/y1/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839982e]
/y1/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f8b7]
/y1/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8]
/lib/libc.so.6(__libc_start_main+0xfe)[0xf758fc0e]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4605, iMonCtr=1
Model crash detected, will try to restart...
SIGSEGV: segmentation violation
Stack trace (13 frames):
/y1/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f]
[0xffffe400]
/y1/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8155092]
/y1/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8159b01]
/y1/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x81518bf]
/y1/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x815a74a]
/y1/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8153291]
/y1/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x807f0e3]
/y1/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x837e9f4]
/y1/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839982e]
/y1/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f8b7]
/y1/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8]
/lib/libc.so.6(__libc_start_main+0xfe)[0xf75c0c0e]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4605, iMonCtr=1
Model crash detected, will try to restart...
SIGSEGV: segmentation violation
Stack trace (13 frames):
/y1/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f]
[0xffffe400]
/y1/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8155092]
/y1/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8159b01]
/y1/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x81518bf]
/y1/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x815a74a]
/y1/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8153291]
/y1/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x807f0e3]
/y1/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x837e9f4]
/y1/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839982e]
/y1/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f8b7]
/y1/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8]
/lib/libc.so.6(__libc_start_main+0xfe)[0xf7617c0e]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4605, iMonCtr=1
Model crash detected, will try to restart...
Sorry, too many model crashes! :-(
Called boinc_finish

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
06 Jun 2012 21:30:38 1050454 14739363 hadcm3n_o5a3_1980_40_007998526_2 596,160 682,228 1.1444
06 Jun 2012 08:20:11 1050454 14739363 hadcm3n_o5a3_1980_40_007998526_2 570,240 636,864 1.1168
05 Jun 2012 19:06:39 1050454 14739363 hadcm3n_o5a3_1980_40_007998526_2 544,320 591,481 1.0866
05 Jun 2012 05:52:32 1050454 14739363 hadcm3n_o5a3_1980_40_007998526_2 518,400 546,127 1.0535
04 Jun 2012 16:41:15 1050454 14739363 hadcm3n_o5a3_1980_40_007998526_2 492,480 500,743 1.0168
04 Jun 2012 04:08:39 1050454 14739363 hadcm3n_o5a3_1980_40_007998526_2 466,560 455,432 0.9761
03 Jun 2012 14:09:41 1050454 14739363 hadcm3n_o5a3_1980_40_007998526_2 440,640 410,080 0.9306
03 Jun 2012 00:56:41 1050454 14739363 hadcm3n_o5a3_1980_40_007998526_2 414,720 364,764 0.8795
02 Jun 2012 11:41:42 1050454 14739363 hadcm3n_o5a3_1980_40_007998526_2 388,800 319,405 0.8215
01 Jun 2012 22:29:33 1050454 14739363 hadcm3n_o5a3_1980_40_007998526_2 362,880 274,120 0.7554
01 Jun 2012 09:18:18 1050454 14739363 hadcm3n_o5a3_1980_40_007998526_2 336,960 228,724 0.6788
31 May 2012 20:55:43 1050454 14739363 hadcm3n_o5a3_1980_40_007998526_2 311,040 183,342 0.5894
31 May 2012 06:47:36 1050454 14739363 hadcm3n_o5a3_1980_40_007998526_2 285,120 137,978 0.4839
30 May 2012 17:38:18 1050454 14739363 hadcm3n_o5a3_1980_40_007998526_2 259,200 92,677 0.3576
30 May 2012 04:43:12 1050454 14739363 hadcm3n_o5a3_1980_40_007998526_2 233,280 47,352 0.2030
29 May 2012 15:08:37 1050454 14739363 hadcm3n_o5a3_1980_40_007998526_2 207,360 2,150 0.0104
29 May 2012 02:00:44 1050454 14739363 hadcm3n_o5a3_1980_40_007998526_2 181,440 315,977 1.7415
28 May 2012 12:49:41 1050454 14739363 hadcm3n_o5a3_1980_40_007998526_2 155,520 270,825 1.7414
27 May 2012 23:41:07 1050454 14739363 hadcm3n_o5a3_1980_40_007998526_2 129,600 225,703 1.7415
27 May 2012 10:31:01 1050454 14739363 hadcm3n_o5a3_1980_40_007998526_2 103,680 180,549 1.7414


©2024 climateprediction.net