climateprediction.net home page
Task 13391800

Task 13391800

Name hadcm3n_o28j_1940_40_007447238_2
Workunit 7644741
Created 16 Sep 2011, 11:31:39 UTC
Sent 16 Sep 2011, 11:39:43 UTC
Report deadline 16 Dec 2011, 19:06:54 UTC
Received 10 Oct 2011, 17:42:24 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 22 (0x00000016) Unknown error code
Computer ID 1227722
Run time 21 days 10 hours 28 min 48 sec
CPU time 17 days 11 hours 5 min 42 sec
Validate state Invalid
Credit 12,130.56
Device peak FLOPS 2.46 GFLOPS
Application version UK Met Office Coupled Model Full Resolution Ocean v6.07
i686-pc-linux-gnu
Stderr
<core_client_version>6.12.34</core_client_version>
<![CDATA[
<message>
process exited with code 22 (0x16, -234)
</message>
<stderr_txt>
Signal 2 received, exiting...
Called boinc_finish
Signal 2 received, exiting...
Called boinc_finish
Suspended CPDN Monitor - Suspend request from BOINC...
Signal 2 received, exiting...
Called boinc_finish
Signal 2 received, exiting...
Called boinc_finish

Model crashed: P_TH_ADJ : NEGATIVE PRESSURE VALUE CREATED.                                                                                                                                                                                                                     tmp/pipe_dummy                                                                  2048    
forrtl: severe (17): syntax error in NAMELIST input, unit 57, file /data/BOINC/projects/climateprediction.net/hadcm3n_o28j_1940_40_007447238/jobs/specsw, line 37, position 55
Image              PC        Routine            Line        Source             
hadcm3n_um_6.07_i  0848EB7D  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  0848D975  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  0845F3CF  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  0841F90D  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  0841F257  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  0843AE11  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  082D62B2  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  0819D0E4  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  081912A0  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  08391957  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  0838F8B7  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  0839BDF8  Unknown               Unknown  Unknown
libc.so.6          F7588E26  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  0804CB11  Unknown               Unknown  Unknown
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=19118, iMonCtr=1
Model crash detected, will try to restart...
forrtl: severe (17): syntax error in NAMELIST input, unit 80, file /data/BOINC/projects/climateprediction.net/hadcm3n_o28j_1940_40_007447238/jobs/speclw, line 39, position 55
Image              PC        Routine            Line        Source             
hadcm3n_um_6.07_i  0848EB7D  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  0848D975  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  0845F3CF  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  0841F90D  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  0841F257  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  0843AE11  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  082D982B  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  0819D1A1  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  081912A0  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  08391957  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  0838F8B7  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  0839BDF8  Unknown               Unknown  Unknown
libc.so.6          F7559E26  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  0804CB11  Unknown               Unknown  Unknown
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=19118, iMonCtr=1
Model crash detected, will try to restart...

Model crashed: ATM_DYN : INVALID THETA DETECTED.                                                                                                                                                                                                                               tmp/pipe_dummy                                                                  2048    

Model crashed: ATM_DYN : INVALID THETA DETECTED.                                                                                                                                                                                                                               tmp/pipe_dummy                                                                  2048    
Signal 2 received, exiting...
Called boinc_finish
Signal 2 received, exiting...
Called boinc_finish
Signal 2 received, exiting...
Called boinc_finish
Signal 2 received, exiting...
Called boinc_finish
Suspended CPDN Monitor - Suspend request from BOINC...
Signal 2 received, exiting...
Called boinc_finish
Signal 2 received, exiting...
Called boinc_finish
Suspended CPDN Monitor - Suspend request from BOINC...
Signal 2 received, exiting...
Called boinc_finish
Signal 2 received, exiting...
Called boinc_finish
Signal 2 received, exiting...
Called boinc_finish
Signal 2 received, exiting...
Called boinc_finish
Signal 2 received, exiting...
Called boinc_finish
Signal 2 received, exiting...
Called boinc_finish
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Signal 2 received, exiting...
Called boinc_finish
Signal 2 received, exiting...
Called boinc_finish
Signal 2 received, exiting...
Called boinc_finish
Signal 2 received, exiting...
Called boinc_finish

Model crashed: ATM_DYN : INVALID THETA DETECTED.                                                                                                                                                                                                                               tmp/pipe_dummy                                                                  2048    
SIGSEGV: segmentation violation
Stack trace (14 frames):
/data/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f]
[0xf77a8400]
/data/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8155092]
/data/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8159b01]
/data/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x81518bf]
/data/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x815a74a]
/data/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8153291]
/data/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x807f0e3]
/data/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x837e9f4]
/data/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839982e]
/data/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f8b7]
/data/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8]
/lib32/libc.so.6(__libc_start_main+0xe6)[0xf75e02c6]
/data/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x804cb11]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=9703, iMonCtr=1
Model crash detected, will try to restart...

Model crashed: ATM_DYN : INVALID THETA DETECTED.                                                                                                                                                                                                                               tmp/pipe_dummy                                                                  2048    

Model crashed: ATM_DYN : INVALID THETA DETECTED.                                                                                                                                                                                                                               tmp/pipe_dummy                                                                  2048    

Model crashed: ATM_DYN : INVALID THETA DETECTED.                                                                                                                                                                                                                               tmp/pipe_dummy                                                                  2048    

Model crashed: ATM_DYN : INVALID THETA DETECTED.                                                                                                                                                                                                                               tmp/pipe_dummy                                                                  2048    
Sorry, too many model crashes! :-(
Called boinc_finish

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 Oct 2011 23:50:44 1165864 13391800 hadcm3n_o28j_1940_40_007447238_2 1,010,880 1,594,023 1.5769
09 Oct 2011 11:39:12 1165864 13391800 hadcm3n_o28j_1940_40_007447238_2 984,960 1,551,565 1.5753
08 Oct 2011 20:35:19 1165864 13391800 hadcm3n_o28j_1940_40_007447238_2 959,040 1,508,744 1.5732
08 Oct 2011 07:06:08 1165864 13391800 hadcm3n_o28j_1940_40_007447238_2 933,120 1,465,950 1.5710
07 Oct 2011 14:43:30 1165864 13391800 hadcm3n_o28j_1940_40_007447238_2 907,200 1,423,352 1.5690
07 Oct 2011 00:00:50 1165864 13391800 hadcm3n_o28j_1940_40_007447238_2 881,280 1,380,915 1.5669
06 Oct 2011 08:27:55 1165864 13391800 hadcm3n_o28j_1940_40_007447238_2 855,360 1,337,754 1.5640
05 Oct 2011 18:43:53 1165864 13391800 hadcm3n_o28j_1940_40_007447238_2 829,440 1,294,406 1.5606
05 Oct 2011 07:28:33 1165864 13391800 hadcm3n_o28j_1940_40_007447238_2 803,520 1,252,105 1.5583
04 Oct 2011 17:31:52 1165864 13391800 hadcm3n_o28j_1940_40_007447238_2 777,600 1,209,201 1.5550
04 Oct 2011 03:52:09 1165864 13391800 hadcm3n_o28j_1940_40_007447238_2 751,680 1,165,838 1.5510
03 Oct 2011 11:40:28 1165864 13391800 hadcm3n_o28j_1940_40_007447238_2 725,760 1,122,989 1.5473
02 Oct 2011 22:27:55 1165864 13391800 hadcm3n_o28j_1940_40_007447238_2 699,840 1,080,079 1.5433
02 Oct 2011 09:51:34 1165864 13391800 hadcm3n_o28j_1940_40_007447238_2 673,920 1,038,027 1.5403
01 Oct 2011 18:58:26 1165864 13391800 hadcm3n_o28j_1940_40_007447238_2 648,000 995,637 1.5365
01 Oct 2011 05:45:47 1165864 13391800 hadcm3n_o28j_1940_40_007447238_2 622,080 953,059 1.5321
30 Sep 2011 16:28:08 1165864 13391800 hadcm3n_o28j_1940_40_007447238_2 596,160 910,062 1.5265
30 Sep 2011 04:27:08 1165864 13391800 hadcm3n_o28j_1940_40_007447238_2 570,240 868,122 1.5224
29 Sep 2011 15:01:20 1165864 13391800 hadcm3n_o28j_1940_40_007447238_2 544,320 825,020 1.5157
29 Sep 2011 02:22:43 1165864 13391800 hadcm3n_o28j_1940_40_007447238_2 518,400 782,018 1.5085


©2024 climateprediction.net