climateprediction.net home page
Task 16922271

Task 16922271

Name hadam3pm2_a989_1959_10_008658148_2
Workunit 8804622
Created 24 Aug 2014, 0:07:41 UTC
Sent 24 Aug 2014, 1:01:37 UTC
Report deadline 6 Aug 2015, 6:21:37 UTC
Received 3 Sep 2014, 22:29:22 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 22 (0x00000016) Unknown error code
Computer ID 1396975
Run time 4 days 2 hours 29 min 51 sec
CPU time 1 days 2 hours 27 min 39 sec
Validate state Invalid
Credit 4,908.40
Device peak FLOPS 3.96 GFLOPS
Application version UK Met Office HadAM3P (global only) with MOSES II landsurface scheme v7.03
i686-pc-linux-gnu
Stderr 
<core_client_version>7.0.65</core_client_version>
<![CDATA[
<message>
process exited with code 22 (0x16, -234)
</message>
<stderr_txt>
oa.pc|9dec.nc
oa.pc|0jan.nc
oa.pc|0feb.nc
oa.pc|0mar.nc
oa.pc|0apr.nc
oa.pc|0may.nc
oa.pc|0jun.nc
oa.pc|0jul.nc
oa.pc|0aug.nc
oa.pc|0sep.nc
oa.pc|0oct.nc
oa.pc|0nov.nc
oa.pe|9dec.nc
oa.pe|0jan.nc
oa.pe|0feb.nc
oa.pe|0mar.nc
oa.pe|0apr.nc
oa.pe|0may.nc
oa.pe|0jun.nc
oa.pe|0jul.nc
oa.pe|0aug.nc
oa.pe|0sep.nc
oa.pe|0oct.nc
oa.pe|0nov.nc
CPDN Monitor - Quit request from BOINC...
oa.pc|1dec.nc
oa.pc|2jan.nc
oa.pc|2feb.nc
oa.pc|2mar.nc
oa.pc|2apr.nc
oa.pc|2may.nc
oa.pc|2jun.nc
oa.pc|2jul.nc
oa.pc|2aug.nc
oa.pc|2sep.nc
oa.pc|2oct.nc
oa.pc|2nov.nc
oa.pe|1dec.nc
oa.pe|2jan.nc
oa.pe|2feb.nc
oa.pe|2mar.nc
oa.pe|2apr.nc
oa.pe|2may.nc
oa.pe|2jun.nc
oa.pe|2jul.nc
oa.pe|2aug.nc
oa.pe|2sep.nc
oa.pe|2oct.nc
oa.pe|2nov.nc
oa.pc|2dec.nc
oa.pc|3jan.nc
oa.pc|3feb.nc
oa.pc|3mar.nc
oa.pc|3apr.nc
oa.pc|3may.nc
oa.pc|3jun.nc
oa.pc|3jul.nc
oa.pc|3aug.nc
oa.pc|3sep.nc
oa.pc|3oct.nc
oa.pc|3nov.nc
oa.pe|2dec.nc
oa.pe|3jan.nc
oa.pe|3feb.nc
oa.pe|3mar.nc
oa.pe|3apr.nc
oa.pe|3may.nc
oa.pe|3jun.nc
oa.pe|3jul.nc
oa.pe|3aug.nc
oa.pe|3sep.nc
oa.pe|3oct.nc
oa.pe|3nov.nc
Suspended CPDN Monitor - Suspend request from BOINC...
oa.pc|3dec.nc
oa.pc|4jan.nc
oa.pc|4feb.nc
oa.pc|4mar.nc
oa.pc|4apr.nc
oa.pc|4may.nc
oa.pc|4jun.nc
oa.pc|4jul.nc
oa.pc|4aug.nc
oa.pc|4sep.nc
oa.pc|4oct.nc
oa.pc|4nov.nc
oa.pe|3dec.nc
oa.pe|4jan.nc
oa.pe|4feb.nc
oa.pe|4mar.nc
oa.pe|4apr.nc
oa.pe|4may.nc
oa.pe|4jun.nc
oa.pe|4jul.nc
oa.pe|4aug.nc
oa.pe|4sep.nc
oa.pe|4oct.nc
oa.pe|4nov.nc
OPEN:  File Creation Failed: No such file or directory
OPEN:  Unable to Open File dataout/a989oa.peg4dec for Read/Write

Model crashed: go
cpdnmonitor: cannot open input file /store/BOINC/projects/climateprediction.net/hadam3pm2_a989_1959_10_008658148/jobs/xhjlo.ihist after 11 attempts
cpdnmonitor: cannot open input file /store/BOINC/projects/climateprediction.net/hadam3pm2_a989_1959_10_008658148/jobs/xhjlo.namelists after 11 attempts
cpdnmonitor: cannot open input file /store/BOINC/projects/climateprediction.net/hadam3pm2_a989_1959_10_008658148/dataout/restart.day after 11 attempts
forrtl: Bad file descriptor
forrtl: severe (30): open failure, unit 6, file stdout
Image              PC        Routine            Line        Source             
hadam3pm2_um_7.03  0845AB9D  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  08459995  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  0842865F  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  083E7E1D  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  083E7767  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  0841C093  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  0831E96F  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  0834C82D  Unknown               Unknown  Unknown
libc.so.6          F0CC8A83  Unknown               Unknown  Unknown
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2481, iMonCtr=1
Model crash detected, will try to restart...
cpdnmonitor: cannot open input file /store/BOINC/projects/climateprediction.net/hadam3pm2_a989_1959_10_008658148/jobs/xhjlo.ihist after 11 attempts
cpdnmonitor: cannot open input file /store/BOINC/projects/climateprediction.net/hadam3pm2_a989_1959_10_008658148/jobs/xhjlo.namelists after 11 attempts
cpdnmonitor: cannot open input file /store/BOINC/projects/climateprediction.net/hadam3pm2_a989_1959_10_008658148/dataout/restart.day after 11 attempts
forrtl: Bad file descriptor
forrtl: severe (30): open failure, unit 6, file stdout
Image              PC        Routine            Line        Source             
hadam3pm2_um_7.03  0845AB9D  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  08459995  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  0842865F  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  083E7E1D  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  083E7767  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  0841C093  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  0831E96F  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  0834C82D  Unknown               Unknown  Unknown
libc.so.6          F0D8AA83  Unknown               Unknown  Unknown
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2481, iMonCtr=1
Model crash detected, will try to restart...
cpdnmonitor: cannot open input file /store/BOINC/projects/climateprediction.net/hadam3pm2_a989_1959_10_008658148/jobs/xhjlo.ihist after 11 attempts
cpdnmonitor: cannot open input file /store/BOINC/projects/climateprediction.net/hadam3pm2_a989_1959_10_008658148/jobs/xhjlo.namelists after 11 attempts
cpdnmonitor: cannot open input file /store/BOINC/projects/climateprediction.net/hadam3pm2_a989_1959_10_008658148/dataout/restart.day after 11 attempts
forrtl: Bad file descriptor
forrtl: severe (30): open failure, unit 6, file stdout
Image              PC        Routine            Line        Source             
hadam3pm2_um_7.03  0845AB9D  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  08459995  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  0842865F  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  083E7E1D  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  083E7767  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  0841C093  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  0831E96F  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  0834C82D  Unknown               Unknown  Unknown
libc.so.6          F0D96A83  Unknown               Unknown  Unknown
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2481, iMonCtr=1
Model crash detected, will try to restart...
cpdnmonitor: cannot open input file /store/BOINC/projects/climateprediction.net/hadam3pm2_a989_1959_10_008658148/jobs/xhjlo.ihist after 11 attempts
cpdnmonitor: cannot open input file /store/BOINC/projects/climateprediction.net/hadam3pm2_a989_1959_10_008658148/jobs/xhjlo.namelists after 11 attempts
cpdnmonitor: cannot open input file /store/BOINC/projects/climateprediction.net/hadam3pm2_a989_1959_10_008658148/dataout/restart.day after 11 attempts
forrtl: Bad file descriptor
forrtl: severe (30): open failure, unit 6, file stdout
Image              PC        Routine            Line        Source             
hadam3pm2_um_7.03  0845AB9D  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  08459995  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  0842865F  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  083E7E1D  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  083E7767  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  0841C093  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  0831E96F  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  0834C82D  Unknown               Unknown  Unknown
libc.so.6          F0D27A83  Unknown               Unknown  Unknown
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2481, iMonCtr=1
Model crash detected, will try to restart...
cpdnmonitor: cannot open input file /store/BOINC/projects/climateprediction.net/hadam3pm2_a989_1959_10_008658148/jobs/xhjlo.ihist after 11 attempts
cpdnmonitor: cannot open input file /store/BOINC/projects/climateprediction.net/hadam3pm2_a989_1959_10_008658148/jobs/xhjlo.namelists after 11 attempts
cpdnmonitor: cannot open input file /store/BOINC/projects/climateprediction.net/hadam3pm2_a989_1959_10_008658148/dataout/restart.day after 11 attempts
forrtl: Bad file descriptor
forrtl: severe (30): open failure, unit 6, file stdout
Image              PC        Routine            Line        Source             
hadam3pm2_um_7.03  0845AB9D  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  08459995  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  0842865F  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  083E7E1D  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  083E7767  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  0841C093  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  0831E96F  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  0834C82D  Unknown               Unknown  Unknown
libc.so.6          F0D34A83  Unknown               Unknown  Unknown
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2481, iMonCtr=1
Model crash detected, will try to restart...
Sorry, too many model crashes! :-(
18:27:35 (2481): called boinc_finish

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
12 Dec 2014 11:16:10 1292107 16922271 hadam3pm2_a989_1959_10_008658148_2 169,988 327,382 1.9259
12 Dec 2014 11:16:10 1292107 16922271 hadam3pm2_a989_1959_10_008658148_2 167,108 321,723 1.9252
12 Dec 2014 11:16:10 1292107 16922271 hadam3pm2_a989_1959_10_008658148_2 164,228 316,034 1.9244
12 Dec 2014 11:16:10 1292107 16922271 hadam3pm2_a989_1959_10_008658148_2 161,348 310,419 1.9239
12 Dec 2014 11:16:10 1292107 16922271 hadam3pm2_a989_1959_10_008658148_2 158,468 304,872 1.9239
12 Dec 2014 11:16:10 1292107 16922271 hadam3pm2_a989_1959_10_008658148_2 155,588 299,346 1.9240
12 Dec 2014 11:16:10 1292107 16922271 hadam3pm2_a989_1959_10_008658148_2 152,708 293,799 1.9239
12 Dec 2014 11:16:10 1292107 16922271 hadam3pm2_a989_1959_10_008658148_2 149,828 288,226 1.9237
12 Dec 2014 11:16:10 1292107 16922271 hadam3pm2_a989_1959_10_008658148_2 146,948 282,648 1.9235
12 Dec 2014 11:16:10 1292107 16922271 hadam3pm2_a989_1959_10_008658148_2 144,068 277,031 1.9229
12 Dec 2014 11:16:09 1292107 16922271 hadam3pm2_a989_1959_10_008658148_2 141,188 271,480 1.9228
12 Dec 2014 11:16:09 1292107 16922271 hadam3pm2_a989_1959_10_008658148_2 135,428 260,371 1.9226
12 Dec 2014 11:16:09 1292107 16922271 hadam3pm2_a989_1959_10_008658148_2 132,548 254,751 1.9220
12 Dec 2014 11:16:09 1292107 16922271 hadam3pm2_a989_1959_10_008658148_2 129,668 249,063 1.9208
12 Dec 2014 11:16:09 1292107 16922271 hadam3pm2_a989_1959_10_008658148_2 126,788 243,669 1.9219
12 Dec 2014 11:16:09 1292107 16922271 hadam3pm2_a989_1959_10_008658148_2 123,908 238,295 1.9232
12 Dec 2014 11:16:09 1292107 16922271 hadam3pm2_a989_1959_10_008658148_2 121,028 232,917 1.9245
12 Dec 2014 11:16:09 1292107 16922271 hadam3pm2_a989_1959_10_008658148_2 118,148 227,309 1.9239
12 Dec 2014 11:16:09 1292107 16922271 hadam3pm2_a989_1959_10_008658148_2 115,268 221,707 1.9234
12 Dec 2014 11:16:09 1292107 16922271 hadam3pm2_a989_1959_10_008658148_2 112,388 216,133 1.9231


©2024 climateprediction.net