Task 22029485

Name	hadam4h_c162_206511_5_883_012038202_1
Workunit	12038202
Created	2 Mar 2021, 10:15:12 UTC
Sent	18 Mar 2021, 1:00:34 UTC
Report deadline	28 Feb 2022, 6:20:34 UTC
Received	11 Apr 2021, 20:38:38 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	12 (0x0000000C) Unknown error code
Computer ID	1241909
Run time	3 days 2 hours 14 min 33 sec
CPU time	3 days 0 hours 51 min 32 sec
Validate state	Invalid
Credit	0.00
Device peak FLOPS	3.53 GFLOPS
Application version	UK Met Office HadAM4 at N216 resolution v8.52 i686-pc-linux-gnu
Peak working set size	1,365.55 MB
Peak swap size	1,386.04 MB
Peak disk usage	12.92 MB
Stderr	<core_client_version>7.16.11</core_client_version> <![CDATA[ <message> process exited with code 12 (0xc, -244)</message> <stderr_txt> CPDN Monitor - Quit request from BOINC... BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/xnnuj.pipe_dummy BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/xnnuj.pipe_dummy BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/xnnuj.pipe_dummy BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/xnnuj.pipe_dummy BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/xnnuj.pipe_dummy BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/xnnuj.pipe_dummy BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/xnnuj.pipe_dummy forrtl: No space left on device forrtl: severe (38): error during write, unit 6, file /var/lib/boinc/projects/climateprediction.net/hadam4h_c162_206511_5_883_012038202/dataout/xnnuj.out Image PC Routine Line Source hadam4_um_8.52_i6 083F6605 Unknown Unknown Unknown hadam4_um_8.52_i6 0843ABB0 Unknown Unknown Unknown hadam4_um_8.52_i6 08438E0F Unknown Unknown Unknown hadam4_um_8.52_i6 081ABA99 Unknown Unknown Unknown hadam4_um_8.52_i6 0811E285 Unknown Unknown Unknown hadam4_um_8.52_i6 0811FF0C Unknown Unknown Unknown hadam4_um_8.52_i6 081D82A7 Unknown Unknown Unknown hadam4_um_8.52_i6 081E121B Unknown Unknown Unknown hadam4_um_8.52_i6 0836E63F Unknown Unknown Unknown Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=15895, iMonCtr=1 Model crash detected, will try to restart... BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/xnnuj.pipe_dummy BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/xnnuj.pipe_dummy BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/xnnuj.pipe_dummy BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/xnnuj.pipe_dummy BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/xnnuj.pipe_dummy BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/xnnuj.pipe_dummy BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/xnnuj.pipe_dummy BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/xnnuj.pipe_dummy CPDN Monitor - Quit request from BOINC... BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/xnnuj.pipe_dummy BUFFOUT: Write Failed: No space left on devCPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/xnnuj.pipe_dummy BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/xnnuj.pipe_dummy BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/xnnuj.pipe_dummy CPDN Monitor - Quit request from BOINC... BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/xnnuj.pipe_dummy BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/xnnuj.pipe_dummy BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/xnnuj.pipe_dummy BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/xnnuj.pipe_dummy BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: STWORK : Error in PP_FILE tmp/xnnuj.pipe_dummy BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/xnnOPEN: File Creation Failed: No such file or directory OPEN: Unable to Open File dataout/c162ga.pbq5nov for Read/Write Model crashed: STWORK : Error opening output PP file on unit 61 tmp/xnnuj.pipe_dummy cpdnmonitor: cannot open input file /var/lib/boinc/projects/climateprediction.net/hadam4h_c162_206511_5_883_012038202/jobs/xnnuj.ihist after 11 attempts cpdnmonitor: cannot open input file /var/lib/boinc/projects/climateprediction.net/hadam4h_c162_206511_5_883_012038202/jobs/xnnuj.namelists after 11 attempts cpdnmonitor: cannot open input file /var/lib/boinc/projects/climateprediction.net/hadam4h_c162_206511_5_883_012038202/dataout/atmos_restart.day after 11 attempts forrtl: Bad file descriptor forrtl: severe (30): open failure, unit 6, file /proc/9146/fd/ Image PC Routine Line Source hadam4_um_8.52_i6 083F6605 Unknown Unknown Unknown hadam4_um_8.52_i6 0843E0A0 Unknown Unknown Unknown hadam4_um_8.52_i6 081DF6C9 Unknown Unknown Unknown hadam4_um_8.52_i6 0836E63F Unknown Unknown Unknown Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5777, iMonCtr=1 Model crash detected, will try to restart... cpdnmonitor: cannot open input file /var/lib/boinc/projects/climateprediction.net/hadam4h_c162_206511_5_883_012038202/jobs/xnnuj.ihist after 11 attempts cpdnmonitor: cannot open input file /var/lib/boinc/projects/climateprediction.net/hadam4h_c162_206511_5_883_012038202/jobs/xnnuj.namelists after 11 attempts cpdnmonitor: cannot open input file /var/lib/boinc/projects/climateprediction.net/hadam4h_c162_206511_5_883_012038202/dataout/atmos_restart.day after 11 attempts forrtl: Bad file descriptor forrtl: severe (30): open failure, unit 6, file /proc/9155/fd/ Image PC Routine Line Source hadam4_um_8.52_i6 083F6605 Unknown Unknown Unknown hadam4_um_8.52_i6 0843E0A0 Unknown Unknown Unknown hadam4_um_8.52_i6 081DF6C9 Unknown Unknown Unknown hadam4_um_8.52_i6 0836E63F Unknown Unknown Unknown Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5777, iMonCtr=1 Model crash detected, will try to restart... cpdnmonitor: cannot open input file /var/lib/boinc/projects/climateprediction.net/hadam4h_c162_206511_5_883_012038202/jobs/xnnuj.ihist after 11 attempts cpdnmonitor: cannot open input file /var/lib/boinc/projects/climateprediction.net/hadam4h_c162_206511_5_883_012038202/jobs/xnnuj.namelists after 11 attempts cpdnmonitor: cannot open input file /var/lib/boinc/projects/climateprediction.net/hadam4h_c162_206511_5_883_012038202/dataout/atmos_restart.day after 11 attempts forrtl: Bad file descriptor forrtl: severe (30): open failure, unit 6, file /proc/9163/fd/ Image PC Routine Line Source hadam4_um_8.52_i6 083F6605 Unknown Unknown Unknown hadam4_um_8.52_i6 0843E0A0 Unknown Unknown Unknown hadam4_um_8.52_i6 081DF6C9 Unknown Unknown Unknown hadam4_um_8.52_i6 0836E63F Unknown Unknown Unknown Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5777, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( </stderr_txt> ]]>