climateprediction.net home page
Task 22316857

Task 22316857

Name oifs_43r3_01a4_2019110100_123_993_12215098_1
Workunit 12215098
Created 24 Feb 2023, 18:57:39 UTC
Sent 24 Feb 2023, 19:05:34 UTC
Report deadline 25 Apr 2023, 19:05:34 UTC
Received 26 Feb 2023, 13:34:59 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 1 (0x00000001) Unknown error code
Computer ID 1534746
Run time 2 hours 4 min 22 sec
CPU time 46 min 33 sec
Validate state Invalid
Credit 0.00
Device peak FLOPS 5.61 GFLOPS
Application version OpenIFS 43r3 v1.21
x86_64-pc-linux-gnu
Peak working set size 4,398.59 MB
Peak swap size 4,974.23 MB
Peak disk usage 1,262.91 MB
Stderr
<core_client_version>7.20.5</core_client_version>
<![CDATA[
<message>
process exited with code 1 (0x1, -255)</message>
<stderr_txt>
(argv0) ../../projects/climateprediction.net/oifs_43r3_1.21_x86_64-pc-linux-gnu
(argv1) start_date: 2019110100
(argv2) exptid: hnw4
(argv3) unique_member_id: 01a4
(argv4) batchid: 993
(argv5) wuid: 12215098
(argv6) fclen: 123
(argv7) app_name: oifs_43r3
(argv8) nthreads: 1
Working directory is: /var/lib/boinc/slots/12
Project directory is: /var/lib/boinc/projects/climateprediction.net/
app name: oifs_43r3
version: 1.21
Location of temp folder: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_12215098
Copying: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_app_1.21_x86_64-pc-linux-gnu.zip to: /var/lib/boinc/slots/12/oifs_43r3_app_1.21_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc/slots/12/oifs_43r3_app_1.21_x86_64-pc-linux-gnu.zip
(argv0) ../../projects/climateprediction.net/oifs_43r3_1.21_x86_64-pc-linux-gnu
(argv1) start_date: 2019110100
(argv2) exptid: hnw4
(argv3) unique_member_id: 01a4
(argv4) batchid: 993
(argv5) wuid: 12215098
(argv6) fclen: 123
(argv7) app_name: oifs_43r3
(argv8) nthreads: 1
Working directory is: /var/lib/boinc/slots/12
Project directory is: /var/lib/boinc/projects/climateprediction.net/
app name: oifs_43r3
version: 1.21
Location of temp folder: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_12215098
..mkdir for temp folder for results failed
Copying: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_app_1.21_x86_64-pc-linux-gnu.zip to: /var/lib/boinc/slots/12/oifs_43r3_app_1.21_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc/slots/12/oifs_43r3_app_1.21_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_32899890692f7e56334086400a45623a to: /var/lib/boinc/slots/12/oifs_43r3_01a4_2019110100_123_993_12215098.zip
Unzipping the namelist zip file: /var/lib/boinc/slots/12/oifs_43r3_01a4_2019110100_123_993_12215098.zip
ic_ancil_file: ic_ancil_12215098
ifsdata_file: ifsdata_12215098
climate_data_file: clim_data_12215098
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_8dc050ecdf1404feb0c78577232c8cf8 to: /var/lib/boinc/slots/12/ic_ancil_12215098.zip
Unzipping the IC ancils zip file: /var/lib/boinc/slots/12/ic_ancil_12215098.zip
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_0739f3ed3703379495337c5be5babf87 to: /var/lib/boinc/slots/12/ifsdata/ifsdata_12215098.zip
Unzipping the ifsdata_zip file: /var/lib/boinc/slots/12/ifsdata/ifsdata_12215098.zip
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc/slots/12/159l_2/clim_data_12215098.zip
Unzipping the climate data zip file: /var/lib/boinc/slots/12/159l_2/clim_data_12215098.zip
Checking for progress XML file: /var/lib/boinc/slots/12/progress_file_12215098.xml
Creating progress file: /var/lib/boinc/slots/12/progress_file_12215098.xml
last_cpu_time: 0
upload_file_number: 0
last_iter: 0
last_upload: 0
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_01a4_2019110100_123_993_12215098_1_r1149954061
The child process has been launched with process id: 25177
Executing the command: /var/lib/boinc/slots/12/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:vm9:1:1:25177:25177] [20230226:114704:1677386824:0.004] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2091] fp = 0x3124020
[EC_DRHOOK:vm9:1:1:25177:25177] [20230226:114704:1677386824:0.004] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:vm9:1:1:25177:25177] [20230226:114704:1677386824:0.004] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:vm9:1:1:25177:25177] [20230226:114704:1677386824:0.004] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:vm9:1:1:25177:25177] [20230226:114704:1677386824:0.006] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:vm9:1:1:25177:25177] [20230226:114704:1677386824:0.006] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:vm9:1:1:25177:25177] [20230226:114704:1677386824:0.006] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:vm9:1:1:25177:25177] [20230226:114704:1677386824:0.006] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:vm9:1:1:25177:25177] [20230226:114704:1677386824:0.006] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:vm9:1:1:25177:25177] [20230226:114704:1677386824:0.006] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:vm9:1:1:25177:25177] [20230226:114704:1677386824:0.006] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:vm9:1:1:25177:25177] [20230226:114704:1677386824:0.006] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:vm9:1:1:25177:25177] [20230226:114704:1677386824:0.006] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:vm9:1:1:25177:25177] [20230226:114704:1677386824:0.006] [ignore_signals@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:vm9:1:1:25177:25177] [20230226:114704:1677386824:0.006] [restore_default_signals@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:vm9:1:1:25177:25177] [20230226:114704:1677386824:0.006] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x810730 (old at 0x1ce8440)
[EC_DRHOOK:vm9:1:1:25177:25177] [20230226:114704:1677386824:0.006] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x810730 (old at (nil))
[EC_DRHOOK:vm9:1:1:25177:25177] [20230226:114704:1677386824:0.006] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x810730 (old at 0x1ce8440)
[EC_DRHOOK:vm9:1:1:25177:25177] [20230226:114704:1677386824:0.006] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x810730 (old at (nil))
[EC_DRHOOK:vm9:1:1:25177:25177] [20230226:114704:1677386824:0.006] [trapfpe_treatment@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:vm9:1:1:25177:25177] [20230226:114704:1677386824:0.006] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x810730 (old at 0x1ce8440)
[EC_DRHOOK:vm9:1:1:25177:25177] [20230226:114704:1677386824:0.006] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x810730 (old at 0x1ce8440)
[EC_DRHOOK:vm9:1:1:25177:25177] [20230226:114704:1677386824:0.006] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x810730 (old at (nil))
[EC_DRHOOK:vm9:1:1:25177:25177] [20230226:114704:1677386824:0.006] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x810730 (old at 0x1ce8440)
[EC_DRHOOK:vm9:1:1:25177:25177] [20230226:114704:1677386824:0.006] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x810730 (old at 0x1ce8440)
[EC_DRHOOK:vm9:1:1:25177:25177] [20230226:114704:1677386824:0.006] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x810730 (old at 0x1ce8440)
[EC_DRHOOK:vm9:1:1:25177:25177] [20230226:114704:1677386824:0.006] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x810730 (old at (nil))
[EC_DRHOOK:vm9:1:1:25177:25177] [20230226:114704:1677386824:0.006] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x810730 (old at (nil))
[EC_DRHOOK:vm9:1:1:25177:25177] [20230226:114704:1677386824:0.006] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x810730 (old at (nil))
[EC_DRHOOK:vm9:1:1:25177:25177] [20230226:114704:1677386824:0.006] [catch_signals@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_1.21_x86_64-pc-linux-gnu
(argv1) start_date: 2019110100
(argv2) exptid: hnw4
(argv3) unique_member_id: 01a4
(argv4) batchid: 993
(argv5) wuid: 12215098
(argv6) fclen: 123
(argv7) app_name: oifs_43r3
(argv8) nthreads: 1
Working directory is: /var/lib/boinc/slots/12
Project directory is: /var/lib/boinc/projects/climateprediction.net/
app name: oifs_43r3
version: 1.21
Location of temp folder: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_12215098
..mkdir for temp folder for results failed
Copying: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_app_1.21_x86_64-pc-linux-gnu.zip to: /var/lib/boinc/slots/12/oifs_43r3_app_1.21_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc/slots/12/oifs_43r3_app_1.21_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_32899890692f7e56334086400a45623a to: /var/lib/boinc/slots/12/oifs_43r3_01a4_2019110100_123_993_12215098.zip
Unzipping the namelist zip file: /var/lib/boinc/slots/12/oifs_43r3_01a4_2019110100_123_993_12215098.zip
ic_ancil_file: ic_ancil_12215098
ifsdata_file: ifsdata_12215098
climate_data_file: clim_data_12215098
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_8dc050ecdf1404feb0c78577232c8cf8 to: /var/lib/boinc/slots/12/ic_ancil_12215098.zip
Unzipping the IC ancils zip file: /var/lib/boinc/slots/12/ic_ancil_12215098.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_0739f3ed3703379495337c5be5babf87 to: /var/lib/boinc/slots/12/ifsdata/ifsdata_12215098.zip
Unzipping the ifsdata_zip file: /var/lib/boinc/slots/12/ifsdata/ifsdata_12215098.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc/slots/12/159l_2/clim_data_12215098.zip
Unzipping the climate data zip file: /var/lib/boinc/slots/12/159l_2/clim_data_12215098.zip
Checking for progress XML file: /var/lib/boinc/slots/12/progress_file_12215098.xml
Opened progress file ok : /var/lib/boinc/slots/12/progress_file_12215098.xml
-- Model is restarting --
Adjusting last_iter, 0, to previous model restart step.
Creating progress file: /var/lib/boinc/slots/12/progress_file_12215098.xml
last_cpu_time: 137
upload_file_number: 0
last_iter: 1
last_upload: 0
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_01a4_2019110100_123_993_12215098_1_r1149954061
The child process has been launched with process id: 25687
Executing the command: /var/lib/boinc/slots/12/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:vm9:1:1:25687:25687] [20230226:173547:1677407747:0.015] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2091] fp = 0x3124020
[EC_DRHOOK:vm9:1:1:25687:25687] [20230226:173547:1677407747:0.015] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:vm9:1:1:25687:25687] [20230226:173547:1677407747:0.015] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:vm9:1:1:25687:25687] [20230226:173547:1677407747:0.015] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:vm9:1:1:25687:25687] [20230226:173547:1677407747:0.015] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:vm9:1:1:25687:25687] [20230226:173547:1677407747:0.015] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:vm9:1:1:25687:25687] [20230226:173547:1677407747:0.015] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:vm9:1:1:25687:25687] [20230226:173547:1677407747:0.015] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:vm9:1:1:25687:25687] [20230226:173547:1677407747:0.015] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:vm9:1:1:25687:25687] [20230226:173547:1677407747:0.015] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:vm9:1:1:25687:25687] [20230226:173547:1677407747:0.015] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:vm9:1:1:25687:25687] [20230226:173547:1677407747:0.016] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:vm9:1:1:25687:25687] [20230226:173547:1677407747:0.016] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:vm9:1:1:25687:25687] [20230226:173547:1677407747:0.016] [ignore_signals@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:vm9:1:1:25687:25687] [20230226:173547:1677407747:0.016] [restore_default_signals@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:vm9:1:1:25687:25687] [20230226:173547:1677407747:0.016] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x810730 (old at 0x1ce8440)
[EC_DRHOOK:vm9:1:1:25687:25687] [20230226:173547:1677407747:0.016] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x810730 (old at (nil))
[EC_DRHOOK:vm9:1:1:25687:25687] [20230226:173547:1677407747:0.016] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x810730 (old at 0x1ce8440)
[EC_DRHOOK:vm9:1:1:25687:25687] [20230226:173547:1677407747:0.016] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x810730 (old at (nil))
[EC_DRHOOK:vm9:1:1:25687:25687] [20230226:173547:1677407747:0.016] [trapfpe_treatment@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:vm9:1:1:25687:25687] [20230226:173547:1677407747:0.016] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x810730 (old at 0x1ce8440)
[EC_DRHOOK:vm9:1:1:25687:25687] [20230226:173547:1677407747:0.016] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x810730 (old at 0x1ce8440)
[EC_DRHOOK:vm9:1:1:25687:25687] [20230226:173547:1677407747:0.016] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x810730 (old at (nil))
[EC_DRHOOK:vm9:1:1:25687:25687] [20230226:173547:1677407747:0.016] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x810730 (old at 0x1ce8440)
[EC_DRHOOK:vm9:1:1:25687:25687] [20230226:173547:1677407747:0.016] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x810730 (old at 0x1ce8440)
[EC_DRHOOK:vm9:1:1:25687:25687] [20230226:173547:1677407747:0.016] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x810730 (old at 0x1ce8440)
[EC_DRHOOK:vm9:1:1:25687:25687] [20230226:173547:1677407747:0.016] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x810730 (old at (nil))
[EC_DRHOOK:vm9:1:1:25687:25687] [20230226:173547:1677407747:0.016] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x810730 (old at (nil))
[EC_DRHOOK:vm9:1:1:25687:25687] [20230226:173547:1677407747:0.016] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x810730 (old at (nil))
[EC_DRHOOK:vm9:1:1:25687:25687] [20230226:173547:1677407747:0.016] [catch_signals@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  19:32:55 STEP    0 H=   0:00 +CPU=130.588
Moving to projects directory: /var/lib/boinc/slots/12/ICMGGhnw4+000000
Moving to projects directory: /var/lib/boinc/slots/12/ICMSHhnw4+000000
Moving to projects directory: /var/lib/boinc/slots/12/ICMUAhnw4+000000
  19:33:14 STEP    1 H=   1:00 +CPU= 19.034
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  19:35:31 STEP    2 H=   2:00 +CPU= 19.583
  19:35:56 STEP    3 H=   3:00 +CPU= 24.565
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  19:36:15 STEP    4 H=   4:00 +CPU= 17.074
  19:36:30 STEP    5 H=   5:00 +CPU= 15.121
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  19:36:59 STEP    6 H=   6:00 +CPU= 26.543
  19:37:14 STEP    7 H=   7:00 +CPU= 14.813
  19:37:29 STEP    8 H=   8:00 +CPU= 15.090
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  19:37:59 STEP    9 H=   9:00 +CPU= 28.764
  19:38:15 STEP   10 H=  10:00 +CPU= 15.924
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  19:38:34 STEP   11 H=  11:00 +CPU= 16.189
  19:38:59 STEP   12 H=  12:00 +CPU= 25.038
Moving to projects directory: /var/lib/boinc/slots/12/ICMGGhnw4+000012
Moving to projects directory: /var/lib/boinc/slots/12/ICMSHhnw4+000012
Moving to projects directory: /var/lib/boinc/slots/12/ICMUAhnw4+000012
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  19:39:18 STEP   13 H=  13:00 +CPU= 17.250
  19:39:34 STEP   14 H=  14:00 +CPU= 16.149
  19:40:02 STEP   15 H=  15:00 +CPU= 27.538
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  19:40:19 STEP   16 H=  16:00 +CPU= 15.680
  19:40:34 STEP   17 H=  17:00 +CPU= 14.602
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  19:41:02 STEP   18 H=  18:00 +CPU= 26.154
  19:41:18 STEP   19 H=  19:00 +CPU= 16.285
  19:41:37 STEP   20 H=  20:00 +CPU= 18.755
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  19:42:06 STEP   21 H=  21:00 +CPU= 27.442
  19:42:23 STEP   22 H=  22:00 +CPU= 16.925
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  19:42:46 STEP   23 H=  23:00 +CPU= 20.188
  19:43:16 STEP   24 H=  24:00 +CPU= 29.958
Moving to projects directory: /var/lib/boinc/slots/12/ICMGGhnw4+000024
Moving to projects directory: /var/lib/boinc/slots/12/ICMSHhnw4+000024
Moving to projects directory: /var/lib/boinc/slots/12/ICMUAhnw4+000024
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_12215098/ICMGGhnw4+000000
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_12215098/ICMSHhnw4+000000
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_12215098/ICMUAhnw4+000000
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_12215098/ICMGGhnw4+000012
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_12215098/ICMSHhnw4+000012
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_12215098/ICMUAhnw4+000012
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_01a4_2019110100_123_993_12215098_1_r1149954061_0.zip
Uploading the intermediate file: upload_file_0.zip
  19:43:30 STEP   25 H=  25:00 +CPU= 14.189
  19:43:47 STEP   26 H=  26:00 +CPU= 17.513
  19:44:15 STEP   27 H=  27:00 +CPU= 27.092
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  19:44:33 STEP   28 H=  28:00 +CPU= 16.757
  19:44:49 STEP   29 H=  29:00 +CPU= 15.394
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  19:45:17 STEP   30 H=  30:00 +CPU= 25.877
  19:45:34 STEP   31 H=  31:00 +CPU= 16.968
  19:45:50 STEP   32 H=  32:00 +CPU= 16.367
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  19:46:16 STEP   33 H=  33:00 +CPU= 24.001
  19:46:33 STEP   34 H=  34:00 +CPU= 16.742
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  19:46:51 STEP   35 H=  35:00 +CPU= 16.253
  19:47:20 STEP   36 H=  36:00 +CPU= 29.276
Moving to projects directory: /var/lib/boinc/slots/12/ICMGGhnw4+000036
Moving to projects directory: /var/lib/boinc/slots/12/ICMSHhnw4+000036
Moving to projects directory: /var/lib/boinc/slots/12/ICMUAhnw4+000036
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  19:47:42 STEP   37 H=  37:00 +CPU= 16.846
  19:47:57 STEP   38 H=  38:00 +CPU= 15.155
  19:48:22 STEP   39 H=  39:00 +CPU= 24.784
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  19:48:41 STEP   40 H=  40:00 +CPU= 17.366
  19:48:56 STEP   41 H=  41:00 +CPU= 14.932
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  19:49:23 STEP   42 H=  42:00 +CPU= 24.737
  19:49:39 STEP   43 H=  43:00 +CPU= 15.729
  19:49:55 STEP   44 H=  44:00 +CPU= 16.318
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  19:50:26 STEP   45 H=  45:00 +CPU= 25.996
  19:50:43 STEP   46 H=  46:00 +CPU= 17.077
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  19:51:02 STEP   47 H=  47:00 +CPU= 17.077
  19:51:33 STEP   48 H=  48:00 +CPU= 31.239
Moving to projects directory: /var/lib/boinc/slots/12/ICMGGhnw4+000048
Moving to projects directory: /var/lib/boinc/slots/12/ICMSHhnw4+000048
Moving to projects directory: /var/lib/boinc/slots/12/ICMUAhnw4+000048
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_12215098/ICMGGhnw4+000024
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_12215098/ICMSHhnw4+000024
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_12215098/ICMUAhnw4+000024
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_12215098/ICMGGhnw4+000036
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_12215098/ICMSHhnw4+000036
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_12215098/ICMUAhnw4+000036
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_01a4_2019110100_123_993_12215098_1_r1149954061_1.zip
Uploading the intermediate file: upload_file_1.zip
  19:51:49 STEP   49 H=  49:00 +CPU= 15.782
  19:52:07 STEP   50 H=  50:00 +CPU= 18.486
  19:52:35 STEP   51 H=  51:00 +CPU= 27.411
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  19:52:53 STEP   52 H=  52:00 +CPU= 16.357
  19:53:10 STEP   53 H=  53:00 +CPU= 16.782
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  19:53:38 STEP   54 H=  54:00 +CPU= 25.694
  19:53:53 STEP   55 H=  55:00 +CPU= 15.519
  19:54:09 STEP   56 H=  56:00 +CPU= 16.158
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  19:54:38 STEP   57 H=  57:00 +CPU= 26.247
  19:54:54 STEP   58 H=  58:00 +CPU= 15.744
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  19:55:11 STEP   59 H=  59:00 +CPU= 15.307
  19:55:39 STEP   60 H=  60:00 +CPU= 27.752
Moving to projects directory: /var/lib/boinc/slots/12/ICMGGhnw4+000060
Moving to projects directory: /var/lib/boinc/slots/12/ICMSHhnw4+000060
Moving to projects directory: /var/lib/boinc/slots/12/ICMUAhnw4+000060
  19:55:52 STEP   61 H=  61:00 +CPU= 13.540
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  19:56:13 STEP   62 H=  62:00 +CPU= 17.050
  19:56:38 STEP   63 H=  63:00 +CPU= 25.678
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  19:56:57 STEP   64 H=  64:00 +CPU= 16.522
  19:57:14 STEP   65 H=  65:00 +CPU= 16.795
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  19:57:46 STEP   66 H=  66:00 +CPU= 29.889
  19:58:01 STEP   67 H=  67:00 +CPU= 15.255
  19:58:19 STEP   68 H=  68:00 +CPU= 17.981
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  19:58:48 STEP   69 H=  69:00 +CPU= 26.744
  19:59:06 STEP   70 H=  70:00 +CPU= 18.148
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  19:59:26 STEP   71 H=  71:00 +CPU= 17.985
  19:59:58 STEP   72 H=  72:00 +CPU= 31.872
Moving to projects directory: /var/lib/boinc/slots/12/ICMGGhnw4+000072
Moving to projects directory: /var/lib/boinc/slots/12/ICMSHhnw4+000072
Moving to projects directory: /var/lib/boinc/slots/12/ICMUAhnw4+000072
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_12215098/ICMGGhnw4+000048
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_12215098/ICMSHhnw4+000048
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_12215098/ICMUAhnw4+000048
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_12215098/ICMGGhnw4+000060
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_12215098/ICMSHhnw4+000060
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_12215098/ICMUAhnw4+000060
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_01a4_2019110100_123_993_12215098_1_r1149954061_2.zip
Uploading the intermediate file: upload_file_2.zip
  20:00:14 STEP   73 H=  73:00 +CPU= 15.689
  20:00:33 STEP   74 H=  74:00 +CPU= 18.884
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  20:01:01 STEP   75 H=  75:00 +CPU= 26.272
  20:01:18 STEP   76 H=  76:00 +CPU= 17.320
  20:01:33 STEP   77 H=  77:00 +CPU= 14.570
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  20:02:05 STEP   78 H=  78:00 +CPU= 29.689
  20:02:21 STEP   79 H=  79:00 +CPU= 15.219
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  20:02:42 STEP   80 H=  80:00 +CPU= 19.016
  20:03:08 STEP   81 H=  81:00 +CPU= 25.967
  20:03:24 STEP   82 H=  82:00 +CPU= 16.716
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  20:03:43 STEP   83 H=  83:00 +CPU= 16.260
  20:04:10 STEP   84 H=  84:00 +CPU= 27.157
Moving to projects directory: /var/lib/boinc/slots/12/ICMGGhnw4+000084
Moving to projects directory: /var/lib/boinc/slots/12/ICMSHhnw4+000084
Moving to projects directory: /var/lib/boinc/slots/12/ICMUAhnw4+000084
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  20:04:31 STEP   85 H=  85:00 +CPU= 19.102
  20:04:48 STEP   86 H=  86:00 +CPU= 16.645
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  20:05:15 STEP   87 H=  87:00 +CPU= 25.054
  20:05:34 STEP   88 H=  88:00 +CPU= 18.604
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  20:06:00 STEP   89 H=  89:00 +CPU= 16.513
  20:06:26 STEP   90 H=  90:00 +CPU= 26.268
  20:06:41 STEP   91 H=  91:00 +CPU= 14.740
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  20:07:03 STEP   92 H=  92:00 +CPU= 18.537
  20:07:27 STEP   93 H=  93:00 +CPU= 23.765
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  20:07:46 STEP   94 H=  94:00 +CPU= 16.784
  20:08:01 STEP   95 H=  95:00 +CPU= 15.562
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  20:08:35 STEP   96 H=  96:00 +CPU= 30.870
Moving to projects directory: /var/lib/boinc/slots/12/ICMGGhnw4+000096
Moving to projects directory: /var/lib/boinc/slots/12/ICMSHhnw4+000096
Moving to projects directory: /var/lib/boinc/slots/12/ICMUAhnw4+000096
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_12215098/ICMGGhnw4+000072
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_12215098/ICMSHhnw4+000072
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_12215098/ICMUAhnw4+000072
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_12215098/ICMGGhnw4+000084
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_12215098/ICMSHhnw4+000084
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_12215098/ICMUAhnw4+000084
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_01a4_2019110100_123_993_12215098_1_r1149954061_3.zip
Uploading the intermediate file: upload_file_3.zip
  20:08:53 STEP   97 H=  97:00 +CPU= 17.620
  20:09:09 STEP   98 H=  98:00 +CPU= 15.615
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  20:09:35 STEP   99 H=  99:00 +CPU= 24.712
  20:09:51 STEP  100 H= 100:00 +CPU= 15.270
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  20:10:10 STEP  101 H= 101:00 +CPU= 17.616
  20:10:38 STEP  102 H= 102:00 +CPU= 27.878
  20:10:53 STEP  103 H= 103:00 +CPU= 15.084
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  20:11:11 STEP  104 H= 104:00 +CPU= 16.101
  20:11:37 STEP  105 H= 105:00 +CPU= 25.883
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  20:11:56 STEP  106 H= 106:00 +CPU= 16.692
  20:12:11 STEP  107 H= 107:00 +CPU= 14.685
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  20:12:42 STEP  108 H= 108:00 +CPU= 28.867
Moving to projects directory: /var/lib/boinc/slots/12/ICMGGhnw4+000108
Moving to projects directory: /var/lib/boinc/slots/12/ICMSHhnw4+000108
Moving to projects directory: /var/lib/boinc/slots/12/ICMUAhnw4+000108
  20:12:58 STEP  109 H= 109:00 +CPU= 16.856
  20:13:15 STEP  110 H= 110:00 +CPU= 16.900
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  20:13:53 STEP  111 H= 111:00 +CPU= 29.924
  20:14:09 STEP  112 H= 112:00 +CPU= 15.962
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  20:14:27 STEP  113 H= 113:00 +CPU= 16.149
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  20:15:10 STEP  114 H= 114:00 +CPU= 27.938
  20:15:26 STEP  115 H= 115:00 +CPU= 16.542
  20:15:43 STEP  116 H= 116:00 +CPU= 16.655
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  20:16:12 STEP  117 H= 117:00 +CPU= 26.597
  20:16:28 STEP  118 H= 118:00 +CPU= 15.791
  20:16:45 STEP  119 H= 119:00 +CPU= 15.984
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  20:17:18 STEP  120 H= 120:00 +CPU= 29.375
Moving to projects directory: /var/lib/boinc/slots/12/ICMGGhnw4+000120
Moving to projects directory: /var/lib/boinc/slots/12/ICMSHhnw4+000120
Moving to projects directory: /var/lib/boinc/slots/12/ICMUAhnw4+000120
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_12215098/ICMGGhnw4+000096
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_12215098/ICMSHhnw4+000096
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_12215098/ICMUAhnw4+000096
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_12215098/ICMGGhnw4+000108
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_12215098/ICMSHhnw4+000108
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_12215098/ICMUAhnw4+000108
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_01a4_2019110100_123_993_12215098_1_r1149954061_4.zip
Uploading the intermediate file: upload_file_4.zip
  20:17:33 STEP  121 H= 121:00 +CPU= 14.452
  20:17:51 STEP  122 H= 122:00 +CPU= 17.620
  20:18:18 STEP  123 H= 123:00 +CPU= 27.493
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  20:18:38 STEP  124 H= 124:00 +CPU= 17.343
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
..The child process has been killed with signal: 9
>>> Printing last 70 lines from file: NODE.001_01

 NSTEP =   122 STEPO   0AAA00AAA
  SPECTRAL NORMS: Pattern  1
 STEPO, YSPSDT_AR1     (  1,...), SP NORMS 2D: 5.185148666583E-01 1.628001130117E-01 6.967330932336E-02
 STEPO, YSPSDT_AR1     (SUM,...), SP NORMS 2D: 5.398429988568E-01
 STEPO, YGPSDT         (  1,...), Grid Point Norms
  FIELD LEV        AVERAGE               MINIMUM               MAXIMUM       
    101  60 -1.60051126209898E-02 -1.90802854557311E+00  1.73464077806779E+00
 STEPO, YGPSDT[clipped](  1,...), Grid Point Norms
  FIELD LEV        AVERAGE               MINIMUM               MAXIMUM       
    101  60 -1.32465148346488E-02 -1.00000000000000E+00  1.00000000000000E+00
  STEPO: evolving YSPSDT_AR1(  1)
   !!!!!!!!!!!!!! WAVE FIELDS INTEGRATED.  DATE IS: 20191106011500  !!!!!!!!!!!! 
   !!!!!!!!!!!!!! WAVE FIELDS INTEGRATED.  DATE IS: 20191106013000  !!!!!!!!!!!! 
   !!!!!!!!!!!!!! WAVE FIELDS INTEGRATED.  DATE IS: 20191106014500  !!!!!!!!!!!! 
   !!!!!!!!!!!!!! WAVE FIELDS INTEGRATED.  DATE IS: 20191106020000  !!!!!!!!!!!! 
 20:17:51 STEP  122 H= 122:00 +CPU= 17.620
  DATE= 2019   11    6
ISTEP=     123: Reset RSOLINC= 1361.0304 RIP0= 1385.1928
 SU_GHGCLIM, RCARDI,RCH4,RN2O,RCFC11,RCFC12,RNO2:   6.240006412635001E-004
  9.628461857022168E-007  4.996611132045250E-007  9.607432727722499E-010
  2.027017283006666E-009  7.940283000000001E-011
 SUECAEC: GLOBAL AVERAGE VOLCANIC SULPHATE FOR (KINDAT,KMINUT)     20191101
        7470  9.999999999999998E-005
SUECSO4: USE AVAILABLE FILES FOR DECADES:   2010 2020
 SUBROUTINE SUECSO4:, UNWEIGHTED AVERAGE:   2.296764401904288E-003
 NSTEP =   123 STEPO   0AAA00AAA
  SPECTRAL NORMS: Pattern  1
 STEPO, YSPSDT_AR1     (  1,...), SP NORMS 2D: 5.092037363523E-01 1.658180038556E-01 6.940258759794E-02
 STEPO, YSPSDT_AR1     (SUM,...), SP NORMS 2D: 5.339696521872E-01
 STEPO, YGPSDT         (  1,...), Grid Point Norms
  FIELD LEV        AVERAGE               MINIMUM               MAXIMUM       
    101  60 -1.13261324348136E-02 -1.75900440622613E+00  1.92616710624713E+00
 STEPO, YGPSDT[clipped](  1,...), Grid Point Norms
  FIELD LEV        AVERAGE               MINIMUM               MAXIMUM       
    101  60 -1.21802092429103E-02 -1.00000000000000E+00  1.00000000000000E+00
  STEPO: evolving YSPSDT_AR1(  1)
   !!!!!!!!!!!!!! WAVE FIELDS INTEGRATED.  DATE IS: 20191106021500  !!!!!!!!!!!! 
   !!!!!!!!!!!!!! WAVE FIELDS INTEGRATED.  DATE IS: 20191106023000  !!!!!!!!!!!! 
   !!!!!!!!!!!!!! WAVE FIELDS INTEGRATED.  DATE IS: 20191106024500  !!!!!!!!!!!! 
   !!!!!!!!!!!!!! WAVE FIELDS INTEGRATED.  DATE IS: 20191106030000  !!!!!!!!!!!! 
 20:18:18 STEP  123 H= 123:00 +CPU= 27.493
 NSTEP =   124 STEPO   0AAA00AAA
  SPECTRAL NORMS: Pattern  1
 STEPO, YSPSDT_AR1     (  1,...), SP NORMS 2D: 5.253858392204E-01 1.637110837922E-01 6.958831679733E-02
 STEPO, YSPSDT_AR1     (SUM,...), SP NORMS 2D: 5.528035191436E-01
 STEPO, YGPSDT         (  1,...), Grid Point Norms
  FIELD LEV        AVERAGE               MINIMUM               MAXIMUM       
    101  60 -4.47895388466097E-04 -2.15279976955843E+00  1.87256507454248E+00
 STEPO, YGPSDT[clipped](  1,...), Grid Point Norms
  FIELD LEV        AVERAGE               MINIMUM               MAXIMUM       
    101  60 -1.62204831209530E-03 -1.00000000000000E+00  1.00000000000000E+00
  STEPO: evolving YSPSDT_AR1(  1)
   !!!!!!!!!!!!!! WAVE FIELDS INTEGRATED.  DATE IS: 20191106031500  !!!!!!!!!!!! 
   !!!!!!!!!!!!!! WAVE FIELDS INTEGRATED.  DATE IS: 20191106033000  !!!!!!!!!!!! 
   !!!!!!!!!!!!!! WAVE FIELDS INTEGRATED.  DATE IS: 20191106034500  !!!!!!!!!!!! 
   !!!!!!!!!!!!!! WAVE FIELDS INTEGRATED.  DATE IS: 20191106040000  !!!!!!!!!!!! 
 20:18:38 STEP  124 H= 124:00 +CPU= 17.343
ISTEP=     125 RSOLINC= 1361.0304 RI0= 1366.0000 ZI0= 1361.0304 RIP0= 1385.2496
ISTEP=     125 ZI0= 1361.0304 ZSEASON= 1.017794747 REA=149597870000.00 RDEASO=148284346438.75
 NSTEP =   125 STEPO   0AAA00AAA
  SPECTRAL NORMS: Pattern  1
 STEPO, YSPSDT_AR1     (  1,...), SP NORMS 2D: 5.150753231841E-01 1.634867981456E-01 6.995325779137E-02
 STEPO, YSPSDT_AR1     (SUM,...), SP NORMS 2D: 5.356150255955E-01
 STEPO, YGPSDT         (  1,...), Grid Point Norms
  FIELD LEV        AVERAGE               MINIMUM               MAXIMUM       
    101  60 -1.86263670534025E-02 -2.08873234100144E+00  1.76298454452940E+00
 STEPO, YGPSDT[clipped](  1,...), Grid Point Norms
  FIELD LEV        AVERAGE               MINIMUM               MAXIMUM       
    101  60 -1.65363175886104E-02 -1.00000000000000E+00  1.00000000000000E+00
------------------------------------------------
CNT0 not found; string returned was: 'STEPO'
>>> Printing last 8 lines from file: ifs.stat

 20:16:50 FULLPOS-B DYNFPOS    120     0.719    0.719    0.719     41:56    161:03 0.10787949281015E-04       2GB       0MB
 20:16:51 0AAA00AAA STEPO      120     1.387    1.387    1.387     41:57    161:04 0.10787949281015E-04       2GB       0MB
 20:17:18 0AAA00AAA STEPO      121    25.112   25.112   26.845     42:22    161:31 0.10787949281015E-04       2GB       0MB
 20:17:33 0AAA00AAA STEPO      122    14.452   14.452   14.461     42:37    161:45 0.10787949281015E-04       2GB       0MB
 20:17:51 0AAA00AAA STEPO      123    17.621   17.621   18.264     42:55    162:03 0.10787949281015E-04       2GB       0MB
 20:18:18 0AAA00AAA STEPO      124    27.493   27.493   27.503     43:22    162:31 0.10787949281015E-04       2GB       0MB
 20:18:38 0AAA00AAA STEPO      125    17.347   17.347   19.358     43:39    162:50 0.10787949281015E-04       2GB       0MB
------------------------------------------------
>>> Printing last 11 lines from file: rcf

 CSTEP   = "     120",
 CTIME   = "00050000      ",
 NSTEPLPP        =         201,         202,          49,      228226,      228227, 5*0,
 IPRGPNSRES      =           1,
 IPRGPEWRES      =           1,
 IPRTRWRES       =           1,
 IPRTRVRES       =           1,
 GMASS0  =   98323.3590226272     ,
 GMASSI  =   98323.0322299277     
 /
------------------------------------------------
>>> Printing last 17 lines from file: waminfo

PARAM=10U/10V,                                                          
REPRES=GG,                                                              
ANALYSIS PERIOD =       0                                               
FORECAST PERIOD =   10198800                                            
WIND TIME STEP IN SECONDS =    3600                                     
ANALYSIS FROM 20191101000000 TO 20191101000000                          
ASSIMILATION NO                                                         
NEW FORECAST NO                                                         
STATUS STORM1= UNFINISHED                                               
STATUS STORM2= UNFINISHED                                               
STATUS STORM3= UNFINISHED                                               
DATE FOR OUTPUT OF BOTH RESTART FILES = 00000000000000                  
LAST DATE FOR SPECTRA FILE OUTPUT = 20200303000000                      
BEGIN DATE FOR USING SURFACE CURRENT = 20191101000000                   
PARALLEL RESTART = YES                                                  
NUMBER OF MPI TASKS USED =          1                                   
------------------------------------------------
>>> Printing last 8 lines from file: /var/lib/boinc/slots/12/progress_file_12215098.xml

<running_values>
  <last_cpu_time>2760.530000</last_cpu_time>
  <upload_file_number>5</upload_file_number>
  <last_iter>125</last_iter>
  <last_upload>432000</last_upload>
  <model_completed>0</model_completed>
</running_values>
------------------------------------------------
..Failed, model did not complete successfully

</stderr_txt>
]]>
No trickles!


©2024 cpdn.org