climateprediction.net home page
Task 22245362

Task 22245362

Name oifs_43r3_ps_0329_2021050100_123_945_12163418_0
Workunit 12163418
Created 28 Nov 2022, 20:02:33 UTC
Sent 29 Nov 2022, 3:15:31 UTC
Report deadline 29 Dec 2022, 3:15:31 UTC
Received 29 Nov 2022, 18:25:33 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 5 (0x00000005) Unknown error code
Computer ID 1536300
Run time 1 hours 4 min 23 sec
CPU time 26 min 38 sec
Validate state Invalid
Credit 241.92
Device peak FLOPS 1.00 GFLOPS
Application version OpenIFS 43r3 Perturbed Surface v1.01
x86_64-pc-linux-gnu
Peak working set size 4,402.07 MB
Peak swap size 4,974.12 MB
Peak disk usage 418.11 MB
Stderr
<core_client_version>7.16.6</core_client_version>
<![CDATA[
<message>
process exited with code 5 (0x5, -251)</message>
<stderr_txt>
172224:1669742544:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ubuntu:1:1:21403:21403] [20221129:172224:1669742544:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:ubuntu:1:1:21403:21403] [20221129:172224:1669742544:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ubuntu:1:1:21403:21403] [20221129:172224:1669742544:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ubuntu:1:1:21403:21403] [20221129:172224:1669742544:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ubuntu:1:1:21403:21403] [20221129:172224:1669742544:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:ubuntu:1:1:21403:21403] [20221129:172224:1669742544:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:ubuntu:1:1:21403:21403] [20221129:172224:1669742544:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:ubuntu:1:1:21403:21403] [20221129:172224:1669742544:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Quit request received from the BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.01_x86_64-pc-linux-gnu
(argv1) start_date: 2021050100
(argv2) exptid: hpi1
(argv3) unique_member_id: 0329
(argv4) batchid: 945
(argv5) wuid: 12163418
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/4
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.01
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163418
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.01_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/4/oifs_43r3_ps_app_1.01_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/4/oifs_43r3_ps_app_1.01_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_c7a05da7c9f6f45461081d09ed8de3f7 to: /var/lib/boinc-client/slots/4/oifs_43r3_ps_0329_2021050100_123_945_12163418.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/4/oifs_43r3_ps_0329_2021050100_123_945_12163418.zip
ic_ancil_file: ic_ancil_12163418
ifsdata_file: ifsdata_12163418
climate_data_file: clim_data_12163418
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
Copying IC ancils from: ../../projects/climateprediction.net/jf_ea0835a83c6cdf62481cfefa1de2dfba to: /var/lib/boinc-client/slots/4/ic_ancil_12163418.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/4/ic_ancil_12163418.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_20cb3334069d94a7780e0fcf92600102 to: /var/lib/boinc-client/slots/4/ifsdata/ifsdata_12163418.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/4/ifsdata/ifsdata_12163418.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/4/159l_2/clim_data_12163418.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/4/159l_2/clim_data_12163418.zip
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0329_2021050100_123_945_12163418_0_r1150544372
The child process has been launched with process id: 21419
Executing the command: /var/lib/boinc-client/slots/4/./master.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:ubuntu:1:1:21419:21419] [20221129:172355:1669742635:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:ubuntu:1:1:21419:21419] [20221129:172355:1669742635:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:ubuntu:1:1:21419:21419] [20221129:172355:1669742635:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:ubuntu:1:1:21419:21419] [20221129:172355:1669742635:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:ubuntu:1:1:21419:21419] [20221129:172355:1669742635:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:ubuntu:1:1:21419:21419] [20221129:172355:1669742635:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:ubuntu:1:1:21419:21419] [20221129:172355:1669742635:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:ubuntu:1:1:21419:21419] [20221129:172355:1669742635:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:ubuntu:1:1:21419:21419] [20221129:172355:1669742635:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:ubuntu:1:1:21419:21419] [20221129:172355:1669742635:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:ubuntu:1:1:21419:21419] [20221129:172355:1669742635:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:ubuntu:1:1:21419:21419] [20221129:172355:1669742635:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:ubuntu:1:1:21419:21419] [20221129:172355:1669742635:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:ubuntu:1:1:21419:21419] [20221129:172355:1669742635:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:ubuntu:1:1:21419:21419] [20221129:172355:1669742635:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:ubuntu:1:1:21419:21419] [20221129:172355:1669742635:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ubuntu:1:1:21419:21419] [20221129:172355:1669742635:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:ubuntu:1:1:21419:21419] [20221129:172355:1669742635:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ubuntu:1:1:21419:21419] [20221129:172355:1669742635:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:ubuntu:1:1:21419:21419] [20221129:172355:1669742635:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:ubuntu:1:1:21419:21419] [20221129:172355:1669742635:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ubuntu:1:1:21419:21419] [20221129:172355:1669742635:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ubuntu:1:1:21419:21419] [20221129:172355:1669742635:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:ubuntu:1:1:21419:21419] [20221129:172355:1669742635:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ubuntu:1:1:21419:21419] [20221129:172355:1669742635:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ubuntu:1:1:21419:21419] [20221129:172355:1669742635:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ubuntu:1:1:21419:21419] [20221129:172355:1669742635:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:ubuntu:1:1:21419:21419] [20221129:172355:1669742635:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:ubuntu:1:1:21419:21419] [20221129:172355:1669742635:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:ubuntu:1:1:21419:21419] [20221129:172355:1669742635:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.01_x86_64-pc-linux-gnu
(argv1) start_date: 2021050100
(argv2) exptid: hpi1
(argv3) unique_member_id: 0329
(argv4) batchid: 945
(argv5) wuid: 12163418
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/4
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.01
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163418
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.01_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/4/oifs_43r3_ps_app_1.01_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/4/oifs_43r3_ps_app_1.01_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_c7a05da7c9f6f45461081d09ed8de3f7 to: /var/lib/boinc-client/slots/4/oifs_43r3_ps_0329_2021050100_123_945_12163418.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/4/oifs_43r3_ps_0329_2021050100_123_945_12163418.zip
ic_ancil_file: ic_ancil_12163418
ifsdata_file: ifsdata_12163418
climate_data_file: clim_data_12163418
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
Copying IC ancils from: ../../projects/climateprediction.net/jf_ea0835a83c6cdf62481cfefa1de2dfba to: /var/lib/boinc-client/slots/4/ic_ancil_12163418.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/4/ic_ancil_12163418.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_20cb3334069d94a7780e0fcf92600102 to: /var/lib/boinc-client/slots/4/ifsdata/ifsdata_12163418.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/4/ifsdata/ifsdata_12163418.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/4/159l_2/clim_data_12163418.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/4/159l_2/clim_data_12163418.zip
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0329_2021050100_123_945_12163418_0_r1150544372
The child process has been launched with process id: 21425
Executing the command: /var/lib/boinc-client/slots/4/./master.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:ubuntu:1:1:21425:21425] [20221129:172527:1669742727:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:ubuntu:1:1:21425:21425] [20221129:172527:1669742727:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:ubuntu:1:1:21425:21425] [20221129:172527:1669742727:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:ubuntu:1:1:21425:21425] [20221129:172527:1669742727:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:ubuntu:1:1:21425:21425] [20221129:172527:1669742727:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:ubuntu:1:1:21425:21425] [20221129:172527:1669742727:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:ubuntu:1:1:21425:21425] [20221129:172527:1669742727:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:ubuntu:1:1:21425:21425] [20221129:172527:1669742727:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:ubuntu:1:1:21425:21425] [20221129:172527:1669742727:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:ubuntu:1:1:21425:21425] [20221129:172527:1669742727:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:ubuntu:1:1:21425:21425] [20221129:172527:1669742727:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:ubuntu:1:1:21425:21425] [20221129:172527:1669742727:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:ubuntu:1:1:21425:21425] [20221129:172527:1669742727:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:ubuntu:1:1:21425:21425] [20221129:172527:1669742727:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:ubuntu:1:1:21425:21425] [20221129:172527:1669742727:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:ubuntu:1:1:21425:21425] [20221129:172527:1669742727:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ubuntu:1:1:21425:21425] [20221129:172527:1669742727:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:ubuntu:1:1:21425:21425] [20221129:172527:1669742727:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ubuntu:1:1:21425:21425] [20221129:172527:1669742727:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:ubuntu:1:1:21425:21425] [20221129:172527:1669742727:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:ubuntu:1:1:21425:21425] [20221129:172527:1669742727:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ubuntu:1:1:21425:21425] [20221129:172527:1669742727:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ubuntu:1:1:21425:21425] [20221129:172527:1669742727:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:ubuntu:1:1:21425:21425] [20221129:172527:1669742727:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ubuntu:1:1:21425:21425] [20221129:172527:1669742727:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ubuntu:1:1:21425:21425] [20221129:172527:1669742727:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ubuntu:1:1:21425:21425] [20221129:172527:1669742727:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:ubuntu:1:1:21425:21425] [20221129:172527:1669742727:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:ubuntu:1:1:21425:21425] [20221129:172527:1669742727:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:ubuntu:1:1:21425:21425] [20221129:172527:1669742727:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.01_x86_64-pc-linux-gnu
(argv1) start_date: 2021050100
(argv2) exptid: hpi1
(argv3) unique_member_id: 0329
(argv4) batchid: 945
(argv5) wuid: 12163418
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/4
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.01
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163418
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.01_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/4/oifs_43r3_ps_app_1.01_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/4/oifs_43r3_ps_app_1.01_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_c7a05da7c9f6f45461081d09ed8de3f7 to: /var/lib/boinc-client/slots/4/oifs_43r3_ps_0329_2021050100_123_945_12163418.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/4/oifs_43r3_ps_0329_2021050100_123_945_12163418.zip
ic_ancil_file: ic_ancil_12163418
ifsdata_file: ifsdata_12163418
climate_data_file: clim_data_12163418
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
Copying IC ancils from: ../../projects/climateprediction.net/jf_ea0835a83c6cdf62481cfefa1de2dfba to: /var/lib/boinc-client/slots/4/ic_ancil_12163418.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/4/ic_ancil_12163418.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_20cb3334069d94a7780e0fcf92600102 to: /var/lib/boinc-client/slots/4/ifsdata/ifsdata_12163418.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/4/ifsdata/ifsdata_12163418.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/4/159l_2/clim_data_12163418.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/4/159l_2/clim_data_12163418.zip
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0329_2021050100_123_945_12163418_0_r1150544372
The child process has been launched with process id: 21429
Executing the command: /var/lib/boinc-client/slots/4/./master.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:ubuntu:1:1:21429:21429] [20221129:172648:1669742808:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:ubuntu:1:1:21429:21429] [20221129:172648:1669742808:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:ubuntu:1:1:21429:21429] [20221129:172648:1669742808:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:ubuntu:1:1:21429:21429] [20221129:172648:1669742808:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:ubuntu:1:1:21429:21429] [20221129:172648:1669742808:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:ubuntu:1:1:21429:21429] [20221129:172648:1669742808:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:ubuntu:1:1:21429:21429] [20221129:172648:1669742808:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:ubuntu:1:1:21429:21429] [20221129:172648:1669742808:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:ubuntu:1:1:21429:21429] [20221129:172648:1669742808:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:ubuntu:1:1:21429:21429] [20221129:172648:1669742808:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:ubuntu:1:1:21429:21429] [20221129:172648:1669742808:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:ubuntu:1:1:21429:21429] [20221129:172648:1669742808:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:ubuntu:1:1:21429:21429] [20221129:172648:1669742808:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:ubuntu:1:1:21429:21429] [20221129:172648:1669742808:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:ubuntu:1:1:21429:21429] [20221129:172648:1669742808:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:ubuntu:1:1:21429:21429] [20221129:172648:1669742808:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ubuntu:1:1:21429:21429] [20221129:172648:1669742808:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:ubuntu:1:1:21429:21429] [20221129:172648:1669742808:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ubuntu:1:1:21429:21429] [20221129:172648:1669742808:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:ubuntu:1:1:21429:21429] [20221129:172648:1669742808:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:ubuntu:1:1:21429:21429] [20221129:172648:1669742808:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ubuntu:1:1:21429:21429] [20221129:172648:1669742808:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ubuntu:1:1:21429:21429] [20221129:172648:1669742808:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:ubuntu:1:1:21429:21429] [20221129:172648:1669742808:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ubuntu:1:1:21429:21429] [20221129:172648:1669742808:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ubuntu:1:1:21429:21429] [20221129:172648:1669742808:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ubuntu:1:1:21429:21429] [20221129:172648:1669742808:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:ubuntu:1:1:21429:21429] [20221129:172648:1669742808:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:ubuntu:1:1:21429:21429] [20221129:172648:1669742808:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:ubuntu:1:1:21429:21429] [20221129:172648:1669742808:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.01_x86_64-pc-linux-gnu
(argv1) start_date: 2021050100
(argv2) exptid: hpi1
(argv3) unique_member_id: 0329
(argv4) batchid: 945
(argv5) wuid: 12163418
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/4
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.01
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163418
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.01_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/4/oifs_43r3_ps_app_1.01_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/4/oifs_43r3_ps_app_1.01_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_c7a05da7c9f6f45461081d09ed8de3f7 to: /var/lib/boinc-client/slots/4/oifs_43r3_ps_0329_2021050100_123_945_12163418.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/4/oifs_43r3_ps_0329_2021050100_123_945_12163418.zip
ic_ancil_file: ic_ancil_12163418
ifsdata_file: ifsdata_12163418
climate_data_file: clim_data_12163418
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
Copying IC ancils from: ../../projects/climateprediction.net/jf_ea0835a83c6cdf62481cfefa1de2dfba to: /var/lib/boinc-client/slots/4/ic_ancil_12163418.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/4/ic_ancil_12163418.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_20cb3334069d94a7780e0fcf92600102 to: /var/lib/boinc-client/slots/4/ifsdata/ifsdata_12163418.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/4/ifsdata/ifsdata_12163418.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/4/159l_2/clim_data_12163418.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/4/159l_2/clim_data_12163418.zip
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0329_2021050100_123_945_12163418_0_r1150544372
The child process has been launched with process id: 21444
Executing the command: /var/lib/boinc-client/slots/4/./master.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:ubuntu:1:1:21444:21444] [20221129:172819:1669742899:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:ubuntu:1:1:21444:21444] [20221129:172819:1669742899:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:ubuntu:1:1:21444:21444] [20221129:172819:1669742899:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:ubuntu:1:1:21444:21444] [20221129:172819:1669742899:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:ubuntu:1:1:21444:21444] [20221129:172819:1669742899:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:ubuntu:1:1:21444:21444] [20221129:172819:1669742899:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:ubuntu:1:1:21444:21444] [20221129:172819:1669742899:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:ubuntu:1:1:21444:21444] [20221129:172819:1669742899:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:ubuntu:1:1:21444:21444] [20221129:172819:1669742899:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:ubuntu:1:1:21444:21444] [20221129:172819:1669742899:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:ubuntu:1:1:21444:21444] [20221129:172819:1669742899:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:ubuntu:1:1:21444:21444] [20221129:172819:1669742899:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:ubuntu:1:1:21444:21444] [20221129:172819:1669742899:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:ubuntu:1:1:21444:21444] [20221129:172819:1669742899:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:ubuntu:1:1:21444:21444] [20221129:172819:1669742899:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:ubuntu:1:1:21444:21444] [20221129:172819:1669742899:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ubuntu:1:1:21444:21444] [20221129:172819:1669742899:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:ubuntu:1:1:21444:21444] [20221129:172819:1669742899:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ubuntu:1:1:21444:21444] [20221129:172819:1669742899:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:ubuntu:1:1:21444:21444] [20221129:172819:1669742899:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:ubuntu:1:1:21444:21444] [20221129:172819:1669742899:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ubuntu:1:1:21444:21444] [20221129:172819:1669742899:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ubuntu:1:1:21444:21444] [20221129:172819:1669742899:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:ubuntu:1:1:21444:21444] [20221129:172819:1669742899:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ubuntu:1:1:21444:21444] [20221129:172819:1669742899:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ubuntu:1:1:21444:21444] [20221129:172819:1669742899:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ubuntu:1:1:21444:21444] [20221129:172819:1669742899:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:ubuntu:1:1:21444:21444] [20221129:172819:1669742899:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:ubuntu:1:1:21444:21444] [20221129:172819:1669742899:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:ubuntu:1:1:21444:21444] [20221129:172819:1669742899:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.01_x86_64-pc-linux-gnu
(argv1) start_date: 2021050100
(argv2) exptid: hpi1
(argv3) unique_member_id: 0329
(argv4) batchid: 945
(argv5) wuid: 12163418
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/4
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.01
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163418
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.01_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/4/oifs_43r3_ps_app_1.01_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/4/oifs_43r3_ps_app_1.01_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_c7a05da7c9f6f45461081d09ed8de3f7 to: /var/lib/boinc-client/slots/4/oifs_43r3_ps_0329_2021050100_123_945_12163418.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/4/oifs_43r3_ps_0329_2021050100_123_945_12163418.zip
ic_ancil_file: ic_ancil_12163418
ifsdata_file: ifsdata_12163418
climate_data_file: clim_data_12163418
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
Copying IC ancils from: ../../projects/climateprediction.net/jf_ea0835a83c6cdf62481cfefa1de2dfba to: /var/lib/boinc-client/slots/4/ic_ancil_12163418.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/4/ic_ancil_12163418.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_20cb3334069d94a7780e0fcf92600102 to: /var/lib/boinc-client/slots/4/ifsdata/ifsdata_12163418.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/4/ifsdata/ifsdata_12163418.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/4/159l_2/clim_data_12163418.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/4/159l_2/clim_data_12163418.zip
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0329_2021050100_123_945_12163418_0_r1150544372
The child process has been launched with process id: 21452
Executing the command: /var/lib/boinc-client/slots/4/./master.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:ubuntu:1:1:21452:21452] [20221129:172950:1669742990:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:ubuntu:1:1:21452:21452] [20221129:172950:1669742990:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:ubuntu:1:1:21452:21452] [20221129:172950:1669742990:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:ubuntu:1:1:21452:21452] [20221129:172950:1669742990:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:ubuntu:1:1:21452:21452] [20221129:172950:1669742990:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:ubuntu:1:1:21452:21452] [20221129:172950:1669742990:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:ubuntu:1:1:21452:21452] [20221129:172950:1669742990:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:ubuntu:1:1:21452:21452] [20221129:172950:1669742990:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:ubuntu:1:1:21452:21452] [20221129:172950:1669742990:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:ubuntu:1:1:21452:21452] [20221129:172950:1669742990:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:ubuntu:1:1:21452:21452] [20221129:172950:1669742990:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:ubuntu:1:1:21452:21452] [20221129:172950:1669742990:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:ubuntu:1:1:21452:21452] [20221129:172950:1669742990:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:ubuntu:1:1:21452:21452] [20221129:172950:1669742990:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:ubuntu:1:1:21452:21452] [20221129:172950:1669742990:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:ubuntu:1:1:21452:21452] [20221129:172950:1669742990:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ubuntu:1:1:21452:21452] [20221129:172950:1669742990:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:ubuntu:1:1:21452:21452] [20221129:172950:1669742990:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ubuntu:1:1:21452:21452] [20221129:172950:1669742990:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:ubuntu:1:1:21452:21452] [20221129:172950:1669742990:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:ubuntu:1:1:21452:21452] [20221129:172950:1669742990:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ubuntu:1:1:21452:21452] [20221129:172950:1669742990:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ubuntu:1:1:21452:21452] [20221129:172950:1669742990:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:ubuntu:1:1:21452:21452] [20221129:172950:1669742990:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ubuntu:1:1:21452:21452] [20221129:172950:1669742990:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ubuntu:1:1:21452:21452] [20221129:172950:1669742990:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ubuntu:1:1:21452:21452] [20221129:172950:1669742990:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:ubuntu:1:1:21452:21452] [20221129:172950:1669742990:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:ubuntu:1:1:21452:21452] [20221129:172950:1669742990:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:ubuntu:1:1:21452:21452] [20221129:172950:1669742990:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:30:22 STEP    0 H=   0:00 +CPU= 28.231
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhpi1+000000
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhpi1+000000
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhpi1+000000
  17:30:42 STEP    1 H=   1:00 +CPU= 20.162
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:31:02 STEP    2 H=   2:00 +CPU= 17.760
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:31:33 STEP    3 H=   3:00 +CPU= 26.878
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:31:53 STEP    4 H=   4:00 +CPU= 17.649
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:32:12 STEP    5 H=   5:00 +CPU= 17.295
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:32:42 STEP    6 H=   6:00 +CPU= 27.369
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:33:01 STEP    7 H=   7:00 +CPU= 17.195
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:33:21 STEP    8 H=   8:00 +CPU= 17.661
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:33:53 STEP    9 H=   9:00 +CPU= 27.516
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:34:12 STEP   10 H=  10:00 +CPU= 17.336
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:34:32 STEP   11 H=  11:00 +CPU= 17.265
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:35:06 STEP   12 H=  12:00 +CPU= 31.264
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhpi1+000012
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhpi1+000012
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhpi1+000012
  17:35:25 STEP   13 H=  13:00 +CPU= 17.299
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:35:45 STEP   14 H=  14:00 +CPU= 17.682
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:36:16 STEP   15 H=  15:00 +CPU= 26.650
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:36:37 STEP   16 H=  16:00 +CPU= 18.864
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:36:56 STEP   17 H=  17:00 +CPU= 17.292
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:37:26 STEP   18 H=  18:00 +CPU= 27.093
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:37:46 STEP   19 H=  19:00 +CPU= 17.845
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:38:06 STEP   20 H=  20:00 +CPU= 18.099
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:38:36 STEP   21 H=  21:00 +CPU= 26.395
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:38:56 STEP   22 H=  22:00 +CPU= 17.272
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:39:17 STEP   23 H=  23:00 +CPU= 19.230
  17:39:48 STEP   24 H=  24:00 +CPU= 30.642
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhpi1+000024
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhpi1+000024
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhpi1+000024
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163418/ICMGGhpi1+000000
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163418/ICMSHhpi1+000000
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163418/ICMUAhpi1+000000
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163418/ICMGGhpi1+000012
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163418/ICMSHhpi1+000012
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163418/ICMUAhpi1+000012
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0329_2021050100_123_945_12163418_0_r1150544372_0.zip
Uploading the intermediate file: upload_file_0.zip
  17:40:06 STEP   25 H=  25:00 +CPU= 17.003
Uploading trickle at timestep: 86400
  17:40:24 STEP   26 H=  26:00 +CPU= 17.735
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:40:57 STEP   27 H=  27:00 +CPU= 28.260
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:41:16 STEP   28 H=  28:00 +CPU= 17.480
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:41:36 STEP   29 H=  29:00 +CPU= 17.277
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:42:07 STEP   30 H=  30:00 +CPU= 28.858
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:42:26 STEP   31 H=  31:00 +CPU= 16.825
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:42:46 STEP   32 H=  32:00 +CPU= 17.709
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:43:15 STEP   33 H=  33:00 +CPU= 26.993
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:43:37 STEP   34 H=  34:00 +CPU= 19.202
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:43:56 STEP   35 H=  35:00 +CPU= 17.276
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:44:27 STEP   36 H=  36:00 +CPU= 29.117
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhpi1+000036
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhpi1+000036
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhpi1+000036
  17:44:49 STEP   37 H=  37:00 +CPU= 18.761
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:45:08 STEP   38 H=  38:00 +CPU= 17.677
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:45:39 STEP   39 H=  39:00 +CPU= 26.344
  17:45:57 STEP   40 H=  40:00 +CPU= 18.080
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:46:18 STEP   41 H=  41:00 +CPU= 18.926
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:46:48 STEP   42 H=  42:00 +CPU= 27.188
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:47:07 STEP   43 H=  43:00 +CPU= 16.905
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:47:27 STEP   44 H=  44:00 +CPU= 17.877
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:47:59 STEP   45 H=  45:00 +CPU= 27.441
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:48:18 STEP   46 H=  46:00 +CPU= 17.395
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:48:38 STEP   47 H=  47:00 +CPU= 17.535
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:49:15 STEP   48 H=  48:00 +CPU= 32.431
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhpi1+000048
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhpi1+000048
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhpi1+000048
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163418/ICMGGhpi1+000024
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163418/ICMSHhpi1+000024
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163418/ICMUAhpi1+000024
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163418/ICMGGhpi1+000036
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163418/ICMSHhpi1+000036
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163418/ICMUAhpi1+000036
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0329_2021050100_123_945_12163418_0_r1150544372_1.zip
Uploading the intermediate file: upload_file_1.zip
  17:49:31 STEP   49 H=  49:00 +CPU= 16.757
Uploading trickle at timestep: 172800
  17:49:49 STEP   50 H=  50:00 +CPU= 17.559
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:50:21 STEP   51 H=  51:00 +CPU= 27.698
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:50:42 STEP   52 H=  52:00 +CPU= 18.290
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:51:01 STEP   53 H=  53:00 +CPU= 17.016
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:51:32 STEP   54 H=  54:00 +CPU= 27.293
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:51:53 STEP   55 H=  55:00 +CPU= 18.597
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:52:14 STEP   56 H=  56:00 +CPU= 18.636
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:52:43 STEP   57 H=  57:00 +CPU= 26.539
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:53:03 STEP   58 H=  58:00 +CPU= 17.585
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:53:24 STEP   59 H=  59:00 +CPU= 18.655
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:53:58 STEP   60 H=  60:00 +CPU= 30.001
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhpi1+000060
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhpi1+000060
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhpi1+000060
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:54:17 STEP   61 H=  61:00 +CPU= 16.871
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:54:37 STEP   62 H=  62:00 +CPU= 17.565
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:55:07 STEP   63 H=  63:00 +CPU= 27.871
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:55:26 STEP   64 H=  64:00 +CPU= 17.326
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:55:46 STEP   65 H=  65:00 +CPU= 17.107
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:56:19 STEP   66 H=  66:00 +CPU= 28.618
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:56:39 STEP   67 H=  67:00 +CPU= 17.977
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:56:58 STEP   68 H=  68:00 +CPU= 17.538
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:57:27 STEP   69 H=  69:00 +CPU= 26.308
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:57:48 STEP   70 H=  70:00 +CPU= 19.158
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:58:09 STEP   71 H=  71:00 +CPU= 18.191
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:58:43 STEP   72 H=  72:00 +CPU= 30.666
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhpi1+000072
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhpi1+000072
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhpi1+000072
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163418/ICMGGhpi1+000048
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163418/ICMSHhpi1+000048
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163418/ICMUAhpi1+000048
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163418/ICMGGhpi1+000060
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163418/ICMSHhpi1+000060
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163418/ICMUAhpi1+000060
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0329_2021050100_123_945_12163418_0_r1150544372_2.zip
Uploading the intermediate file: upload_file_2.zip
  17:59:03 STEP   73 H=  73:00 +CPU= 17.103
Uploading trickle at timestep: 259200
  17:59:21 STEP   74 H=  74:00 +CPU= 18.232
double free or corruption (out)

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 May 2023 15:03:31 1536300 22245362 oifs_43r3_ps_0329_2021050100_123_945_12163418_0 259,200 1,565 0.0060
09 May 2023 15:03:31 1536300 22245362 oifs_43r3_ps_0329_2021050100_123_945_12163418_0 172,800 1,057 0.0061
09 May 2023 15:03:31 1536300 22245362 oifs_43r3_ps_0329_2021050100_123_945_12163418_0 86,400 539 0.0062


©2024 climateprediction.net