<core_client_version>7.18.1</core_client_version>
<![CDATA[
<message>
Disk usage limit exceeded</message>
<stderr_txt>
s_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 1999050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0487
(argv4) batchid: 968
(argv5) wuid: 12185131
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/3
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185131
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/3/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/3/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_4ce993dbc6c729bac87aa4c2d121ea46 to: /var/lib/boinc-client/slots/3/oifs_43r3_ps_0487_1999050100_123_968_12185131.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/3/oifs_43r3_ps_0487_1999050100_123_968_12185131.zip
ic_ancil_file: ic_ancil_12185131
ifsdata_file: ifsdata_12185131
climate_data_file: clim_data_12185131
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_4db807956b9124aab14c971849aa9e07 to: /var/lib/boinc-client/slots/3/ic_ancil_12185131.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/3/ic_ancil_12185131.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_522568e51133232bb15f9efbb4689cf0 to: /var/lib/boinc-client/slots/3/ifsdata/ifsdata_12185131.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/3/ifsdata/ifsdata_12185131.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/3/159l_2/clim_data_12185131.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/3/159l_2/clim_data_12185131.zip
Checking for progress XML file: /var/lib/boinc-client/slots/3/progress_file_12185131.xml
Opened progress file ok : /var/lib/boinc-client/slots/3/progress_file_12185131.xml
-- Model is restarting --
Adjusting last_iter, 1554, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/3/progress_file_12185131.xml
last_cpu_time: 35856
upload_file_number: 64
last_iter: 1535
last_upload: 5529600
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0487_1999050100_123_968_12185131_0_r1170178595
The child process has been launched with process id: 2870
Executing the command: /var/lib/boinc-client/slots/3/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:obata-H87-HD3:1:1:2870:2870] [20230121:174938:1674319778:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:obata-H87-HD3:1:1:2870:2870] [20230121:174938:1674319778:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:obata-H87-HD3:1:1:2870:2870] [20230121:174938:1674319778:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:obata-H87-HD3:1:1:2870:2870] [20230121:174938:1674319778:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:obata-H87-HD3:1:1:2870:2870] [20230121:174938:1674319778:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:obata-H87-HD3:1:1:2870:2870] [20230121:174938:1674319778:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647)
[EC_DRHOOK:obata-H87-HD3:1:1:2870:2870] [20230121:174938:1674319778:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:obata-H87-HD3:1:1:2870:2870] [20230121:174938:1674319778:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:obata-H87-HD3:1:1:2870:2870] [20230121:174938:1674319778:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:obata-H87-HD3:1:1:2870:2870] [20230121:174938:1674319778:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:obata-H87-HD3:1:1:2870:2870] [20230121:174938:1674319778:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:obata-H87-HD3:1:1:2870:2870] [20230121:174938:1674319778:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:obata-H87-HD3:1:1:2870:2870] [20230121:174938:1674319778:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:obata-H87-HD3:1:1:2870:2870] [20230121:174938:1674319778:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:obata-H87-HD3:1:1:2870:2870] [20230121:174938:1674319778:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:obata-H87-HD3:1:1:2870:2870] [20230121:174938:1674319778:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:obata-H87-HD3:1:1:2870:2870] [20230121:174938:1674319778:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:obata-H87-HD3:1:1:2870:2870] [20230121:174938:1674319778:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:obata-H87-HD3:1:1:2870:2870] [20230121:174938:1674319778:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:obata-H87-HD3:1:1:2870:2870] [20230121:174938:1674319778:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:obata-H87-HD3:1:1:2870:2870] [20230121:174938:1674319778:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:obata-H87-HD3:1:1:2870:2870] [20230121:174938:1674319778:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:obata-H87-HD3:1:1:2870:2870] [20230121:174938:1674319778:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:obata-H87-HD3:1:1:2870:2870] [20230121:174938:1674319778:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:obata-H87-HD3:1:1:2870:2870] [20230121:174938:1674319778:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:obata-H87-HD3:1:1:2870:2870] [20230121:174938:1674319778:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:obata-H87-HD3:1:1:2870:2870] [20230121:174938:1674319778:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:obata-H87-HD3:1:1:2870:2870] [20230121:174938:1674319778:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:obata-H87-HD3:1:1:2870:2870] [20230121:174938:1674319778:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:obata-H87-HD3:1:1:2870:2870] [20230121:174938:1674319778:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
17:50:20 STEP 1536 H=1536:00 +CPU= 32.240
17:50:38 STEP 1537 H=1537:00 +CPU= 18.364
17:50:56 STEP 1538 H=1538:00 +CPU= 18.397
17:51:25 STEP 1539 H=1539:00 +CPU= 28.234
17:51:43 STEP 1540 H=1540:00 +CPU= 18.385
17:52:01 STEP 1541 H=1541:00 +CPU= 17.870
17:52:30 STEP 1542 H=1542:00 +CPU= 28.907
17:52:48 STEP 1543 H=1543:00 +CPU= 17.871
17:53:08 STEP 1544 H=1544:00 +CPU= 19.315
17:53:37 STEP 1545 H=1545:00 +CPU= 29.203
17:53:56 STEP 1546 H=1546:00 +CPU= 18.668
17:54:15 STEP 1547 H=1547:00 +CPU= 18.270
17:54:47 STEP 1548 H=1548:00 +CPU= 31.628
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+001548
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+001548
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+001548
17:55:05 STEP 1549 H=1549:00 +CPU= 18.305
17:55:24 STEP 1550 H=1550:00 +CPU= 18.998
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
17:56:05 STEP 1551 H=1551:00 +CPU= 29.458
17:56:24 STEP 1552 H=1552:00 +CPU= 18.938
17:56:42 STEP 1553 H=1553:00 +CPU= 17.430
17:57:10 STEP 1554 H=1554:00 +CPU= 27.742
17:57:27 STEP 1555 H=1555:00 +CPU= 16.942
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 1999050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0487
(argv4) batchid: 968
(argv5) wuid: 12185131
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/3
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185131
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/3/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/3/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_4ce993dbc6c729bac87aa4c2d121ea46 to: /var/lib/boinc-client/slots/3/oifs_43r3_ps_0487_1999050100_123_968_12185131.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/3/oifs_43r3_ps_0487_1999050100_123_968_12185131.zip
ic_ancil_file: ic_ancil_12185131
ifsdata_file: ifsdata_12185131
climate_data_file: clim_data_12185131
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_4db807956b9124aab14c971849aa9e07 to: /var/lib/boinc-client/slots/3/ic_ancil_12185131.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/3/ic_ancil_12185131.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_522568e51133232bb15f9efbb4689cf0 to: /var/lib/boinc-client/slots/3/ifsdata/ifsdata_12185131.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/3/ifsdata/ifsdata_12185131.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/3/159l_2/clim_data_12185131.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/3/159l_2/clim_data_12185131.zip
Checking for progress XML file: /var/lib/boinc-client/slots/3/progress_file_12185131.xml
Opened progress file ok : /var/lib/boinc-client/slots/3/progress_file_12185131.xml
-- Model is restarting --
Adjusting last_iter, 1556, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/3/progress_file_12185131.xml
last_cpu_time: 36305
upload_file_number: 64
last_iter: 1535
last_upload: 5529600
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0487_1999050100_123_968_12185131_0_r1170178595
The child process has been launched with process id: 2906
Executing the command: /var/lib/boinc-client/slots/3/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:obata-H87-HD3:1:1:2906:2906] [20230122:184705:1674409625:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:obata-H87-HD3:1:1:2906:2906] [20230122:184705:1674409625:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:obata-H87-HD3:1:1:2906:2906] [20230122:184705:1674409625:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:obata-H87-HD3:1:1:2906:2906] [20230122:184705:1674409625:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:obata-H87-HD3:1:1:2906:2906] [20230122:184705:1674409625:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:obata-H87-HD3:1:1:2906:2906] [20230122:184705:1674409625:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647)
[EC_DRHOOK:obata-H87-HD3:1:1:2906:2906] [20230122:184705:1674409625:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:obata-H87-HD3:1:1:2906:2906] [20230122:184705:1674409625:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:obata-H87-HD3:1:1:2906:2906] [20230122:184705:1674409625:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:obata-H87-HD3:1:1:2906:2906] [20230122:184705:1674409625:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:obata-H87-HD3:1:1:2906:2906] [20230122:184705:1674409625:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:obata-H87-HD3:1:1:2906:2906] [20230122:184705:1674409625:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:obata-H87-HD3:1:1:2906:2906] [20230122:184705:1674409625:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:obata-H87-HD3:1:1:2906:2906] [20230122:184705:1674409625:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:obata-H87-HD3:1:1:2906:2906] [20230122:184705:1674409625:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:obata-H87-HD3:1:1:2906:2906] [20230122:184705:1674409625:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:obata-H87-HD3:1:1:2906:2906] [20230122:184705:1674409625:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:obata-H87-HD3:1:1:2906:2906] [20230122:184705:1674409625:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:obata-H87-HD3:1:1:2906:2906] [20230122:184705:1674409625:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:obata-H87-HD3:1:1:2906:2906] [20230122:184705:1674409625:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:obata-H87-HD3:1:1:2906:2906] [20230122:184705:1674409625:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:obata-H87-HD3:1:1:2906:2906] [20230122:184705:1674409625:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:obata-H87-HD3:1:1:2906:2906] [20230122:184705:1674409625:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:obata-H87-HD3:1:1:2906:2906] [20230122:184705:1674409625:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:obata-H87-HD3:1:1:2906:2906] [20230122:184705:1674409625:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:obata-H87-HD3:1:1:2906:2906] [20230122:184705:1674409625:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:obata-H87-HD3:1:1:2906:2906] [20230122:184705:1674409625:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:obata-H87-HD3:1:1:2906:2906] [20230122:184705:1674409625:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:obata-H87-HD3:1:1:2906:2906] [20230122:184705:1674409625:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:obata-H87-HD3:1:1:2906:2906] [20230122:184705:1674409625:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
18:47:54 STEP 1536 H=1536:00 +CPU= 31.832
18:48:15 STEP 1537 H=1537:00 +CPU= 17.717
18:48:33 STEP 1538 H=1538:00 +CPU= 18.174
18:49:01 STEP 1539 H=1539:00 +CPU= 28.027
18:49:19 STEP 1540 H=1540:00 +CPU= 18.148
18:49:37 STEP 1541 H=1541:00 +CPU= 17.638
18:50:06 STEP 1542 H=1542:00 +CPU= 28.809
18:50:24 STEP 1543 H=1543:00 +CPU= 17.829
18:50:42 STEP 1544 H=1544:00 +CPU= 18.294
18:51:11 STEP 1545 H=1545:00 +CPU= 28.315
18:51:29 STEP 1546 H=1546:00 +CPU= 18.259
18:51:47 STEP 1547 H=1547:00 +CPU= 17.743
18:52:19 STEP 1548 H=1548:00 +CPU= 31.249
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+001548
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+001548
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+001548
18:52:37 STEP 1549 H=1549:00 +CPU= 17.822
18:52:55 STEP 1550 H=1550:00 +CPU= 18.308
18:53:23 STEP 1551 H=1551:00 +CPU= 28.223
18:53:42 STEP 1552 H=1552:00 +CPU= 18.308
18:54:00 STEP 1553 H=1553:00 +CPU= 17.751
18:54:29 STEP 1554 H=1554:00 +CPU= 28.820
18:54:47 STEP 1555 H=1555:00 +CPU= 17.754
18:55:05 STEP 1556 H=1556:00 +CPU= 18.296
18:55:33 STEP 1557 H=1557:00 +CPU= 28.150
18:55:52 STEP 1558 H=1558:00 +CPU= 18.252
18:56:10 STEP 1559 H=1559:00 +CPU= 18.297
18:56:43 STEP 1560 H=1560:00 +CPU= 32.817
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+001560
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+001560
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+001560
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185131/ICMGGhq0f+001536
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185131/ICMSHhq0f+001536
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185131/ICMUAhq0f+001536
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185131/ICMGGhq0f+001548
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185131/ICMSHhq0f+001548
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185131/ICMUAhq0f+001548
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0487_1999050100_123_968_12185131_0_r1170178595_64.zip
Uploading the intermediate file: upload_file_64.zip
18:57:01 STEP 1561 H=1561:00 +CPU= 17.832
18:57:21 STEP 1562 H=1562:00 +CPU= 18.724
18:57:51 STEP 1563 H=1563:00 +CPU= 29.231
18:58:10 STEP 1564 H=1564:00 +CPU= 18.831
18:58:28 STEP 1565 H=1565:00 +CPU= 17.888
18:58:59 STEP 1566 H=1566:00 +CPU= 29.675
18:59:18 STEP 1567 H=1567:00 +CPU= 18.100
18:59:37 STEP 1568 H=1568:00 +CPU= 18.865
19:00:08 STEP 1569 H=1569:00 +CPU= 29.235
19:00:27 STEP 1570 H=1570:00 +CPU= 18.878
19:00:45 STEP 1571 H=1571:00 +CPU= 18.164
19:01:17 STEP 1572 H=1572:00 +CPU= 31.612
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+001572
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+001572
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+001572
19:01:36 STEP 1573 H=1573:00 +CPU= 18.551
19:01:56 STEP 1574 H=1574:00 +CPU= 19.239
19:02:28 STEP 1575 H=1575:00 +CPU= 29.621
19:02:47 STEP 1576 H=1576:00 +CPU= 18.608
19:03:05 STEP 1577 H=1577:00 +CPU= 18.009
19:03:35 STEP 1578 H=1578:00 +CPU= 29.157
19:03:54 STEP 1579 H=1579:00 +CPU= 18.162
19:04:13 STEP 1580 H=1580:00 +CPU= 18.659
19:04:42 STEP 1581 H=1581:00 +CPU= 28.350
19:05:01 STEP 1582 H=1582:00 +CPU= 18.801
19:05:20 STEP 1583 H=1583:00 +CPU= 18.729
19:06:00 STEP 1584 H=1584:00 +CPU= 33.617
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+001584
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+001584
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+001584
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185131/ICMGGhq0f+001560
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185131/ICMSHhq0f+001560
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185131/ICMUAhq0f+001560
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185131/ICMGGhq0f+001572
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185131/ICMSHhq0f+001572
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185131/ICMUAhq0f+001572
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0487_1999050100_123_968_12185131_0_r1170178595_65.zip
Uploading the intermediate file: upload_file_65.zip
19:06:20 STEP 1585 H=1585:00 +CPU= 18.846
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
19:07:03 STEP 1586 H=1586:00 +CPU= 19.598
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
19:07:47 STEP 1587 H=1587:00 +CPU= 30.144
19:08:08 STEP 1588 H=1588:00 +CPU= 19.499
19:08:29 STEP 1589 H=1589:00 +CPU= 19.008
19:09:02 STEP 1590 H=1590:00 +CPU= 30.637
19:09:22 STEP 1591 H=1591:00 +CPU= 19.055
19:09:44 STEP 1592 H=1592:00 +CPU= 19.548
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
19:10:29 STEP 1593 H=1593:00 +CPU= 30.154
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
19:11:02 STEP 1594 H=1594:00 +CPU= 19.512
19:11:23 STEP 1595 H=1595:00 +CPU= 19.186
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 1999050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0487
(argv4) batchid: 968
(argv5) wuid: 12185131
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/3
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185131
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/3/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/3/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_4ce993dbc6c729bac87aa4c2d121ea46 to: /var/lib/boinc-client/slots/3/oifs_43r3_ps_0487_1999050100_123_968_12185131.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/3/oifs_43r3_ps_0487_1999050100_123_968_12185131.zip
ic_ancil_file: ic_ancil_12185131
ifsdata_file: ifsdata_12185131
climate_data_file: clim_data_12185131
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_4db807956b9124aab14c971849aa9e07 to: /var/lib/boinc-client/slots/3/ic_ancil_12185131.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/3/ic_ancil_12185131.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_522568e51133232bb15f9efbb4689cf0 to: /var/lib/boinc-client/slots/3/ifsdata/ifsdata_12185131.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/3/ifsdata/ifsdata_12185131.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/3/159l_2/clim_data_12185131.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/3/159l_2/clim_data_12185131.zip
Checking for progress XML file: /var/lib/boinc-client/slots/3/progress_file_12185131.xml
Opened progress file ok : /var/lib/boinc-client/slots/3/progress_file_12185131.xml
-- Model is restarting --
Adjusting last_iter, 1596, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/3/progress_file_12185131.xml
last_cpu_time: 37667
upload_file_number: 66
last_iter: 1583
last_upload: 5702400
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0487_1999050100_123_968_12185131_0_r1170178595
The child process has been launched with process id: 21834
Executing the command: /var/lib/boinc-client/slots/3/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:obata-H87-HD3:1:1:21834:21834] [20230122:220126:1674421286:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:obata-H87-HD3:1:1:21834:21834] [20230122:220126:1674421286:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:obata-H87-HD3:1:1:21834:21834] [20230122:220126:1674421286:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:obata-H87-HD3:1:1:21834:21834] [20230122:220126:1674421286:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:obata-H87-HD3:1:1:21834:21834] [20230122:220126:1674421286:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:obata-H87-HD3:1:1:21834:21834] [20230122:220126:1674421286:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647)
[EC_DRHOOK:obata-H87-HD3:1:1:21834:21834] [20230122:220126:1674421286:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:obata-H87-HD3:1:1:21834:21834] [20230122:220126:1674421286:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:obata-H87-HD3:1:1:21834:21834] [20230122:220126:1674421286:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:obata-H87-HD3:1:1:21834:21834] [20230122:220126:1674421286:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:obata-H87-HD3:1:1:21834:21834] [20230122:220126:1674421286:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:obata-H87-HD3:1:1:21834:21834] [20230122:220126:1674421286:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:obata-H87-HD3:1:1:21834:21834] [20230122:220126:1674421286:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:obata-H87-HD3:1:1:21834:21834] [20230122:220126:1674421286:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:obata-H87-HD3:1:1:21834:21834] [20230122:220126:1674421286:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:obata-H87-HD3:1:1:21834:21834] [20230122:220126:1674421286:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:obata-H87-HD3:1:1:21834:21834] [20230122:220126:1674421286:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:obata-H87-HD3:1:1:21834:21834] [20230122:220126:1674421286:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:obata-H87-HD3:1:1:21834:21834] [20230122:220126:1674421286:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:obata-H87-HD3:1:1:21834:21834] [20230122:220126:1674421286:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:obata-H87-HD3:1:1:21834:21834] [20230122:220126:1674421286:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:obata-H87-HD3:1:1:21834:21834] [20230122:220126:1674421286:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:obata-H87-HD3:1:1:21834:21834] [20230122:220126:1674421286:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:obata-H87-HD3:1:1:21834:21834] [20230122:220126:1674421286:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:obata-H87-HD3:1:1:21834:21834] [20230122:220126:1674421286:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:obata-H87-HD3:1:1:21834:21834] [20230122:220126:1674421286:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:obata-H87-HD3:1:1:21834:21834] [20230122:220126:1674421286:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:obata-H87-HD3:1:1:21834:21834] [20230122:220126:1674421286:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:obata-H87-HD3:1:1:21834:21834] [20230122:220126:1674421286:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:obata-H87-HD3:1:1:21834:21834] [20230122:220126:1674421286:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:13:34 STEP 1584 H=1584:00 +CPU= 33.491
22:13:52 STEP 1585 H=1585:00 +CPU= 18.563
22:14:12 STEP 1586 H=1586:00 +CPU= 19.013
22:14:41 STEP 1587 H=1587:00 +CPU= 29.171
22:15:00 STEP 1588 H=1588:00 +CPU= 19.090
22:15:19 STEP 1589 H=1589:00 +CPU= 18.424
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:16:03 STEP 1590 H=1590:00 +CPU= 31.033
22:16:23 STEP 1591 H=1591:00 +CPU= 18.768
22:16:42 STEP 1592 H=1592:00 +CPU= 18.951
22:17:11 STEP 1593 H=1593:00 +CPU= 29.238
22:17:31 STEP 1594 H=1594:00 +CPU= 19.070
22:17:49 STEP 1595 H=1595:00 +CPU= 18.419
22:18:22 STEP 1596 H=1596:00 +CPU= 32.253
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+001596
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+001596
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+001596
22:18:41 STEP 1597 H=1597:00 +CPU= 18.545
22:19:00 STEP 1598 H=1598:00 +CPU= 19.060
22:19:30 STEP 1599 H=1599:00 +CPU= 29.364
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:20:02 STEP 1600 H=1600:00 +CPU= 19.835
22:20:22 STEP 1601 H=1601:00 +CPU= 19.094
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:21:04 STEP 1602 H=1602:00 +CPU= 30.146
22:21:23 STEP 1603 H=1603:00 +CPU= 18.391
22:21:42 STEP 1604 H=1604:00 +CPU= 18.834
22:22:13 STEP 1605 H=1605:00 +CPU= 29.795
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:22:44 STEP 1606 H=1606:00 +CPU= 19.324
22:23:03 STEP 1607 H=1607:00 +CPU= 18.893
22:23:38 STEP 1608 H=1608:00 +CPU= 33.855
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+001608
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+001608
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+001608
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185131/ICMGGhq0f+001584
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185131/ICMSHhq0f+001584
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185131/ICMUAhq0f+001584
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185131/ICMGGhq0f+001596
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185131/ICMSHhq0f+001596
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185131/ICMUAhq0f+001596
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0487_1999050100_123_968_12185131_0_r1170178595_66.zip
Uploading the intermediate file: upload_file_66.zip
22:23:57 STEP 1609 H=1609:00 +CPU= 18.417
22:24:16 STEP 1610 H=1610:00 +CPU= 18.880
22:24:45 STEP 1611 H=1611:00 +CPU= 29.206
22:25:05 STEP 1612 H=1612:00 +CPU= 18.972
22:25:23 STEP 1613 H=1613:00 +CPU= 18.450
22:25:53 STEP 1614 H=1614:00 +CPU= 29.797
22:26:12 STEP 1615 H=1615:00 +CPU= 18.411
22:26:31 STEP 1616 H=1616:00 +CPU= 19.122
22:27:01 STEP 1617 H=1617:00 +CPU= 29.310
22:27:21 STEP 1618 H=1618:00 +CPU= 19.042
22:27:40 STEP 1619 H=1619:00 +CPU= 18.631
22:28:14 STEP 1620 H=1620:00 +CPU= 32.533
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+001620
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+001620
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+001620
22:28:32 STEP 1621 H=1621:00 +CPU= 18.442
22:28:52 STEP 1622 H=1622:00 +CPU= 19.118
22:29:21 STEP 1623 H=1623:00 +CPU= 29.216
22:29:40 STEP 1624 H=1624:00 +CPU= 18.993
22:29:59 STEP 1625 H=1625:00 +CPU= 18.508
22:30:29 STEP 1626 H=1626:00 +CPU= 29.802
22:30:48 STEP 1627 H=1627:00 +CPU= 18.474
22:31:07 STEP 1628 H=1628:00 +CPU= 19.076
22:31:36 STEP 1629 H=1629:00 +CPU= 29.190
22:31:56 STEP 1630 H=1630:00 +CPU= 19.110
22:32:16 STEP 1631 H=1631:00 +CPU= 19.232
22:32:51 STEP 1632 H=1632:00 +CPU= 34.100
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+001632
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+001632
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+001632
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185131/ICMGGhq0f+001608
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185131/ICMSHhq0f+001608
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185131/ICMUAhq0f+001608
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185131/ICMGGhq0f+001620
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185131/ICMSHhq0f+001620
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185131/ICMUAhq0f+001620
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0487_1999050100_123_968_12185131_0_r1170178595_67.zip
Uploading the intermediate file: upload_file_67.zip
22:33:10 STEP 1633 H=1633:00 +CPU= 18.559
22:33:29 STEP 1634 H=1634:00 +CPU= 19.010
22:34:00 STEP 1635 H=1635:00 +CPU= 29.424
22:34:19 STEP 1636 H=1636:00 +CPU= 19.140
22:34:38 STEP 1637 H=1637:00 +CPU= 18.501
22:35:09 STEP 1638 H=1638:00 +CPU= 30.102
22:35:28 STEP 1639 H=1639:00 +CPU= 18.618
22:35:47 STEP 1640 H=1640:00 +CPU= 19.042
22:36:18 STEP 1641 H=1641:00 +CPU= 29.570
22:36:38 STEP 1642 H=1642:00 +CPU= 19.465
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:37:28 STEP 1643 H=1643:00 +CPU= 18.758
22:38:02 STEP 1644 H=1644:00 +CPU= 32.811
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+001644
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+001644
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+001644
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:38:52 STEP 1645 H=1645:00 +CPU= 18.747
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:39:24 STEP 1646 H=1646:00 +CPU= 19.428
22:39:53 STEP 1647 H=1647:00 +CPU= 29.438
22:40:13 STEP 1648 H=1648:00 +CPU= 19.263
22:40:31 STEP 1649 H=1649:00 +CPU= 18.500
22:41:02 STEP 1650 H=1650:00 +CPU= 30.023
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:44:22 STEP 1651 H=1651:00 +CPU= 18.641
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:45:04 STEP 1652 H=1652:00 +CPU= 19.242
22:45:34 STEP 1653 H=1653:00 +CPU= 29.740
22:45:53 STEP 1654 H=1654:00 +CPU= 18.965
22:46:12 STEP 1655 H=1655:00 +CPU= 19.127
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:46:59 STEP 1656 H=1656:00 +CPU= 34.551
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+001656
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+001656
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+001656
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185131/ICMGGhq0f+001632
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185131/ICMSHhq0f+001632
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185131/ICMUAhq0f+001632
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185131/ICMGGhq0f+001644
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185131/ICMSHhq0f+001644
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185131/ICMUAhq0f+001644
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0487_1999050100_123_968_12185131_0_r1170178595_68.zip
Uploading the intermediate file: upload_file_68.zip
22:47:18 STEP 1657 H=1657:00 +CPU= 18.651
22:47:39 STEP 1658 H=1658:00 +CPU= 19.799
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
[EC_DRHOOK:obata-H87-HD3:1:1:21834:21834] [20230122:224809:1674424089:2802.727] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1538] Received signal#15 (SIGTERM) :: 3996MB (heap), 4413MB (maxrss), 0MB (maxstack), 0 (paging), nsigs = 1
[EC_DRHOOK:obata-H87-HD3:1:1:21834:21834] [20230122:224809:1674424089:2802.727] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1542] Also activating Harakiri-alarm (SIGALRM=14) to expire after 500s elapsed to prevent hangs, nsigs = 1
[EC_DRHOOK:obata-H87-HD3:1:1:21834:21834] [20230122:224809:1674424089:2802.728] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1544] Harakiri signal handler 'signal_harakiri' for signal#14 (SIGALRM) installed at 0x81f3c0 (old at (nil))
[EC_DRHOOK:obata-H87-HD3:1:1:21834:21834] [20230122:224809:1674424089:2802.728] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1617] Signal#15 was caused by unrecognized si_code [memaddr=0x1], nsigs = 1
[EC_DRHOOK:obata-H87-HD3:1:1:21834:21834] [20230122:224809:1674424089:2802.728] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1686] Starting DrHook backtrace for signal#15, nsigs = 1
[EC_DRHOOK:obata-H87-HD3:1:1:21834:21834] [20230122:224809:1674424089:2802.728] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3843] 3996 MB (maxheap), 4413 MB (maxrss), 0 MB (maxstack)
[EC_DRHOOK:obata-H87-HD3:1:1:21834:21834] [20230122:224809:1674424089:2802.728] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : MASTER
[EC_DRHOOK:obata-H87-HD3:1:1:21834:21834] [20230122:224809:1674424089:2802.728] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT0
[EC_DRHOOK:obata-H87-HD3:1:1:21834:21834] [20230122:224809:1674424089:2802.728] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT1
[EC_DRHOOK:obata-H87-HD3:1:1:21834:21834] [20230122:224809:1674424089:2802.728] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT2
[EC_DRHOOK:obata-H87-HD3:1:1:21834:21834] [20230122:224809:1674424089:2802.728] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT3
[EC_DRHOOK:obata-H87-HD3:1:1:21834:21834] [20230122:224809:1674424089:2802.728] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT4
[EC_DRHOOK:obata-H87-HD3:1:1:21834:21834] [20230122:224809:1674424089:2802.728] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : STEPO
[EC_DRHOOK:obata-H87-HD3:1:1:21834:21834] [20230122:224809:1674424089:2802.728] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : SCAN2M
[EC_DRHOOK:obata-H87-HD3:1:1:21834:21834] [20230122:224809:1674424089:2802.728] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : GP_MODEL_HEAP
[EC_DRHOOK:obata-H87-HD3:1:1:21834:21834] [20230122:224809:1674424089:2802.728] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : GP_MODEL
[EC_DRHOOK:obata-H87-HD3:1:1:21834:21834] [20230122:224809:1674424089:2802.728] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : EC_PHYS_DRV
[EC_DRHOOK:obata-H87-HD3:1:1:21834:21834] [20230122:224809:1674424089:2802.728] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : >OMP-PHYSICS CLDPP T/S (1002)
[EC_DRHOOK:obata-H87-HD3:1:1:21834:21834] [20230122:224809:1674424089:2802.728] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : EC_PHYS
[EC_DRHOOK:obata-H87-HD3:1:1:21834:21834] [20230122:224809:1674424089:2802.728] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CALLPAR
[EC_DRHOOK:obata-H87-HD3:1:1:21834:21834] [20230122:224809:1674424089:2802.728] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CLOUD_LAYER
[EC_DRHOOK:obata-H87-HD3:1:1:21834:21834] [20230122:224809:1674424089:2802.728] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CLOUDSC
[EC_DRHOOK:obata-H87-HD3:1:1:21834:21834] [20230122:224810:1674424090:2803.728] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1734] DrHook backtrace done for signal#15, nsigs = 1
[EC_DRHOOK:obata-H87-HD3:1:1:21834:21834] [20230122:224810:1674424090:2803.728] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1785] Calling previous signal handler at 0x1ce8bb0 for signal#15, nsigs = 1
forrtl: error (78): process killed (SIGTERM)
Image PC Routine Line Source
oifs_43r3_model.e 0000000001CE916B Unknown Unknown Unknown
oifs_43r3_model.e 000000000081FFF1 Unknown Unknown Unknown
oifs_43r3_model.e 0000000001DC9090 Unknown Unknown Unknown
oifs_43r3_model.e 0000000001035310 cloudsc_ 3145 cloudsc.F90
oifs_43r3_model.e 0000000001014D57 cloud_layer_ 131 cloud_layer.F90
oifs_43r3_model.e 0000000000F1A790 callpar_ 1223 callpar.F90
oifs_43r3_model.e 0000000000E4940F ec_phys_ 670 ec_phys.F90
oifs_43r3_model.e 0000000000E3610C ec_phys_drv_ 599 ec_phys_drv.F90
oifs_43r3_model.e 0000000000BFA3D3 gp_model_ 613 gp_model.F90
oifs_43r3_model.e 00000000012E9482 gp_model_heap_ 74 gp_model_heap.F90
oifs_43r3_model.e 0000000000BBC1CE scan2m_ 535 scan2m.F90
oifs_43r3_model.e 000000000057A664 stepo_ 327 stepo.F90
oifs_43r3_model.e 000000000055DEEC cnt4_ 1133 cnt4.F90
oifs_43r3_model.e 00000000005471C9 cnt3_ 267 cnt3.F90
oifs_43r3_model.e 0000000000546412 cnt2_ 88 cnt2.F90
oifs_43r3_model.e 0000000000545E28 cnt1_ 92 cnt1.F90
oifs_43r3_model.e 000000000040708D cnt0_ 146 cnt0.F90
oifs_43r3_model.e 000000000040220F MAIN__ 96 master.F90
oifs_43r3_model.e 00000000004021A2 Unknown Unknown Unknown
oifs_43r3_model.e 0000000001DCA390 Unknown Unknown Unknown
oifs_43r3_model.e 000000000040206E Unknown Unknown Unknown
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 1999050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0487
(argv4) batchid: 968
(argv5) wuid: 12185131
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/3
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185131
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/3/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/3/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_4ce993dbc6c729bac87aa4c2d121ea46 to: /var/lib/boinc-client/slots/3/oifs_43r3_ps_0487_1999050100_123_968_12185131.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/3/oifs_43r3_ps_0487_1999050100_123_968_12185131.zip
ic_ancil_file: ic_ancil_12185131
ifsdata_file: ifsdata_12185131
climate_data_file: clim_data_12185131
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_4db807956b9124aab14c971849aa9e07 to: /var/lib/boinc-client/slots/3/ic_ancil_12185131.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/3/ic_ancil_12185131.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_522568e51133232bb15f9efbb4689cf0 to: /var/lib/boinc-client/slots/3/ifsdata/ifsdata_12185131.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/3/ifsdata/ifsdata_12185131.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/3/159l_2/clim_data_12185131.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/3/159l_2/clim_data_12185131.zip
Checking for progress XML file: /var/lib/boinc-client/slots/3/progress_file_12185131.xml
Opened progress file ok : /var/lib/boinc-client/slots/3/progress_file_12185131.xml
-- Model is restarting --
Adjusting last_iter, 1659, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/3/progress_file_12185131.xml
last_cpu_time: 39392
upload_file_number: 69
last_iter: 1655
last_upload: 5961600
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0487_1999050100_123_968_12185131_0_r1170178595
The child process has been launched with process id: 2833
Executing the command: /var/lib/boinc-client/slots/3/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:obata-H87-HD3:1:1:2833:2833] [20230123:173159:1674491519:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:obata-H87-HD3:1:1:2833:2833] [20230123:173159:1674491519:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:obata-H87-HD3:1:1:2833:2833] [20230123:173159:1674491519:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:obata-H87-HD3:1:1:2833:2833] [20230123:173200:1674491520:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:obata-H87-HD3:1:1:2833:2833] [20230123:173200:1674491520:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:obata-H87-HD3:1:1:2833:2833] [20230123:173200:1674491520:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647)
[EC_DRHOOK:obata-H87-HD3:1:1:2833:2833] [20230123:173200:1674491520:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:obata-H87-HD3:1:1:2833:2833] [20230123:173200:1674491520:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:obata-H87-HD3:1:1:2833:2833] [20230123:173200:1674491520:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:obata-H87-HD3:1:1:2833:2833] [20230123:173200:1674491520:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:obata-H87-HD3:1:1:2833:2833] [20230123:173200:1674491520:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:obata-H87-HD3:1:1:2833:2833] [20230123:173200:1674491520:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:obata-H87-HD3:1:1:2833:2833] [20230123:173200:1674491520:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:obata-H87-HD3:1:1:2833:2833] [20230123:173200:1674491520:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:obata-H87-HD3:1:1:2833:2833] [20230123:173200:1674491520:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:obata-H87-HD3:1:1:2833:2833] [20230123:173200:1674491520:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:obata-H87-HD3:1:1:2833:2833] [20230123:173200:1674491520:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:obata-H87-HD3:1:1:2833:2833] [20230123:173200:1674491520:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:obata-H87-HD3:1:1:2833:2833] [20230123:173200:1674491520:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:obata-H87-HD3:1:1:2833:2833] [20230123:173200:1674491520:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:obata-H87-HD3:1:1:2833:2833] [20230123:173200:1674491520:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:obata-H87-HD3:1:1:2833:2833] [20230123:173200:1674491520:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:obata-H87-HD3:1:1:2833:2833] [20230123:173200:1674491520:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:obata-H87-HD3:1:1:2833:2833] [20230123:173200:1674491520:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:obata-H87-HD3:1:1:2833:2833] [20230123:173200:1674491520:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:obata-H87-HD3:1:1:2833:2833] [20230123:173200:1674491520:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:obata-H87-HD3:1:1:2833:2833] [20230123:173200:1674491520:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:obata-H87-HD3:1:1:2833:2833] [20230123:173200:1674491520:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:obata-H87-HD3:1:1:2833:2833] [20230123:173200:1674491520:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:obata-H87-HD3:1:1:2833:2833] [20230123:173200:1674491520:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
17:32:47 STEP 1656 H=1656:00 +CPU= 32.920
17:33:09 STEP 1657 H=1657:00 +CPU= 18.322
17:33:28 STEP 1658 H=1658:00 +CPU= 19.094
17:33:57 STEP 1659 H=1659:00 +CPU= 29.252
17:34:16 STEP 1660 H=1660:00 +CPU= 18.888
17:34:36 STEP 1661 H=1661:00 +CPU= 18.703
17:35:05 STEP 1662 H=1662:00 +CPU= 29.231
17:35:23 STEP 1663 H=1663:00 +CPU= 18.055
17:35:43 STEP 1664 H=1664:00 +CPU= 19.313
17:36:14 STEP 1665 H=1665:00 +CPU= 29.273
17:36:35 STEP 1666 H=1666:00 +CPU= 19.671
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
17:37:05 STEP 1667 H=1667:00 +CPU= 18.895
17:37:39 STEP 1668 H=1668:00 +CPU= 32.443
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+001668
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+001668
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+001668
17:37:59 STEP 1669 H=1669:00 +CPU= 18.715
17:38:18 STEP 1670 H=1670:00 +CPU= 19.215
17:38:49 STEP 1671 H=1671:00 +CPU= 29.433
17:39:09 STEP 1672 H=1672:00 +CPU= 19.178
17:39:28 STEP 1673 H=1673:00 +CPU= 18.641
17:39:59 STEP 1674 H=1674:00 +CPU= 30.073
17:40:19 STEP 1675 H=1675:00 +CPU= 18.703
17:40:40 STEP 1676 H=1676:00 +CPU= 19.281
17:41:11 STEP 1677 H=1677:00 +CPU= 29.533
17:41:31 STEP 1678 H=1678:00 +CPU= 19.262
17:41:52 STEP 1679 H=1679:00 +CPU= 19.282
17:42:30 STEP 1680 H=1680:00 +CPU= 34.799
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+001680
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+001680
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+001680
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185131/ICMGGhq0f+001656
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185131/ICMSHhq0f+001656
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185131/ICMUAhq0f+001656
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185131/ICMGGhq0f+001668
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185131/ICMSHhq0f+001668
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185131/ICMUAhq0f+001668
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0487_1999050100_123_968_12185131_0_r1170178595_69.zip
Uploading the intermediate file: upload_file_69.zip
17:42:50 STEP 1681 H=1681:00 +CPU= 18.897
17:43:10 STEP 1682 H=1682:00 +CPU= 19.425
17:43:40 STEP 1683 H=1683:00 +CPU= 29.572
17:43:59 STEP 1684 H=1684:00 +CPU= 18.832
17:44:18 STEP 1685 H=1685:00 +CPU= 18.285
17:44:49 STEP 1686 H=1686:00 +CPU= 29.772
17:45:08 STEP 1687 H=1687:00 +CPU= 18.902
17:45:28 STEP 1688 H=1688:00 +CPU= 19.090
17:45:58 STEP 1689 H=1689:00 +CPU= 29.609
17:46:19 STEP 1690 H=1690:00 +CPU= 19.760
17:46:39 STEP 1691 H=1691:00 +CPU= 19.489
Abort request received from BOINC client, ending the child process
</stderr_txt>
]]>
