<core_client_version>7.24.1</core_client_version>
<![CDATA[
<stderr_txt>
OOK:lwalden-VirtualBox:1:1:4232:4232] [20240815:142439:1723757079:46267.387] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] : GP_MODEL_HEAP
[EC_DRHOOK:lwalden-VirtualBox:1:1:4232:4232] [20240815:142440:1723757080:46268.388] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1734] DrHook backtrace done for signal#15, nsigs = 1
[EC_DRHOOK:lwalden-VirtualBox:1:1:4232:4232] [20240815:142440:1723757080:46268.388] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1785] Calling previous signal handler at 0x1cf8da0 for signal#15, nsigs = 1
[EC_DRHOOK:lwalden-VirtualBox:1:1:4232:4232] [20240815:142440:1723757080:46268.388] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1796] Returned from previous signal handler (at 0x1cf8da0, signal#15, time taken = 0s), nsigs = 1
[EC_DRHOOK:lwalden-VirtualBox:1:1:4232:4232] [20240815:142440:1723757080:46268.388] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1841] Error _exit(143) upon receipt of signal#15, nsigs = 1
(argv0) ../../projects/climateprediction.net/oifs_43r3_bl_1.13_x86_64-pc-linux-gnu
(argv1) start_date: 2016092300
(argv2) exptid: h7zg
(argv3) unique_member_id: a2uz
(argv4) batchid: 1018
(argv5) wuid: 12292034
(argv6) fclen: 20
(argv7) app_name: oifs_43r3_bl
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/1
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_bl
version: 1.13
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292034
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/1/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/1/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_2d3c504413e2c45173dc0203462da426 to: /var/lib/boinc-client/slots/1/oifs_43r3_bl_a2uz_2016092300_20_1018_12292034.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/1/oifs_43r3_bl_a2uz_2016092300_20_1018_12292034.zip
ic_ancil_file: ic_ancil_12292034
ifsdata_file: ifsdata_12292034
climate_data_file: clim_data_12292034
horiz_resolution: 159
vert_resolution: 91
grid_type: l_2
upload_interval: 96
utstep: 900
nfrres: restart dump frequency (steps) 48
Copying IC ancils from: ../../projects/climateprediction.net/jf_f7ecfd27aa1e68bbe7e8d19800e25c73 to: /var/lib/boinc-client/slots/1/ic_ancil_12292034.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/1/ic_ancil_12292034.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_a4d90e3fe34c39506d3e63844b2656bf to: /var/lib/boinc-client/slots/1/ifsdata/ifsdata_12292034.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/1/ifsdata/ifsdata_12292034.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/1/159l_2/clim_data_12292034.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/1/159l_2/clim_data_12292034.zip
Checking for progress XML file: /var/lib/boinc-client/slots/1/progress_file_12292034.xml
Opened progress file ok : /var/lib/boinc-client/slots/1/progress_file_12292034.xml
-- Model is restarting --
Adjusting last_iter, 1913, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/1/progress_file_12292034.xml
last_cpu_time: 63748
upload_file_number: 19
last_iter: 1873
last_upload: 1641600
model_completed: 0
total_length_of_simulation: 1728000
result_base_name: oifs_43r3_bl_a2uz_2016092300_20_1018_12292034_1_r632911349
The child process has been launched with process id: 1677
Executing the command: /var/lib/boinc-client/slots/1/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:lwalden-VirtualBox:1:1:1677:1677] [20240815:221232:1723785152:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2091] fp = 0x314f440
[EC_DRHOOK:lwalden-VirtualBox:1:1:1677:1677] [20240815:221232:1723785152:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:lwalden-VirtualBox:1:1:1677:1677] [20240815:221232:1723785152:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:lwalden-VirtualBox:1:1:1677:1677] [20240815:221232:1723785152:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:lwalden-VirtualBox:1:1:1677:1677] [20240815:221232:1723785152:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:lwalden-VirtualBox:1:1:1677:1677] [20240815:221232:1723785152:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647)
[EC_DRHOOK:lwalden-VirtualBox:1:1:1677:1677] [20240815:221232:1723785152:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:lwalden-VirtualBox:1:1:1677:1677] [20240815:221232:1723785152:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:lwalden-VirtualBox:1:1:1677:1677] [20240815:221232:1723785152:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:lwalden-VirtualBox:1:1:1677:1677] [20240815:221232:1723785152:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:lwalden-VirtualBox:1:1:1677:1677] [20240815:221232:1723785152:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:lwalden-VirtualBox:1:1:1677:1677] [20240815:221232:1723785152:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:lwalden-VirtualBox:1:1:1677:1677] [20240815:221232:1723785152:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:lwalden-VirtualBox:1:1:1677:1677] [20240815:221232:1723785152:0.000] [ignore_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:lwalden-VirtualBox:1:1:1677:1677] [20240815:221232:1723785152:0.000] [restore_default_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:lwalden-VirtualBox:1:1:1677:1677] [20240815:221232:1723785152:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:lwalden-VirtualBox:1:1:1677:1677] [20240815:221232:1723785152:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:lwalden-VirtualBox:1:1:1677:1677] [20240815:221232:1723785152:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:lwalden-VirtualBox:1:1:1677:1677] [20240815:221232:1723785152:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x820cf0 (old at (nil))
[EC_DRHOOK:lwalden-VirtualBox:1:1:1677:1677] [20240815:221232:1723785152:0.000] [trapfpe_treatment@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:lwalden-VirtualBox:1:1:1677:1677] [20240815:221232:1723785152:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:lwalden-VirtualBox:1:1:1677:1677] [20240815:221232:1723785152:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:lwalden-VirtualBox:1:1:1677:1677] [20240815:221232:1723785152:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x820cf0 (old at (nil))
[EC_DRHOOK:lwalden-VirtualBox:1:1:1677:1677] [20240815:221232:1723785152:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:lwalden-VirtualBox:1:1:1677:1677] [20240815:221232:1723785152:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:lwalden-VirtualBox:1:1:1677:1677] [20240815:221232:1723785152:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:lwalden-VirtualBox:1:1:1677:1677] [20240815:221232:1723785152:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x820cf0 (old at (nil))
[EC_DRHOOK:lwalden-VirtualBox:1:1:1677:1677] [20240815:221232:1723785152:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x820cf0 (old at (nil))
[EC_DRHOOK:lwalden-VirtualBox:1:1:1677:1677] [20240815:221232:1723785152:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:lwalden-VirtualBox:1:1:1677:1677] [20240815:221232:1723785152:0.000] [catch_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:13:35 STEP 1872 H= 468:00 +CPU= 31.235
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:14:09 STEP 1873 H= 468:15 +CPU= 28.136
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
double free or corruption (top)
[EC_DRHOOK:lwalden-VirtualBox:1:1:1677:1677] [20240815:221439:1723785279:126.854] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1538] Received signal#15 (SIGTERM) :: 3933MB (heap), 4749MB (maxrss), 0MB (maxstack), 0 (paging), nsigs = 1
[EC_DRHOOK:lwalden-VirtualBox:1:1:1677:1677] [20240815:221439:1723785279:126.854] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1542] Also activating Harakiri-alarm (SIGALRM=14) to expire after 500s elapsed to prevent hangs, nsigs = 1
[EC_DRHOOK:lwalden-VirtualBox:1:1:1677:1677] [20240815:221439:1723785279:126.854] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1544] Harakiri signal handler 'signal_harakiri' for signal#14 (SIGALRM) installed at 0x820890 (old at (nil))
[EC_DRHOOK:lwalden-VirtualBox:1:1:1677:1677] [20240815:221439:1723785279:126.854] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1617] Signal#15 was caused by unrecognized si_code [memaddr=0x1], nsigs = 1
[EC_DRHOOK:lwalden-VirtualBox:1:1:1677:1677] [20240815:221439:1723785279:126.854] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1686] Starting DrHook backtrace for signal#15, nsigs = 1
[EC_DRHOOK:lwalden-VirtualBox:1:1:1677:1677] [20240815:221439:1723785279:126.854] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3843] 3933 MB (maxheap), 4749 MB (maxrss), 0 MB (maxstack)
[EC_DRHOOK:lwalden-VirtualBox:1:1:1677:1677] [20240815:221439:1723785279:126.854] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] : MASTER
[EC_DRHOOK:lwalden-VirtualBox:1:1:1677:1677] [20240815:221439:1723785279:126.854] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] : CNT0
[EC_DRHOOK:lwalden-VirtualBox:1:1:1677:1677] [20240815:221439:1723785279:126.854] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] : CNT1
[EC_DRHOOK:lwalden-VirtualBox:1:1:1677:1677] [20240815:221439:1723785279:126.854] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] : CNT2
[EC_DRHOOK:lwalden-VirtualBox:1:1:1677:1677] [20240815:221439:1723785279:126.854] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] : CNT3
[EC_DRHOOK:lwalden-VirtualBox:1:1:1677:1677] [20240815:221439:1723785279:126.854] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] : CNT4
[EC_DRHOOK:lwalden-VirtualBox:1:1:1677:1677] [20240815:221439:1723785279:126.854] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] : STEPO
[EC_DRHOOK:lwalden-VirtualBox:1:1:1677:1677] [20240815:221439:1723785279:126.854] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] : SCAN2M
[EC_DRHOOK:lwalden-VirtualBox:1:1:1677:1677] [20240815:221439:1723785279:126.854] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] : GP_MODEL_HEAP
[EC_DRHOOK:lwalden-VirtualBox:1:1:1677:1677] [20240815:221439:1723785279:126.854] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] : GP_MODEL
[EC_DRHOOK:lwalden-VirtualBox:1:1:1677:1677] [20240815:221439:1723785279:126.854] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] : EC_PHYS_DRV
[EC_DRHOOK:lwalden-VirtualBox:1:1:1677:1677] [20240815:221439:1723785279:126.854] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] : >OMP-PHYSICS (1001)
[EC_DRHOOK:lwalden-VirtualBox:1:1:1677:1677] [20240815:221439:1723785279:126.855] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] : EC_PHYS
[EC_DRHOOK:lwalden-VirtualBox:1:1:1677:1677] [20240815:221439:1723785279:126.855] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] : CALLPAR
[EC_DRHOOK:lwalden-VirtualBox:1:1:1677:1677] [20240815:221439:1723785279:126.855] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] : CLOUD_LAYER
[EC_DRHOOK:lwalden-VirtualBox:1:1:1677:1677] [20240815:221439:1723785279:126.855] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] : CLOUDSC
[EC_DRHOOK:lwalden-VirtualBox:1:1:1677:1677] [20240815:221440:1723785280:127.855] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1734] DrHook backtrace done for signal#15, nsigs = 1
[EC_DRHOOK:lwalden-VirtualBox:1:1:1677:1677] [20240815:221440:1723785280:127.855] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1785] Calling previous signal handler at 0x1cf8da0 for signal#15, nsigs = 1
forrtl: error (78): process killed (SIGTERM)
Image PC Routine Line Source
oifs_43r3_model.e 0000000001CF935B Unknown Unknown Unknown
oifs_43r3_model.e 00000000008214C1 Unknown Unknown Unknown
oifs_43r3_model.e 0000000001DD5490 Unknown Unknown Unknown
oifs_43r3_model.e 000000000117FC15 cloudsc_ 3136 cloudsc.F90
oifs_43r3_model.e 000000000115C847 cloud_layer_ 131 cloud_layer.F90
oifs_43r3_model.e 0000000000FBB064 callpar_ 1223 callpar.F90
oifs_43r3_model.e 0000000000F5FA6B ec_phys_ 670 ec_phys.F90
oifs_43r3_model.e 0000000000F4DBC4 ec_phys_drv_ 599 ec_phys_drv.F90
oifs_43r3_model.e 0000000000BD7131 gp_model_ 613 gp_model.F90
oifs_43r3_model.e 00000000012C5002 gp_model_heap_ 74 gp_model_heap.F90
oifs_43r3_model.e 0000000000BACBAE scan2m_ 535 scan2m.F90
oifs_43r3_model.e 0000000000688153 stepo_ 327 stepo.F90
oifs_43r3_model.e 00000000004CD1DC cnt4_ 1133 cnt4.F90
oifs_43r3_model.e 00000000004B5D85 cnt3_ 267 cnt3.F90
oifs_43r3_model.e 00000000004B4F52 cnt2_ 88 cnt2.F90
oifs_43r3_model.e 000000000049D638 cnt1_ 92 cnt1.F90
oifs_43r3_model.e 0000000000406FBD cnt0_ 146 cnt0.F90
oifs_43r3_model.e 0000000000401E7F MAIN__ 96 master.F90
oifs_43r3_model.e 0000000000401E12 Unknown Unknown Unknown
oifs_43r3_model.e 0000000001DCC34A Unknown Unknown Unknown
oifs_43r3_model.e 0000000001DCDBE7 Unknown Unknown Unknown
oifs_43r3_model.e 0000000000401CD5 Unknown Unknown Unknown
(argv0) ../../projects/climateprediction.net/oifs_43r3_bl_1.13_x86_64-pc-linux-gnu
(argv1) start_date: 2016092300
(argv2) exptid: h7zg
(argv3) unique_member_id: a2uz
(argv4) batchid: 1018
(argv5) wuid: 12292034
(argv6) fclen: 20
(argv7) app_name: oifs_43r3_bl
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/1
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_bl
version: 1.13
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292034
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/1/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/1/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_2d3c504413e2c45173dc0203462da426 to: /var/lib/boinc-client/slots/1/oifs_43r3_bl_a2uz_2016092300_20_1018_12292034.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/1/oifs_43r3_bl_a2uz_2016092300_20_1018_12292034.zip
ic_ancil_file: ic_ancil_12292034
ifsdata_file: ifsdata_12292034
climate_data_file: clim_data_12292034
horiz_resolution: 159
vert_resolution: 91
grid_type: l_2
upload_interval: 96
utstep: 900
nfrres: restart dump frequency (steps) 48
Copying IC ancils from: ../../projects/climateprediction.net/jf_f7ecfd27aa1e68bbe7e8d19800e25c73 to: /var/lib/boinc-client/slots/1/ic_ancil_12292034.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/1/ic_ancil_12292034.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_a4d90e3fe34c39506d3e63844b2656bf to: /var/lib/boinc-client/slots/1/ifsdata/ifsdata_12292034.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/1/ifsdata/ifsdata_12292034.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/1/159l_2/clim_data_12292034.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/1/159l_2/clim_data_12292034.zip
Checking for progress XML file: /var/lib/boinc-client/slots/1/progress_file_12292034.xml
Opened progress file ok : /var/lib/boinc-client/slots/1/progress_file_12292034.xml
-- Model is restarting --
Adjusting last_iter, 1874, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/1/progress_file_12292034.xml
last_cpu_time: 63832
upload_file_number: 19
last_iter: 1873
last_upload: 1641600
model_completed: 0
total_length_of_simulation: 1728000
result_base_name: oifs_43r3_bl_a2uz_2016092300_20_1018_12292034_1_r632911349
The child process has been launched with process id: 1676
Executing the command: /var/lib/boinc-client/slots/1/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:lwalden-VirtualBox:1:1:1676:1676] [20240815:222259:1723785779:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2091] fp = 0x314f440
[EC_DRHOOK:lwalden-VirtualBox:1:1:1676:1676] [20240815:222259:1723785779:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:lwalden-VirtualBox:1:1:1676:1676] [20240815:222259:1723785779:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:lwalden-VirtualBox:1:1:1676:1676] [20240815:222259:1723785779:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:lwalden-VirtualBox:1:1:1676:1676] [20240815:222259:1723785779:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:lwalden-VirtualBox:1:1:1676:1676] [20240815:222259:1723785779:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647)
[EC_DRHOOK:lwalden-VirtualBox:1:1:1676:1676] [20240815:222259:1723785779:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:lwalden-VirtualBox:1:1:1676:1676] [20240815:222259:1723785779:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:lwalden-VirtualBox:1:1:1676:1676] [20240815:222259:1723785779:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:lwalden-VirtualBox:1:1:1676:1676] [20240815:222259:1723785779:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:lwalden-VirtualBox:1:1:1676:1676] [20240815:222259:1723785779:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:lwalden-VirtualBox:1:1:1676:1676] [20240815:222259:1723785779:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:lwalden-VirtualBox:1:1:1676:1676] [20240815:222259:1723785779:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:lwalden-VirtualBox:1:1:1676:1676] [20240815:222259:1723785779:0.000] [ignore_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:lwalden-VirtualBox:1:1:1676:1676] [20240815:222259:1723785779:0.000] [restore_default_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:lwalden-VirtualBox:1:1:1676:1676] [20240815:222259:1723785779:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:lwalden-VirtualBox:1:1:1676:1676] [20240815:222259:1723785779:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:lwalden-VirtualBox:1:1:1676:1676] [20240815:222259:1723785779:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:lwalden-VirtualBox:1:1:1676:1676] [20240815:222259:1723785779:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x820cf0 (old at (nil))
[EC_DRHOOK:lwalden-VirtualBox:1:1:1676:1676] [20240815:222259:1723785779:0.000] [trapfpe_treatment@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:lwalden-VirtualBox:1:1:1676:1676] [20240815:222259:1723785779:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:lwalden-VirtualBox:1:1:1676:1676] [20240815:222259:1723785779:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:lwalden-VirtualBox:1:1:1676:1676] [20240815:222259:1723785779:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x820cf0 (old at (nil))
[EC_DRHOOK:lwalden-VirtualBox:1:1:1676:1676] [20240815:222259:1723785779:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:lwalden-VirtualBox:1:1:1676:1676] [20240815:222259:1723785779:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:lwalden-VirtualBox:1:1:1676:1676] [20240815:222259:1723785779:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:lwalden-VirtualBox:1:1:1676:1676] [20240815:222259:1723785779:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x820cf0 (old at (nil))
[EC_DRHOOK:lwalden-VirtualBox:1:1:1676:1676] [20240815:222259:1723785779:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x820cf0 (old at (nil))
[EC_DRHOOK:lwalden-VirtualBox:1:1:1676:1676] [20240815:222259:1723785779:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:lwalden-VirtualBox:1:1:1676:1676] [20240815:222259:1723785779:0.000] [catch_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:24:16 STEP 1872 H= 468:00 +CPU= 31.889
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:24:56 STEP 1873 H= 468:15 +CPU= 31.856
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:25:30 STEP 1874 H= 468:30 +CPU= 25.650
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:26:11 STEP 1875 H= 468:45 +CPU= 29.879
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:26:49 STEP 1876 H= 469:00 +CPU= 31.748
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:27:29 STEP 1877 H= 469:15 +CPU= 31.709
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:28:12 STEP 1878 H= 469:30 +CPU= 32.993
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:28:53 STEP 1879 H= 469:45 +CPU= 32.267
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:29:32 STEP 1880 H= 470:00 +CPU= 31.680
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:29:57 STEP 1881 H= 470:15 +CPU= 20.997
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:30:39 STEP 1882 H= 470:30 +CPU= 33.061
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:31:24 STEP 1883 H= 470:45 +CPU= 34.409
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:32:13 STEP 1884 H= 471:00 +CPU= 38.556
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGh7zg+001884
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHh7zg+001884
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAh7zg+001884
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:32:48 STEP 1885 H= 471:15 +CPU= 29.464
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:33:27 STEP 1886 H= 471:30 +CPU= 31.164
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:34:05 STEP 1887 H= 471:45 +CPU= 30.174
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:34:46 STEP 1888 H= 472:00 +CPU= 32.939
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:35:26 STEP 1889 H= 472:15 +CPU= 31.249
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:36:06 STEP 1890 H= 472:30 +CPU= 32.248
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:36:46 STEP 1891 H= 472:45 +CPU= 31.553
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:37:26 STEP 1892 H= 473:00 +CPU= 31.788
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:38:06 STEP 1893 H= 473:15 +CPU= 32.107
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:38:44 STEP 1894 H= 473:30 +CPU= 30.076
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:39:09 STEP 1895 H= 473:45 +CPU= 21.278
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:39:53 STEP 1896 H= 474:00 +CPU= 31.155
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGh7zg+001896
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHh7zg+001896
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAh7zg+001896
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:40:39 STEP 1897 H= 474:15 +CPU= 35.479
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:41:28 STEP 1898 H= 474:30 +CPU= 37.115
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:42:16 STEP 1899 H= 474:45 +CPU= 34.688
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:43:04 STEP 1900 H= 475:00 +CPU= 35.221
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:43:46 STEP 1901 H= 475:15 +CPU= 33.891
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:44:25 STEP 1902 H= 475:30 +CPU= 31.331
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:45:06 STEP 1903 H= 475:45 +CPU= 31.477
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:45:46 STEP 1904 H= 476:00 +CPU= 33.795
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:46:26 STEP 1905 H= 476:15 +CPU= 31.774
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:47:05 STEP 1906 H= 476:30 +CPU= 30.771
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:47:35 STEP 1907 H= 476:45 +CPU= 24.012
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:48:18 STEP 1908 H= 477:00 +CPU= 35.665
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGh7zg+001908
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHh7zg+001908
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAh7zg+001908
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:49:00 STEP 1909 H= 477:15 +CPU= 33.158
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:49:43 STEP 1910 H= 477:30 +CPU= 32.823
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:50:24 STEP 1911 H= 477:45 +CPU= 33.026
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:51:07 STEP 1912 H= 478:00 +CPU= 35.021
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:51:56 STEP 1913 H= 478:15 +CPU= 38.499
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:52:43 STEP 1914 H= 478:30 +CPU= 37.287
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:53:00 STEP 1915 H= 478:45 +CPU= 14.980
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:53:57 STEP 1916 H= 479:00 +CPU= 44.545
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:54:48 STEP 1917 H= 479:15 +CPU= 40.317
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:55:38 STEP 1918 H= 479:30 +CPU= 39.636
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:56:28 STEP 1919 H= 479:45 +CPU= 39.621
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:57:41 STEP 1920 H= 480:00 +CPU= 54.531
..The child process terminated with status: 0
>>> Printing last 70 lines from file: NODE.001_01
DDH------------------------------------------------------------------------
I I
I TIME = 480.000H DDH * Diagnostic Var., AREA GLOBAL I
I I
I-------------------------------------------------------------------------I
I VARIABLE I Initial value I Current value I
I-------------------------------------------------------------------------I
I OMEGA (P/S) I -0.4158329646E-21 I 0.5164453950E-06 I
ICloud fractionI 0.0000000000E+00 I 0.1054596835E-01 I
I Relat. Hum. I 0.2200996920E+00 I 0.2794403983E+00 I
I PASS 01 I 0.0000000000E+00 I 0.1945287250E+03 I
DDH------------------------------------------------------------------------
MAXGPFV : MAX. VALUE = 0.000000000000000E+000
MAXGPFV : MAX. VALUE = 0.000000000000000E+000
NSTEP = 1920 SCAN2M_HPOS P
IO-STREAM SETUP - IOTYPE = 2 NUMIOPROCS = 1 CPATH =
ICMGGh7zg+001920 MODE=a
IO-STREAM CLOSED - ICMGGh7zg+001920
MPI-TASK: 1 - 321433732 BYTES IN 7 RECORDS TRANSFERRED IN 0.0011 SECONDS292212.5 Mbytes/s, TOTAL TIME= 0.0276( 4.0%)
22:57:41 STEP 1920 H= 480:00 +CPU= 54.531
END CNT3
NSTEP = 1920 CNT0 000000000 41.965 41.965 52.227 0.803 0 0 0 0 0
IO-STREAM STATISTICS, TOTAL NO OF REC/BYTES READ 46/ 1069125708 WRITTEN 1430/************
-TASK-OPENED-OPEN-RECS IN -KBYTE IN -RECS OUT-KBYTE OUT-WALL -WALL IN-WALL
OU-TOT IN -TOT OUT
1 28 0 46 1044068 1430 ************ 38.4 35.8 2.6 36.9 7.6
===-=== START OF TIMING STATISTICS ===-===
STATS FOR ALL TASKS
NUM ROUTINE CALLS MEAN(ms) MAX(ms) FRAC(%) UNBAL(%)
0 CNT0 - COMPLETE EXECUTION 1 2082194.3 2082194.3 100.00 0.00
1 CNT4 - FORWARD INTEGRATION 1 2043548.7 2043548.7 98.14 0.00
8 SCAN2M - GRID-POINT DYNAMICS 53 6730.7 6730.7 17.13 0.00
9 SPCM - SPECTRAL COMP. 48 1432.3 1432.3 3.30 0.00
10 SCAN2M - PHYSICS 49 19516.3 19516.3 45.93 0.00
11 IOPACK - OUTPUT P.P. RESULTS 4 2589.4 2589.4 0.50 0.00
12 SPNORM - SPECTRAL NORM COMP. 3 41.4 41.4 0.01 0.00
17 GRIDFPOS IN CNT4 4 42.8 42.8 0.01 0.00
27 RESTART FILES - WRITING 1 10357.6 10357.6 0.50 0.00
28 RESTART FILES - READING 1 37977.6 37977.6 1.82 0.00
29 SU4FPOS IN CNT4 4 0.0 0.0 0.00 0.00
30 DYNFPOS IN CNT4 4 7295.7 7295.7 1.40 0.00
31 POSDDH IN STEPO 1 1.3 1.3 0.00 0.00
37 CPGLAG - SL COMPUTATIONS 49 9144.1 9144.1 21.52 0.00
39 SU0YOMB 1 394.5 394.5 0.02 0.00
51 SCAN2M - SL COMM. PART 1 49 32.5 32.5 0.08 0.00
54 SPCM - M TO S/S TO M TRANSP. 48 606.3 606.3 1.40 0.00
55 SPCIMPF - S TO M/M TO S TRANSP. 48 119.2 119.2 0.27 0.00
56 SPNORM - SPECTRAL NORM COMM. 3 0.3 0.3 0.00 0.00
102 LTINV_CTL - INVERSE LEGENDRE TRANSFORM 108 498.1 498.1 2.58 0.00
103 LTDIR_CTL - DIRECT LEGENDRE TRANSFORM 100 268.1 268.1 1.29 0.00
106 FTDIR_CTL - DIRECT FOURIER TRANSFORM 100 88.4 88.4 0.42 0.00
107 FTINV_CTL - INVERSE FOURIER TRANSFORM 108 171.6 171.6 0.89 0.00
140 SULEG - COMP. OF LEGENDRE POL. 1 43.0 43.0 0.00 0.00
152 LTINV_CTL - M TO L TRANSPOSITION 108 103.9 103.9 0.54 0.00
153 LTDIR_CTL - L TO M TRANSPOSITION 100 59.8 59.8 0.29 0.00
157 FTINV_CTL - L TO G TRANSPOSITION 108 56.0 56.0 0.29 0.00
158 FTDIR_CTL - G TO L TRANSPOSITION 100 51.8 51.8 0.25 0.00
400 GSTATS 6667 0.0 0.0 0.00 0.00
401 GSTATS HOOK 6149 0.0 0.0 0.00 0.00
TOTAL MEASURED IMBALANCE = 0.0 SECONDS, 0.0 PERCENT
TOTAL WALLCLOCK TIME 2082.2 CPU TIME 1615.4 VECTOR TIME 1615.4
===-=== END OF TIMING STATISTICS ===-===
FORECAST DAYS PER DAY 845.6
*** END CNT0 ***
------------------------------------------------
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGh7zg+001920
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHh7zg+001920
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAh7zg+001920
Adding to the zip: /var/lib/boinc-client/slots/1/NODE.001_01
Adding to the zip: /var/lib/boinc-client/slots/1/ifs.stat
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292034/ICMUAh7zg+001884
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292034/ICMUAh7zg+001872
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292034/ICMUAh7zg+001896
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292034/ICMGGh7zg+001920
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292034/ICMGGh7zg+001836
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292034/ICMGGh7zg+001824
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292034/ICMGGh7zg+001908
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292034/ICMGGh7zg+001896
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292034/ICMGGh7zg+001848
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292034/ICMUAh7zg+001824
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292034/ICMUAh7zg+001908
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292034/ICMSHh7zg+001908
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292034/ICMUAh7zg+001836
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292034/ICMUAh7zg+001848
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292034/ICMSHh7zg+001872
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292034/ICMSHh7zg+001920
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292034/ICMGGh7zg+001884
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292034/ICMGGh7zg+001860
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292034/ICMUAh7zg+001920
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292034/ICMGGh7zg+001872
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292034/ICMSHh7zg+001848
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292034/ICMSHh7zg+001896
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292034/ICMUAh7zg+001860
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292034/ICMSHh7zg+001860
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292034/ICMSHh7zg+001836
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292034/ICMSHh7zg+001884
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292034/ICMSHh7zg+001824
Zipping up the final file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a2uz_2016092300_20_1018_12292034_1_r632911349_19.zip
Uploading the final file: upload_file_19.zip
Uploading trickle at timestep: 1727100
23:01:14 (1672): called boinc_finish(0)
</stderr_txt>
]]>
