climateprediction.net home page
Task 22435149

Task 22435149

Name oifs_43r3_bl_a1cf_2016092300_20_1017_12283570_0
Workunit 12283570
Created 7 Jun 2024, 13:22:10 UTC
Sent 7 Jun 2024, 13:26:54 UTC
Report deadline 6 Aug 2024, 13:26:54 UTC
Received 9 Jun 2024, 13:34:32 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 0 (0x00000000)
Computer ID 1506017
Run time 22 hours 43 min 41 sec
CPU time 15 hours 12 min 6 sec
Validate state Invalid
Credit 1,976.32
Device peak FLOPS 7.35 GFLOPS
Application version OpenIFS 43r3 Baroclinic Lifecycle v1.13
x86_64-pc-linux-gnu
Peak working set size 5,519.05 MB
Peak swap size 5,980.80 MB
Peak disk usage 2,223.10 MB
Stderr
<core_client_version>7.20.5</core_client_version>
<![CDATA[
<stderr_txt>
d from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:53:13 STEP 1362 H= 340:30 +CPU= 36.880
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:54:29 STEP 1363 H= 340:45 +CPU= 38.102
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:55:51 STEP 1364 H= 341:00 +CPU= 35.570
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:57:07 STEP 1365 H= 341:15 +CPU= 36.900
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:58:40 STEP 1366 H= 341:30 +CPU= 34.897
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:59:52 STEP 1367 H= 341:45 +CPU= 35.893
Suspend request received from the BOINC client, suspending the child process
[EC_DRHOOK:jesus:1:1:3773:3773] [20240609:060008:1717905608:60064.491] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1538] Received signal#15 (SIGTERM) :: 2169MB (heap), 5276MB (maxrss), 0MB (maxstack), 0 (paging), nsigs = 1
[EC_DRHOOK:jesus:1:1:3773:3773] [20240609:060008:1717905608:60064.496] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1542] Also activating Harakiri-alarm (SIGALRM=14) to expire after 500s elapsed to prevent hangs, nsigs = 1
[EC_DRHOOK:jesus:1:1:3773:3773] [20240609:060008:1717905608:60064.496] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1544] Harakiri signal handler 'signal_harakiri' for signal#14 (SIGALRM) installed at 0x820890 (old at (nil))
[EC_DRHOOK:jesus:1:1:3773:3773] [20240609:060008:1717905608:60064.497] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1617] Signal#15 was caused by unrecognized si_code [memaddr=0x1], nsigs = 1
[EC_DRHOOK:jesus:1:1:3773:3773] [20240609:060008:1717905608:60064.497] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1686] Starting DrHook backtrace for signal#15, nsigs = 1
[EC_DRHOOK:jesus:1:1:3773:3773] [20240609:060008:1717905608:60064.497] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3843] 2225 MB (maxheap), 5276 MB (maxrss), 0 MB (maxstack)
[EC_DRHOOK:jesus:1:1:3773:3773] [20240609:060008:1717905608:60064.498] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :  MASTER 
[EC_DRHOOK:jesus:1:1:3773:3773] [20240609:060008:1717905608:60064.499] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :   CNT0 
[EC_DRHOOK:jesus:1:1:3773:3773] [20240609:060008:1717905608:60064.499] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :    CNT1 
[EC_DRHOOK:jesus:1:1:3773:3773] [20240609:060008:1717905608:60064.499] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :     CNT2 
[EC_DRHOOK:jesus:1:1:3773:3773] [20240609:060008:1717905608:60064.499] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :      CNT3 
[EC_DRHOOK:jesus:1:1:3773:3773] [20240609:060008:1717905608:60064.499] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :       CNT4 
[EC_DRHOOK:jesus:1:1:3773:3773] [20240609:060008:1717905608:60064.499] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :        DYNFPOS 
[EC_DRHOOK:jesus:1:1:3773:3773] [20240609:060008:1717905608:60064.499] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :         STEPO_FPOS 
[EC_DRHOOK:jesus:1:1:3773:3773] [20240609:060008:1717905608:60064.499] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :          IOFPOS 
[EC_DRHOOK:jesus:1:1:3773:3773] [20240609:060008:1717905608:60064.499] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :           WRPLFP 
[EC_DRHOOK:jesus:1:1:3773:3773] [20240609:060008:1717905608:60064.499] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :            WROUTGPGB 
[EC_DRHOOK:jesus:1:1:3773:3773] [20240609:060008:1717905608:60064.499] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :             IOSTREAM_MIX:IO_PUT 
[EC_DRHOOK:jesus:1:1:3773:3773] [20240609:060008:1717905608:60064.499] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :              IOSTREAM_MIX:GRID_OUT 
[EC_DRHOOK:jesus:1:1:3773:3773] [20240609:060008:1717905608:60064.499] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :               IOSTREAM_MIX:WRITE_RECORD 
[EC_DRHOOK:jesus:1:1:3773:3773] [20240609:060008:1717905608:60064.499] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :                IOSTREAM_MIX:GRIB_ENCODE 
[EC_DRHOOK:jesus:1:1:3773:3773] [20240609:060008:1717905608:60064.499] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :                 >OMP-IOSTREAM_MIX: GRIB_AP(1484) 
[EC_DRHOOK:jesus:1:1:3773:3773] [20240609:060008:1717905608:60064.499] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :                  IOSTREAM_MIX:GRIB_UPDATE_MESSAGE 
[EC_DRHOOK:jesus:1:1:3773:3773] [20240609:060008:1717905608:60064.499] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :                   GRIB_API:IGRIB_SET_REAL8_ARRAY 
[EC_DRHOOK:jesus:1:1:3773:3773] [20240609:060009:1717905609:60065.501] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1734] DrHook backtrace done for signal#15, nsigs = 1
[EC_DRHOOK:jesus:1:1:3773:3773] [20240609:060009:1717905609:60065.501] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1785] Calling previous signal handler at 0x1cf8da0 for signal#15, nsigs = 1
forrtl: error (78): process killed (SIGTERM)
Image              PC                Routine            Line        Source             
oifs_43r3_model.e  0000000001CF935B  Unknown               Unknown  Unknown
oifs_43r3_model.e  00000000008214C1  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001DD5490  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001B83D4D  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001B83F52  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001B585E5  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001AFE24C  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001AE753B  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001ADD4CB  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000000A6D405  grib_api_interfac         362  grib_api_interface.F90
oifs_43r3_model.e  0000000000A4AC78  iostream_mix_mp_w        3122  iostream_mix.F90
oifs_43r3_model.e  0000000000A41E63  iostream_mix_mp_g        2087  iostream_mix.F90
oifs_43r3_model.e  0000000000A3CD09  iostream_mix_mp_i        1311  iostream_mix.F90
oifs_43r3_model.e  0000000000500FAC  wroutgpgb_                234  wroutgpgb.F90
oifs_43r3_model.e  00000000005034D2  wrplfp_                   194  wrplfp.F90
oifs_43r3_model.e  00000000004FCCDD  iofpos_                    95  iofpos.F90
oifs_43r3_model.e  00000000004D7ABF  stepo_fpos_               556  stepo_fpos.F90
oifs_43r3_model.e  00000000004CFC43  dynfpos_                  170  dynfpos.F90
oifs_43r3_model.e  00000000004CCCC8  cnt4_                     947  cnt4.F90
oifs_43r3_model.e  00000000004B5D85  cnt3_                     267  cnt3.F90
oifs_43r3_model.e  00000000004B4F52  cnt2_                      88  cnt2.F90
oifs_43r3_model.e  000000000049D638  cnt1_                      92  cnt1.F90
oifs_43r3_model.e  0000000000406FBD  cnt0_                     146  cnt0.F90
oifs_43r3_model.e  0000000000401E7F  MAIN__                     96  master.F90
oifs_43r3_model.e  0000000000401E12  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001DCC34A  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001DCDBE7  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000000401CD5  Unknown               Unknown  Unknown
(argv0) ../../projects/climateprediction.net/oifs_43r3_bl_1.13_x86_64-pc-linux-gnu
(argv1) start_date: 2016092300
(argv2) exptid: h7zg
(argv3) unique_member_id: a1cf
(argv4) batchid: 1017
(argv5) wuid: 12283570
(argv6) fclen: 20
(argv7) app_name: oifs_43r3_bl
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/7
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_bl
version: 1.13
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283570
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/7/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/7/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_a9a7717cacd5dd98191fce256f07ae42 to: /var/lib/boinc-client/slots/7/oifs_43r3_bl_a1cf_2016092300_20_1017_12283570.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/7/oifs_43r3_bl_a1cf_2016092300_20_1017_12283570.zip
ic_ancil_file: ic_ancil_12283570
ifsdata_file: ifsdata_12283570
climate_data_file: clim_data_12283570
horiz_resolution: 159
vert_resolution: 91
grid_type: l_2
upload_interval: 96
utstep: 900
nfrres: restart dump frequency (steps) 48
Copying IC ancils from: ../../projects/climateprediction.net/jf_f7ecfd27aa1e68bbe7e8d19800e25c73 to: /var/lib/boinc-client/slots/7/ic_ancil_12283570.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/7/ic_ancil_12283570.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_e719ed3bec905f31f22b367cfd35ccda to: /var/lib/boinc-client/slots/7/ifsdata/ifsdata_12283570.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/7/ifsdata/ifsdata_12283570.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/7/159l_2/clim_data_12283570.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/7/159l_2/clim_data_12283570.zip
Checking for progress XML file: /var/lib/boinc-client/slots/7/progress_file_12283570.xml
Opened progress file ok : /var/lib/boinc-client/slots/7/progress_file_12283570.xml
-- Model is restarting --
Adjusting last_iter, 1368, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/7/progress_file_12283570.xml
last_cpu_time: 51419
upload_file_number: 14
last_iter: 1345
last_upload: 1209600
model_completed: 0
total_length_of_simulation: 1728000
result_base_name: oifs_43r3_bl_a1cf_2016092300_20_1017_12283570_0_r934476021
The child process has been launched with process id: 3470
Executing the command: /var/lib/boinc-client/slots/7/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:jesus:1:1:3470:3470] [20240609:120159:1717927319:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2091] fp = 0x314f440
[EC_DRHOOK:jesus:1:1:3470:3470] [20240609:120159:1717927319:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:jesus:1:1:3470:3470] [20240609:120159:1717927319:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:jesus:1:1:3470:3470] [20240609:120159:1717927319:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:jesus:1:1:3470:3470] [20240609:120159:1717927319:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:jesus:1:1:3470:3470] [20240609:120159:1717927319:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:jesus:1:1:3470:3470] [20240609:120159:1717927319:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:jesus:1:1:3470:3470] [20240609:120159:1717927319:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:jesus:1:1:3470:3470] [20240609:120159:1717927319:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:jesus:1:1:3470:3470] [20240609:120159:1717927319:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:jesus:1:1:3470:3470] [20240609:120159:1717927319:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:jesus:1:1:3470:3470] [20240609:120159:1717927319:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:jesus:1:1:3470:3470] [20240609:120159:1717927319:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:jesus:1:1:3470:3470] [20240609:120159:1717927319:0.000] [ignore_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:jesus:1:1:3470:3470] [20240609:120159:1717927319:0.000] [restore_default_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:jesus:1:1:3470:3470] [20240609:120159:1717927319:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:jesus:1:1:3470:3470] [20240609:120159:1717927319:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:jesus:1:1:3470:3470] [20240609:120159:1717927319:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:jesus:1:1:3470:3470] [20240609:120159:1717927319:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x820cf0 (old at (nil))
[EC_DRHOOK:jesus:1:1:3470:3470] [20240609:120159:1717927319:0.000] [trapfpe_treatment@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:jesus:1:1:3470:3470] [20240609:120159:1717927319:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:jesus:1:1:3470:3470] [20240609:120159:1717927319:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:jesus:1:1:3470:3470] [20240609:120159:1717927319:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x820cf0 (old at (nil))
[EC_DRHOOK:jesus:1:1:3470:3470] [20240609:120159:1717927319:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:jesus:1:1:3470:3470] [20240609:120159:1717927319:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:jesus:1:1:3470:3470] [20240609:120159:1717927319:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:jesus:1:1:3470:3470] [20240609:120159:1717927319:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x820cf0 (old at (nil))
[EC_DRHOOK:jesus:1:1:3470:3470] [20240609:120159:1717927319:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x820cf0 (old at (nil))
[EC_DRHOOK:jesus:1:1:3470:3470] [20240609:120159:1717927319:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:jesus:1:1:3470:3470] [20240609:120159:1717927319:0.000] [catch_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:04:49 STEP 1344 H= 336:00 +CPU= 36.807
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:07:28 STEP 1345 H= 336:15 +CPU= 30.305
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:09:46 STEP 1346 H= 336:30 +CPU= 33.191
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:11:06 STEP 1347 H= 336:45 +CPU= 28.751
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:13:41 STEP 1348 H= 337:00 +CPU= 35.150
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:17:57 STEP 1349 H= 337:15 +CPU= 31.219
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:23:30 STEP 1350 H= 337:30 +CPU= 32.784
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:25:14 STEP 1351 H= 337:45 +CPU= 33.912
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:26:50 STEP 1352 H= 338:00 +CPU= 34.245
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:28:21 STEP 1353 H= 338:15 +CPU= 34.090
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:29:29 STEP 1354 H= 338:30 +CPU= 34.508
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:31:29 STEP 1355 H= 338:45 +CPU= 33.417
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:33:13 STEP 1356 H= 339:00 +CPU= 41.720
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+001356
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+001356
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+001356
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:34:40 STEP 1357 H= 339:15 +CPU= 33.761
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:36:33 STEP 1358 H= 339:30 +CPU= 33.712
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:38:05 STEP 1359 H= 339:45 +CPU= 33.156
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:39:43 STEP 1360 H= 340:00 +CPU= 34.533
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:41:20 STEP 1361 H= 340:15 +CPU= 32.946
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:42:55 STEP 1362 H= 340:30 +CPU= 31.565
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:44:26 STEP 1363 H= 340:45 +CPU= 32.837
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:45:40 STEP 1364 H= 341:00 +CPU= 33.026
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:47:08 STEP 1365 H= 341:15 +CPU= 33.444
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:48:34 STEP 1366 H= 341:30 +CPU= 32.467
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:49:49 STEP 1367 H= 341:45 +CPU= 33.375
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:51:15 STEP 1368 H= 342:00 +CPU= 40.803
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+001368
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+001368
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+001368
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:52:40 STEP 1369 H= 342:15 +CPU= 32.654
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:53:53 STEP 1370 H= 342:30 +CPU= 32.726
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:55:07 STEP 1371 H= 342:45 +CPU= 33.375
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:56:32 STEP 1372 H= 343:00 +CPU= 32.269
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:57:47 STEP 1373 H= 343:15 +CPU= 33.079
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:59:13 STEP 1374 H= 343:30 +CPU= 32.151
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:00:37 STEP 1375 H= 343:45 +CPU= 33.228
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:02:05 STEP 1376 H= 344:00 +CPU= 34.275
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:03:33 STEP 1377 H= 344:15 +CPU= 32.818
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:04:58 STEP 1378 H= 344:30 +CPU= 33.323
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:06:04 STEP 1379 H= 344:45 +CPU= 33.997
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:08:06 STEP 1380 H= 345:00 +CPU= 41.649
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+001380
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+001380
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+001380
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:09:33 STEP 1381 H= 345:15 +CPU= 34.101
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:11:01 STEP 1382 H= 345:30 +CPU= 33.922
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:12:31 STEP 1383 H= 345:45 +CPU= 33.483
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:14:12 STEP 1384 H= 346:00 +CPU= 33.982
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:15:29 STEP 1385 H= 346:15 +CPU= 34.003
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:16:44 STEP 1386 H= 346:30 +CPU= 33.469
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:18:19 STEP 1387 H= 346:45 +CPU= 33.260
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:19:58 STEP 1388 H= 347:00 +CPU= 33.750
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:21:24 STEP 1389 H= 347:15 +CPU= 33.716
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:22:40 STEP 1390 H= 347:30 +CPU= 34.876
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:24:09 STEP 1391 H= 347:45 +CPU= 32.072
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:26:22 STEP 1392 H= 348:00 +CPU= 43.959
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+001392
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+001392
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+001392
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:27:33 STEP 1393 H= 348:15 +CPU= 32.940
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:29:00 STEP 1394 H= 348:30 +CPU= 33.101
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:30:29 STEP 1395 H= 348:45 +CPU= 33.554
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:31:45 STEP 1396 H= 349:00 +CPU= 34.723
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:33:00 STEP 1397 H= 349:15 +CPU= 34.084
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:34:30 STEP 1398 H= 349:30 +CPU= 33.392
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:35:46 STEP 1399 H= 349:45 +CPU= 33.980
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:37:14 STEP 1400 H= 350:00 +CPU= 34.735
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:38:41 STEP 1401 H= 350:15 +CPU= 33.909
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:39:57 STEP 1402 H= 350:30 +CPU= 35.177
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:41:26 STEP 1403 H= 350:45 +CPU= 33.574
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:43:10 STEP 1404 H= 351:00 +CPU= 41.523
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+001404
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+001404
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+001404
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:44:26 STEP 1405 H= 351:15 +CPU= 33.318
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:45:53 STEP 1406 H= 351:30 +CPU= 33.738
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:47:26 STEP 1407 H= 351:45 +CPU= 32.915
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:48:54 STEP 1408 H= 352:00 +CPU= 34.294
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:50:10 STEP 1409 H= 352:15 +CPU= 33.830
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:51:35 STEP 1410 H= 352:30 +CPU= 33.803
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:52:50 STEP 1411 H= 352:45 +CPU= 33.852
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:54:17 STEP 1412 H= 353:00 +CPU= 34.426
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:55:34 STEP 1413 H= 353:15 +CPU= 34.611
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:56:59 STEP 1414 H= 353:30 +CPU= 34.166
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:58:17 STEP 1415 H= 353:45 +CPU= 34.195
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:00:08 STEP 1416 H= 354:00 +CPU= 40.155
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+001416
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+001416
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+001416
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:01:24 STEP 1417 H= 354:15 +CPU= 33.119
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:02:47 STEP 1418 H= 354:30 +CPU= 32.624
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:04:02 STEP 1419 H= 354:45 +CPU= 33.984
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:05:33 STEP 1420 H= 355:00 +CPU= 34.260
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:06:47 STEP 1421 H= 355:15 +CPU= 33.079
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:08:14 STEP 1422 H= 355:30 +CPU= 32.678
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:09:30 STEP 1423 H= 355:45 +CPU= 32.637
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:10:54 STEP 1424 H= 356:00 +CPU= 32.815
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:12:08 STEP 1425 H= 356:15 +CPU= 32.653
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:13:32 STEP 1426 H= 356:30 +CPU= 32.678
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:14:50 STEP 1427 H= 356:45 +CPU= 33.595
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:16:52 STEP 1428 H= 357:00 +CPU= 41.269
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+001428
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+001428
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+001428
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:18:25 STEP 1429 H= 357:15 +CPU= 33.115
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:19:47 STEP 1430 H= 357:30 +CPU= 32.473
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:21:01 STEP 1431 H= 357:45 +CPU= 33.505
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:22:38 STEP 1432 H= 358:00 +CPU= 32.401
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:24:03 STEP 1433 H= 358:15 +CPU= 33.271
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:25:15 STEP 1434 H= 358:30 +CPU= 33.246
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:26:29 STEP 1435 H= 358:45 +CPU= 32.711
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:28:09 STEP 1436 H= 359:00 +CPU= 33.769
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:29:44 STEP 1437 H= 359:15 +CPU= 32.077
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:31:11 STEP 1438 H= 359:30 +CPU= 32.764
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:32:34 STEP 1439 H= 359:45 +CPU= 33.151
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:33:54 STEP 1440 H= 360:00 +CPU= 40.767
..The child process terminated with status: 0
>>> Printing last 70 lines from file: NODE.001_01

 
DDH------------------------------------------------------------------------
I                                                                         I
I TIME =  360.000H DDH * Diagnostic Var.,     AREA                GLOBAL  I
I                                                                         I
I-------------------------------------------------------------------------I
I   VARIABLE   I      Initial value         I        Current value        I
I-------------------------------------------------------------------------I
I  OMEGA (P/S) I     -0.1509026121E-20      I       0.1034373755E-04      I
ICloud fractionI      0.0000000000E+00      I       0.2413114951E-01      I
I  Relat. Hum. I      0.5268700567E-01      I       0.1634456663E+00      I
I  PASS 01     I      0.0000000000E+00      I       0.4719746838E+03      I
DDH------------------------------------------------------------------------
  
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
 NSTEP =  1440 SCAN2M_HPOS  P
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ICMGGh7zg+001440 MODE=a
 IO-STREAM CLOSED - ICMGGh7zg+001440
 MPI-TASK:   1 -    295317284 BYTES IN      7 RECORDS TRANSFERRED IN    0.0289 SECONDS 10218.6 Mbytes/s, TOTAL TIME=    0.0719(40.2%)
 14:33:54 STEP 1440 H= 360:00 +CPU= 40.767
 END CNT3
 NSTEP =  1440 CNT0   000000000  30.513  30.513  47.885   0.637               0               0               0               0               0
IO-STREAM STATISTICS, TOTAL NO OF REC/BYTES READ           46/  1069125708 WRITTEN         2860/************
 -TASK-OPENED-OPEN-RECS IN -KBYTE IN    -RECS OUT-KBYTE OUT-WALL   -WALL IN-WALL
  OU-TOT IN -TOT OUT
   1     53    0       46      1044068     2860 ************    12.5     4.8     7.7     6.0    38.3
===-=== START OF TIMING STATISTICS ===-===
 
STATS FOR ALL TASKS
 NUM ROUTINE                                     CALLS  MEAN(ms)   MAX(ms)   FRAC(%)  UNBAL(%)
   0 CNT0     - COMPLETE EXECUTION                   1 9115122.8 9115122.8    100.00      0.00
   1 CNT4     - FORWARD INTEGRATION                  1 9107327.4 9107327.4     99.91      0.00
   8 SCAN2M - GRID-POINT DYNAMICS                  105   14180.6   14180.6     16.34      0.00
   9 SPCM     - SPECTRAL COMP.                      96    3792.8    3792.8      3.99      0.00
  10 SCAN2M - PHYSICS                               97   43279.3   43279.3     46.06      0.00
  11 IOPACK   - OUTPUT P.P. RESULTS                  8    3580.1    3580.1      0.31      0.00
  12 SPNORM   - SPECTRAL NORM COMP.                  5      81.0      81.0      0.00      0.00
  17 GRIDFPOS IN CNT4                                8      58.7      58.7      0.01      0.00
  27 RESTART FILES - WRITING                         2   14310.8   14310.8      0.31      0.00
  28 RESTART FILES - READING                         1    7333.2    7333.2      0.08      0.00
  29 SU4FPOS IN CNT4                                 8       3.6       3.6      0.00      0.00
  30 DYNFPOS IN CNT4                                 8   20113.3   20113.3      1.77      0.00
  31 POSDDH IN STEPO                                 1      26.1      26.1      0.00      0.00
  37 CPGLAG   - SL COMPUTATIONS                     97   18248.0   18248.0     19.42      0.00
  39 SU0YOMB                                         1     210.3     210.3      0.00      0.00
  51 SCAN2M   - SL COMM. PART 1                     97      38.2      38.2      0.04      0.00
  54 SPCM     - M TO S/S TO M TRANSP.               96    1877.3    1877.3      1.98      0.00
  55 SPCIMPF  - S TO M/M TO S TRANSP.               96     379.5     379.5      0.40      0.00
  56 SPNORM   - SPECTRAL NORM COMM.                  5      12.9      12.9      0.00      0.00
 102 LTINV_CTL   - INVERSE LEGENDRE TRANSFORM      212    1338.1    1338.1      3.11      0.00
 103 LTDIR_CTL   - DIRECT LEGENDRE TRANSFORM       200     990.0     990.0      2.17      0.00
 106 FTDIR_CTL   - DIRECT FOURIER TRANSFORM        200     786.3     786.3      1.73      0.00
 107 FTINV_CTL   - INVERSE FOURIER TRANSFORM       212     456.7     456.7      1.06      0.00
 140 SULEG       - COMP. OF LEGENDRE POL.            1      72.3      72.3      0.00      0.00
 152 LTINV_CTL   - M TO L TRANSPOSITION            212     126.1     126.1      0.29      0.00
 153 LTDIR_CTL   - L TO M TRANSPOSITION            200      51.0      51.0      0.11      0.00
 157 FTINV_CTL   - L TO G TRANSPOSITION            212     337.7     337.7      0.79      0.00
 158 FTDIR_CTL   - G TO L TRANSPOSITION            200     133.2     133.2      0.29      0.00
 400 GSTATS                                      12215       0.0       0.0      0.00      0.00
 401 GSTATS HOOK                                 11311       0.0       0.0      0.00      0.00
TOTAL MEASURED IMBALANCE =       0.0 SECONDS,  0.0 PERCENT
TOTAL WALLCLOCK TIME     9115.1 CPU TIME    3307.2 VECTOR TIME     3307.2


===-=== END   OF TIMING STATISTICS ===-===
FORECAST DAYS PER DAY      142.3
  *** END CNT0 *** 
------------------------------------------------
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+001440
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+001440
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+001440
Adding to the zip: /var/lib/boinc-client/slots/7/NODE.001_01
Adding to the zip: /var/lib/boinc-client/slots/7/ifs.stat
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283570/ICMGGh7zg+001416
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283570/ICMSHh7zg+001416
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283570/ICMUAh7zg+001344
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283570/ICMGGh7zg+001440
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283570/ICMSHh7zg+001428
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283570/ICMUAh7zg+001404
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283570/ICMGGh7zg+001404
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283570/ICMSHh7zg+001368
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283570/ICMSHh7zg+001392
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283570/ICMUAh7zg+001416
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283570/ICMGGh7zg+001356
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283570/ICMUAh7zg+001428
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283570/ICMUAh7zg+001380
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283570/ICMGGh7zg+001392
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283570/ICMSHh7zg+001440
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283570/ICMSHh7zg+001356
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283570/ICMSHh7zg+001380
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283570/ICMGGh7zg+001368
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283570/ICMUAh7zg+001356
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283570/ICMUAh7zg+001392
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283570/ICMUAh7zg+001440
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283570/ICMUAh7zg+001368
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283570/ICMGGh7zg+001380
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283570/ICMSHh7zg+001344
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283570/ICMGGh7zg+001428
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283570/ICMSHh7zg+001404
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12283570/ICMGGh7zg+001344
Zipping up the final file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a1cf_2016092300_20_1017_12283570_0_r934476021_14.zip
Uploading the final file: upload_file_14.zip
Uploading trickle at timestep: 1295100
14:37:28 (2902): called boinc_finish(0)

</stderr_txt>
<message>
upload failure: <file_xfer_error>
  <file_name>oifs_43r3_bl_a1cf_2016092300_20_1017_12283570_0_r934476021_15.zip</file_name>
  <error_code>-161 (not found)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>oifs_43r3_bl_a1cf_2016092300_20_1017_12283570_0_r934476021_16.zip</file_name>
  <error_code>-161 (not found)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>oifs_43r3_bl_a1cf_2016092300_20_1017_12283570_0_r934476021_17.zip</file_name>
  <error_code>-161 (not found)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>oifs_43r3_bl_a1cf_2016092300_20_1017_12283570_0_r934476021_18.zip</file_name>
  <error_code>-161 (not found)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>oifs_43r3_bl_a1cf_2016092300_20_1017_12283570_0_r934476021_19.zip</file_name>
  <error_code>-161 (not found)</error_code>
</file_xfer_error>
</message>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 Jun 2024 13:35:40 1506017 22435149 oifs_43r3_bl_a1cf_2016092300_20_1017_12283570_0 1,295,100 54,726 0.0423


©2024 cpdn.org