climateprediction.net home page
Task 22309928

Task 22309928

Name oifs_43r3_bl_a29o_2016092300_15_991_12209727_0
Workunit 12209727
Created 13 Feb 2023, 17:25:21 UTC
Sent 13 Feb 2023, 18:09:42 UTC
Report deadline 14 Apr 2023, 18:09:42 UTC
Received 14 Feb 2023, 1:24:07 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 1 (0x00000001) Unknown error code
Computer ID 1538400
Run time 6 min 16 sec
CPU time 3 min 50 sec
Validate state Invalid
Credit 0.00
Device peak FLOPS 1.00 GFLOPS
Application version OpenIFS 43r3 Baroclinic Lifecycle v1.11
x86_64-pc-linux-gnu
Peak working set size 4,813.47 MB
Peak swap size 5,930.78 MB
Peak disk usage 43.43 MB
Stderr 
<core_client_version>7.16.6</core_client_version>
<![CDATA[
<message>
process exited with code 1 (0x1, -255)</message>
<stderr_txt>
UM               MAXIMUM        
         AVE   0.000000000000000E+00 0.000000000000000E+00 0.000000000000000E+00
 GPNORM CSFL                 AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.000000000000000E+00 0.000000000000000E+00 0.000000000000000E+00
 GPNORM PFRC                 AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.000000000000000E+00 0.000000000000000E+00 0.000000000000000E+00
 NSTEP =     0 STEPO   A00000000
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
 NSTEP =     0 SCAN2M_HPOS  P
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ICMGGh7zg+000000 MODE=w
 IO-STREAM CLOSED - ICMGGh7zg+000000
 MPI-TASK:   1 -    603650378 BYTES IN      6 RECORDS TRANSFERRED IN    0.0002 SECONDS******** Mbytes/s, TOTAL TIME=    0.0971( 0.2%)
 NSTEP =     0 STEPO   0AA000000
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ./ICMSHh7zg+000000 MODE=a
 IO-STREAM CLOSED - ./ICMSHh7zg+000000
 MPI-TASK:   1 -    707765250 BYTES IN    202 RECORDS TRANSFERRED IN    0.0169 SECONDS 41879.6 Mbytes/s, TOTAL TIME=    0.5553( 3.0%)
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ./ICMUAh7zg+000000 MODE=a
 IO-STREAM CLOSED - ./ICMUAh7zg+000000
 MPI-TASK:   1 -   4897842839 BYTES IN    125 RECORDS TRANSFERRED IN    0.0063 SECONDS777435.4 Mbytes/s, TOTAL TIME=    0.3269( 1.9%)
 STEPO_FPOS WAS : B00000F00E
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ./ICMUAh7zg+000000 MODE=a
 IO-STREAM CLOSED - ./ICMUAh7zg+000000
 MPI-TASK:   1 -    297541545 BYTES IN      2 RECORDS TRANSFERRED IN    0.0003 SECONDS991805.2 Mbytes/s, TOTAL TIME=    0.0129( 2.3%)
 STEPO_FPOS WAS : V00000000Y
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ./ICMUAh7zg+000000 MODE=a
 IO-STREAM CLOSED - ./ICMUAh7zg+000000
 MPI-TASK:   1 -    306115178 BYTES IN      4 RECORDS TRANSFERRED IN    0.0006 SECONDS510192.0 Mbytes/s, TOTAL TIME=    0.0170( 3.5%)
 STEPO_FPOS WAS : T00000000M
 NSTEP =     0 STEPO   0AAA00AAA
------------------------------------------------
CNT0 not found; string returned was: 'STEPO'
>>> Printing last 7 lines from file: ifs.stat
 18:19:31 000000000 CNT3      -999     0.913    0.913    0.923      0:00      0:00 0.00000000000000E+00       1GB       0MB
 18:19:41 A00000000 STEPO        0     8.494    8.494   10.549      0:08      0:10 0.55515332309566E-07       2GB       0MB
 18:19:41 0AA000000 STEPO        0     0.145    0.145    0.146      0:08      0:10 0.55515332309566E-07       2GB       0MB
 18:19:44 FULLPOS-B DYNFPOS      0     2.579    2.579    2.592      0:11      0:13 0.55515332309566E-07       2GB       0MB
 18:19:51 FULLPOS-V DYNFPOS      0     7.091    7.091    7.120      0:18      0:20 0.55515332309566E-07       2GB       0MB
 18:19:53 FULLPOS-T DYNFPOS      0     2.380    2.380    2.404      0:20      0:22 0.55515332309566E-07       2GB       0MB
 18:19:55 0AAA00AAA STEPO        0     1.581    1.581    1.586      0:22      0:24 0.55515332309566E-07       2GB       0MB
------------------------------------------------
>>> Printing last 8 lines from file: /var/lib/boinc-client/slots/20/progress_file_12209727.xml

<running_values>
  <last_cpu_time>104.590000</last_cpu_time>
  <upload_file_number>0</upload_file_number>
  <last_iter>0</last_iter>
  <last_upload>0</last_upload>
  <model_completed>0</model_completed>
</running_values>
------------------------------------------------
..Failed, model did not complete successfully
(argv0) ../../projects/climateprediction.net/oifs_43r3_bl_1.11_x86_64-pc-linux-gnu
(argv1) start_date: 2016092300
(argv2) exptid: h7zg
(argv3) unique_member_id: a29o
(argv4) batchid: 991
(argv5) wuid: 12209727
(argv6) fclen: 15
(argv7) app_name: oifs_43r3_bl
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/20
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_bl
version: 1.11
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209727
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_app_1.11_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/20/oifs_43r3_bl_app_1.11_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/20/oifs_43r3_bl_app_1.11_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_9b764424e410f0488577abcf89a990e8 to: /var/lib/boinc-client/slots/20/oifs_43r3_bl_a29o_2016092300_15_991_12209727.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/20/oifs_43r3_bl_a29o_2016092300_15_991_12209727.zip
ic_ancil_file: ic_ancil_12209727
ifsdata_file: ifsdata_12209727
climate_data_file: clim_data_12209727
horiz_resolution: 159
vert_resolution: 91
grid_type: l_2
upload_interval: 96
utstep: 900
nfrres: restart dump frequency (steps) 48
Copying IC ancils from: ../../projects/climateprediction.net/jf_f7ecfd27aa1e68bbe7e8d19800e25c73 to: /var/lib/boinc-client/slots/20/ic_ancil_12209727.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/20/ic_ancil_12209727.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_3aebe97bec22696a929a1cca5c3c3cd1 to: /var/lib/boinc-client/slots/20/ifsdata/ifsdata_12209727.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/20/ifsdata/ifsdata_12209727.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/20/159l_2/clim_data_12209727.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/20/159l_2/clim_data_12209727.zip
Checking for progress XML file: /var/lib/boinc-client/slots/20/progress_file_12209727.xml
Opened progress file ok : /var/lib/boinc-client/slots/20/progress_file_12209727.xml
-- Model is restarting --
Adjusting last_iter, 0, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/20/progress_file_12209727.xml
last_cpu_time: 104
upload_file_number: 0
last_iter: 1
last_upload: 0
model_completed: 0
total_length_of_simulation: 1296000
result_base_name: oifs_43r3_bl_a29o_2016092300_15_991_12209727_0_r491768272
The child process has been launched with process id: 1532773
Executing the command: /var/lib/boinc-client/slots/20/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:LM32:1:1:1532773:1532773] [20230213:182111:1676330471:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2091] fp = 0x30e4a00
[EC_DRHOOK:LM32:1:1:1532773:1532773] [20230213:182111:1676330471:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:LM32:1:1:1532773:1532773] [20230213:182111:1676330471:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:LM32:1:1:1532773:1532773] [20230213:182111:1676330471:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:LM32:1:1:1532773:1532773] [20230213:182111:1676330471:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:LM32:1:1:1532773:1532773] [20230213:182111:1676330471:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:LM32:1:1:1532773:1532773] [20230213:182111:1676330471:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:LM32:1:1:1532773:1532773] [20230213:182111:1676330471:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:LM32:1:1:1532773:1532773] [20230213:182111:1676330471:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:LM32:1:1:1532773:1532773] [20230213:182111:1676330471:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:LM32:1:1:1532773:1532773] [20230213:182111:1676330471:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:LM32:1:1:1532773:1532773] [20230213:182111:1676330471:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:LM32:1:1:1532773:1532773] [20230213:182111:1676330471:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:LM32:1:1:1532773:1532773] [20230213:182111:1676330471:0.001] [ignore_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:LM32:1:1:1532773:1532773] [20230213:182111:1676330471:0.001] [restore_default_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:LM32:1:1:1532773:1532773] [20230213:182111:1676330471:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:LM32:1:1:1532773:1532773] [20230213:182111:1676330471:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f520 (old at (nil))
[EC_DRHOOK:LM32:1:1:1532773:1532773] [20230213:182111:1676330471:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:LM32:1:1:1532773:1532773] [20230213:182111:1676330471:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f520 (old at (nil))
[EC_DRHOOK:LM32:1:1:1532773:1532773] [20230213:182111:1676330471:0.001] [trapfpe_treatment@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:LM32:1:1:1532773:1532773] [20230213:182111:1676330471:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:LM32:1:1:1532773:1532773] [20230213:182111:1676330471:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:LM32:1:1:1532773:1532773] [20230213:182111:1676330471:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f520 (old at (nil))
[EC_DRHOOK:LM32:1:1:1532773:1532773] [20230213:182111:1676330471:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:LM32:1:1:1532773:1532773] [20230213:182111:1676330471:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:LM32:1:1:1532773:1532773] [20230213:182111:1676330471:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:LM32:1:1:1532773:1532773] [20230213:182111:1676330471:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f520 (old at (nil))
[EC_DRHOOK:LM32:1:1:1532773:1532773] [20230213:182111:1676330471:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f520 (old at (nil))
[EC_DRHOOK:LM32:1:1:1532773:1532773] [20230213:182111:1676330471:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f520 (old at (nil))
[EC_DRHOOK:LM32:1:1:1532773:1532773] [20230213:182111:1676330471:0.001] [catch_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_bl_1.11_x86_64-pc-linux-gnu
(argv1) start_date: 2016092300
(argv2) exptid: h7zg
(argv3) unique_member_id: a29o
(argv4) batchid: 991
(argv5) wuid: 12209727
(argv6) fclen: 15
(argv7) app_name: oifs_43r3_bl
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/20
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_bl
version: 1.11
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209727
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_app_1.11_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/20/oifs_43r3_bl_app_1.11_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/20/oifs_43r3_bl_app_1.11_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_9b764424e410f0488577abcf89a990e8 to: /var/lib/boinc-client/slots/20/oifs_43r3_bl_a29o_2016092300_15_991_12209727.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/20/oifs_43r3_bl_a29o_2016092300_15_991_12209727.zip
ic_ancil_file: ic_ancil_12209727
ifsdata_file: ifsdata_12209727
climate_data_file: clim_data_12209727
horiz_resolution: 159
vert_resolution: 91
grid_type: l_2
upload_interval: 96
utstep: 900
nfrres: restart dump frequency (steps) 48
Copying IC ancils from: ../../projects/climateprediction.net/jf_f7ecfd27aa1e68bbe7e8d19800e25c73 to: /var/lib/boinc-client/slots/20/ic_ancil_12209727.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/20/ic_ancil_12209727.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_3aebe97bec22696a929a1cca5c3c3cd1 to: /var/lib/boinc-client/slots/20/ifsdata/ifsdata_12209727.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/20/ifsdata/ifsdata_12209727.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/20/159l_2/clim_data_12209727.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/20/159l_2/clim_data_12209727.zip
Checking for progress XML file: /var/lib/boinc-client/slots/20/progress_file_12209727.xml
Opened progress file ok : /var/lib/boinc-client/slots/20/progress_file_12209727.xml
-- Model is restarting --
Adjusting last_iter, 0, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/20/progress_file_12209727.xml
last_cpu_time: 132
upload_file_number: 0
last_iter: 1
last_upload: 0
model_completed: 0
total_length_of_simulation: 1296000
result_base_name: oifs_43r3_bl_a29o_2016092300_15_991_12209727_0_r491768272
The child process has been launched with process id: 1534537
Executing the command: /var/lib/boinc-client/slots/20/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:LM32:1:1:1534537:1534537] [20230213:184438:1676331878:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2091] fp = 0x30e4a00
[EC_DRHOOK:LM32:1:1:1534537:1534537] [20230213:184438:1676331878:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:LM32:1:1:1534537:1534537] [20230213:184438:1676331878:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:LM32:1:1:1534537:1534537] [20230213:184438:1676331878:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:LM32:1:1:1534537:1534537] [20230213:184438:1676331878:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:LM32:1:1:1534537:1534537] [20230213:184438:1676331878:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:LM32:1:1:1534537:1534537] [20230213:184438:1676331878:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:LM32:1:1:1534537:1534537] [20230213:184438:1676331878:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:LM32:1:1:1534537:1534537] [20230213:184438:1676331878:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:LM32:1:1:1534537:1534537] [20230213:184438:1676331878:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:LM32:1:1:1534537:1534537] [20230213:184438:1676331878:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:LM32:1:1:1534537:1534537] [20230213:184438:1676331878:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:LM32:1:1:1534537:1534537] [20230213:184438:1676331878:0.001] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:LM32:1:1:1534537:1534537] [20230213:184438:1676331878:0.001] [ignore_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:LM32:1:1:1534537:1534537] [20230213:184438:1676331878:0.001] [restore_default_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:LM32:1:1:1534537:1534537] [20230213:184438:1676331878:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:LM32:1:1:1534537:1534537] [20230213:184438:1676331878:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f520 (old at (nil))
[EC_DRHOOK:LM32:1:1:1534537:1534537] [20230213:184438:1676331878:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:LM32:1:1:1534537:1534537] [20230213:184438:1676331878:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f520 (old at (nil))
[EC_DRHOOK:LM32:1:1:1534537:1534537] [20230213:184438:1676331878:0.001] [trapfpe_treatment@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:LM32:1:1:1534537:1534537] [20230213:184438:1676331878:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:LM32:1:1:1534537:1534537] [20230213:184438:1676331878:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:LM32:1:1:1534537:1534537] [20230213:184438:1676331878:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f520 (old at (nil))
[EC_DRHOOK:LM32:1:1:1534537:1534537] [20230213:184438:1676331878:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:LM32:1:1:1534537:1534537] [20230213:184438:1676331878:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:LM32:1:1:1534537:1534537] [20230213:184438:1676331878:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:LM32:1:1:1534537:1534537] [20230213:184438:1676331878:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f520 (old at (nil))
[EC_DRHOOK:LM32:1:1:1534537:1534537] [20230213:184438:1676331878:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f520 (old at (nil))
[EC_DRHOOK:LM32:1:1:1534537:1534537] [20230213:184438:1676331878:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f520 (old at (nil))
[EC_DRHOOK:LM32:1:1:1534537:1534537] [20230213:184438:1676331878:0.001] [catch_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_bl_1.11_x86_64-pc-linux-gnu
(argv1) start_date: 2016092300
(argv2) exptid: h7zg
(argv3) unique_member_id: a29o
(argv4) batchid: 991
(argv5) wuid: 12209727
(argv6) fclen: 15
(argv7) app_name: oifs_43r3_bl
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/20
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_bl
version: 1.11
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209727
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_app_1.11_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/20/oifs_43r3_bl_app_1.11_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/20/oifs_43r3_bl_app_1.11_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_9b764424e410f0488577abcf89a990e8 to: /var/lib/boinc-client/slots/20/oifs_43r3_bl_a29o_2016092300_15_991_12209727.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/20/oifs_43r3_bl_a29o_2016092300_15_991_12209727.zip
ic_ancil_file: ic_ancil_12209727
ifsdata_file: ifsdata_12209727
climate_data_file: clim_data_12209727
horiz_resolution: 159
vert_resolution: 91
grid_type: l_2
upload_interval: 96
utstep: 900
nfrres: restart dump frequency (steps) 48
Copying IC ancils from: ../../projects/climateprediction.net/jf_f7ecfd27aa1e68bbe7e8d19800e25c73 to: /var/lib/boinc-client/slots/20/ic_ancil_12209727.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/20/ic_ancil_12209727.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_3aebe97bec22696a929a1cca5c3c3cd1 to: /var/lib/boinc-client/slots/20/ifsdata/ifsdata_12209727.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/20/ifsdata/ifsdata_12209727.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/20/159l_2/clim_data_12209727.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/20/159l_2/clim_data_12209727.zip
Checking for progress XML file: /var/lib/boinc-client/slots/20/progress_file_12209727.xml
Opened progress file ok : /var/lib/boinc-client/slots/20/progress_file_12209727.xml
-- Model is restarting --
Adjusting last_iter, 0, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/20/progress_file_12209727.xml
last_cpu_time: 140
upload_file_number: 0
last_iter: 1
last_upload: 0
model_completed: 0
total_length_of_simulation: 1296000
result_base_name: oifs_43r3_bl_a29o_2016092300_15_991_12209727_0_r491768272
The child process has been launched with process id: 1534619
Executing the command: /var/lib/boinc-client/slots/20/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:LM32:1:1:1534619:1534619] [20230213:184600:1676331960:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2091] fp = 0x30e4a00
[EC_DRHOOK:LM32:1:1:1534619:1534619] [20230213:184600:1676331960:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:LM32:1:1:1534619:1534619] [20230213:184600:1676331960:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:LM32:1:1:1534619:1534619] [20230213:184600:1676331960:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:LM32:1:1:1534619:1534619] [20230213:184600:1676331960:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:LM32:1:1:1534619:1534619] [20230213:184600:1676331960:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:LM32:1:1:1534619:1534619] [20230213:184600:1676331960:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:LM32:1:1:1534619:1534619] [20230213:184600:1676331960:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:LM32:1:1:1534619:1534619] [20230213:184600:1676331960:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:LM32:1:1:1534619:1534619] [20230213:184600:1676331960:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:LM32:1:1:1534619:1534619] [20230213:184600:1676331960:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:LM32:1:1:1534619:1534619] [20230213:184600:1676331960:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:LM32:1:1:1534619:1534619] [20230213:184600:1676331960:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:LM32:1:1:1534619:1534619] [20230213:184600:1676331960:0.000] [ignore_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:LM32:1:1:1534619:1534619] [20230213:184600:1676331960:0.000] [restore_default_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:LM32:1:1:1534619:1534619] [20230213:184600:1676331960:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:LM32:1:1:1534619:1534619] [20230213:184600:1676331960:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f520 (old at (nil))
[EC_DRHOOK:LM32:1:1:1534619:1534619] [20230213:184600:1676331960:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:LM32:1:1:1534619:1534619] [20230213:184600:1676331960:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f520 (old at (nil))
[EC_DRHOOK:LM32:1:1:1534619:1534619] [20230213:184600:1676331960:0.000] [trapfpe_treatment@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:LM32:1:1:1534619:1534619] [20230213:184600:1676331960:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:LM32:1:1:1534619:1534619] [20230213:184600:1676331960:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:LM32:1:1:1534619:1534619] [20230213:184600:1676331960:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f520 (old at (nil))
[EC_DRHOOK:LM32:1:1:1534619:1534619] [20230213:184600:1676331960:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:LM32:1:1:1534619:1534619] [20230213:184600:1676331960:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:LM32:1:1:1534619:1534619] [20230213:184600:1676331960:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:LM32:1:1:1534619:1534619] [20230213:184600:1676331960:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f520 (old at (nil))
[EC_DRHOOK:LM32:1:1:1534619:1534619] [20230213:184600:1676331960:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f520 (old at (nil))
[EC_DRHOOK:LM32:1:1:1534619:1534619] [20230213:184600:1676331960:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f520 (old at (nil))
[EC_DRHOOK:LM32:1:1:1534619:1534619] [20230213:184600:1676331960:0.001] [catch_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_bl_1.11_x86_64-pc-linux-gnu
(argv1) start_date: 2016092300
(argv2) exptid: h7zg
(argv3) unique_member_id: a29o
(argv4) batchid: 991
(argv5) wuid: 12209727
(argv6) fclen: 15
(argv7) app_name: oifs_43r3_bl
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/20
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_bl
version: 1.11
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209727
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_app_1.11_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/20/oifs_43r3_bl_app_1.11_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/20/oifs_43r3_bl_app_1.11_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_9b764424e410f0488577abcf89a990e8 to: /var/lib/boinc-client/slots/20/oifs_43r3_bl_a29o_2016092300_15_991_12209727.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/20/oifs_43r3_bl_a29o_2016092300_15_991_12209727.zip
ic_ancil_file: ic_ancil_12209727
ifsdata_file: ifsdata_12209727
climate_data_file: clim_data_12209727
horiz_resolution: 159
vert_resolution: 91
grid_type: l_2
upload_interval: 96
utstep: 900
nfrres: restart dump frequency (steps) 48
Copying IC ancils from: ../../projects/climateprediction.net/jf_f7ecfd27aa1e68bbe7e8d19800e25c73 to: /var/lib/boinc-client/slots/20/ic_ancil_12209727.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/20/ic_ancil_12209727.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_3aebe97bec22696a929a1cca5c3c3cd1 to: /var/lib/boinc-client/slots/20/ifsdata/ifsdata_12209727.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/20/ifsdata/ifsdata_12209727.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/20/159l_2/clim_data_12209727.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/20/159l_2/clim_data_12209727.zip
Checking for progress XML file: /var/lib/boinc-client/slots/20/progress_file_12209727.xml
Opened progress file ok : /var/lib/boinc-client/slots/20/progress_file_12209727.xml
-- Model is restarting --
Adjusting last_iter, 0, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/20/progress_file_12209727.xml
last_cpu_time: 168
upload_file_number: 0
last_iter: 1
last_upload: 0
model_completed: 0
total_length_of_simulation: 1296000
result_base_name: oifs_43r3_bl_a29o_2016092300_15_991_12209727_0_r491768272
The child process has been launched with process id: 1534800
Executing the command: /var/lib/boinc-client/slots/20/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:LM32:1:1:1534800:1534800] [20230213:184903:1676332143:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2091] fp = 0x30e4a00
[EC_DRHOOK:LM32:1:1:1534800:1534800] [20230213:184903:1676332143:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:LM32:1:1:1534800:1534800] [20230213:184903:1676332143:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:LM32:1:1:1534800:1534800] [20230213:184903:1676332143:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:LM32:1:1:1534800:1534800] [20230213:184903:1676332143:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:LM32:1:1:1534800:1534800] [20230213:184903:1676332143:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:LM32:1:1:1534800:1534800] [20230213:184903:1676332143:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:LM32:1:1:1534800:1534800] [20230213:184903:1676332143:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:LM32:1:1:1534800:1534800] [20230213:184903:1676332143:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:LM32:1:1:1534800:1534800] [20230213:184903:1676332143:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:LM32:1:1:1534800:1534800] [20230213:184903:1676332143:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:LM32:1:1:1534800:1534800] [20230213:184903:1676332143:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:LM32:1:1:1534800:1534800] [20230213:184903:1676332143:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:LM32:1:1:1534800:1534800] [20230213:184903:1676332143:0.000] [ignore_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:LM32:1:1:1534800:1534800] [20230213:184903:1676332143:0.000] [restore_default_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:LM32:1:1:1534800:1534800] [20230213:184903:1676332143:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:LM32:1:1:1534800:1534800] [20230213:184903:1676332143:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f520 (old at (nil))
[EC_DRHOOK:LM32:1:1:1534800:1534800] [20230213:184903:1676332143:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:LM32:1:1:1534800:1534800] [20230213:184903:1676332143:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f520 (old at (nil))
[EC_DRHOOK:LM32:1:1:1534800:1534800] [20230213:184903:1676332143:0.001] [trapfpe_treatment@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:LM32:1:1:1534800:1534800] [20230213:184903:1676332143:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:LM32:1:1:1534800:1534800] [20230213:184903:1676332143:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:LM32:1:1:1534800:1534800] [20230213:184903:1676332143:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f520 (old at (nil))
[EC_DRHOOK:LM32:1:1:1534800:1534800] [20230213:184903:1676332143:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:LM32:1:1:1534800:1534800] [20230213:184903:1676332143:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:LM32:1:1:1534800:1534800] [20230213:184903:1676332143:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:LM32:1:1:1534800:1534800] [20230213:184903:1676332143:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f520 (old at (nil))
[EC_DRHOOK:LM32:1:1:1534800:1534800] [20230213:184903:1676332143:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f520 (old at (nil))
[EC_DRHOOK:LM32:1:1:1534800:1534800] [20230213:184903:1676332143:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f520 (old at (nil))
[EC_DRHOOK:LM32:1:1:1534800:1534800] [20230213:184903:1676332143:0.001] [catch_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_bl_1.11_x86_64-pc-linux-gnu
(argv1) start_date: 2016092300
(argv2) exptid: h7zg
(argv3) unique_member_id: a29o
(argv4) batchid: 991
(argv5) wuid: 12209727
(argv6) fclen: 15
(argv7) app_name: oifs_43r3_bl
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/20
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_bl
version: 1.11
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209727
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_app_1.11_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/20/oifs_43r3_bl_app_1.11_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/20/oifs_43r3_bl_app_1.11_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_9b764424e410f0488577abcf89a990e8 to: /var/lib/boinc-client/slots/20/oifs_43r3_bl_a29o_2016092300_15_991_12209727.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/20/oifs_43r3_bl_a29o_2016092300_15_991_12209727.zip
ic_ancil_file: ic_ancil_12209727
ifsdata_file: ifsdata_12209727
climate_data_file: clim_data_12209727
horiz_resolution: 159
vert_resolution: 91
grid_type: l_2
upload_interval: 96
utstep: 900
nfrres: restart dump frequency (steps) 48
Copying IC ancils from: ../../projects/climateprediction.net/jf_f7ecfd27aa1e68bbe7e8d19800e25c73 to: /var/lib/boinc-client/slots/20/ic_ancil_12209727.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/20/ic_ancil_12209727.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_3aebe97bec22696a929a1cca5c3c3cd1 to: /var/lib/boinc-client/slots/20/ifsdata/ifsdata_12209727.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/20/ifsdata/ifsdata_12209727.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/20/159l_2/clim_data_12209727.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/20/159l_2/clim_data_12209727.zip
Checking for progress XML file: /var/lib/boinc-client/slots/20/progress_file_12209727.xml
Opened progress file ok : /var/lib/boinc-client/slots/20/progress_file_12209727.xml
-- Model is restarting --
Adjusting last_iter, 0, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/20/progress_file_12209727.xml
last_cpu_time: 186
upload_file_number: 0
last_iter: 1
last_upload: 0
model_completed: 0
total_length_of_simulation: 1296000
result_base_name: oifs_43r3_bl_a29o_2016092300_15_991_12209727_0_r491768272
The child process has been launched with process id: 1534954
Executing the command: /var/lib/boinc-client/slots/20/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:LM32:1:1:1534954:1534954] [20230213:185114:1676332274:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2091] fp = 0x30e4a00
[EC_DRHOOK:LM32:1:1:1534954:1534954] [20230213:185114:1676332274:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:LM32:1:1:1534954:1534954] [20230213:185114:1676332274:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:LM32:1:1:1534954:1534954] [20230213:185114:1676332274:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:LM32:1:1:1534954:1534954] [20230213:185114:1676332274:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:LM32:1:1:1534954:1534954] [20230213:185114:1676332274:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:LM32:1:1:1534954:1534954] [20230213:185114:1676332274:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:LM32:1:1:1534954:1534954] [20230213:185114:1676332274:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:LM32:1:1:1534954:1534954] [20230213:185114:1676332274:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:LM32:1:1:1534954:1534954] [20230213:185114:1676332274:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:LM32:1:1:1534954:1534954] [20230213:185114:1676332274:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:LM32:1:1:1534954:1534954] [20230213:185114:1676332274:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:LM32:1:1:1534954:1534954] [20230213:185114:1676332274:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:LM32:1:1:1534954:1534954] [20230213:185114:1676332274:0.000] [ignore_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:LM32:1:1:1534954:1534954] [20230213:185114:1676332274:0.001] [restore_default_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:LM32:1:1:1534954:1534954] [20230213:185114:1676332274:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:LM32:1:1:1534954:1534954] [20230213:185114:1676332274:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f520 (old at (nil))
[EC_DRHOOK:LM32:1:1:1534954:1534954] [20230213:185114:1676332274:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:LM32:1:1:1534954:1534954] [20230213:185114:1676332274:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f520 (old at (nil))
[EC_DRHOOK:LM32:1:1:1534954:1534954] [20230213:185114:1676332274:0.001] [trapfpe_treatment@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:LM32:1:1:1534954:1534954] [20230213:185114:1676332274:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:LM32:1:1:1534954:1534954] [20230213:185114:1676332274:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:LM32:1:1:1534954:1534954] [20230213:185114:1676332274:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f520 (old at (nil))
[EC_DRHOOK:LM32:1:1:1534954:1534954] [20230213:185114:1676332274:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:LM32:1:1:1534954:1534954] [20230213:185114:1676332274:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:LM32:1:1:1534954:1534954] [20230213:185114:1676332274:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:LM32:1:1:1534954:1534954] [20230213:185114:1676332274:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f520 (old at (nil))
[EC_DRHOOK:LM32:1:1:1534954:1534954] [20230213:185114:1676332274:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f520 (old at (nil))
[EC_DRHOOK:LM32:1:1:1534954:1534954] [20230213:185114:1676332274:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f520 (old at (nil))
[EC_DRHOOK:LM32:1:1:1534954:1534954] [20230213:185114:1676332274:0.001] [catch_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
..The child process has been killed with signal: 9
>>> Printing last 70 lines from file: NODE.001_01

Closing NetCDF file ifsdata/aerosol_cams_climatology_43R3a.nc
 UPDTIM MACC AEROSOL CLIMATOLOGY IS LOADED
SUECAEC: INLEV,ISYEAR,IEYEAR=  0  1957  2010
 suecaec NAERMACC =  1
 UPDT RAESHDU =  2000.0
 SUECAEC: GLOBAL AVERAGE VOLCANIC SULPHATE FOR (KINDAT,KMINUT)     20160923
          90  9.999999999999998E-005
  DATE= 2016    9   23
 NORMS AT NSTEP CNT4    0
 SPECTRAL NORMS -   LOG(PREHYDS)    0.115129254649702E+02
 LEV     VORTICITY             DIVERGENCE           TEMPERATURE           KINETIC ENERGY    
 AVE 0.515897411628180E-05 0.555153323095658E-07 0.203801492760430E+03 0.467483430978028E+02
 GPNORM  HUMIDITY            AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.147230402520698E-03 0.000000000000000E+00 0.115672278969467E-02
 GPNORM  LIQUID WATER        AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.000000000000000E+00 0.000000000000000E+00 0.000000000000000E+00
 GPNORM  ICE WATER           AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.000000000000000E+00 0.000000000000000E+00 0.000000000000000E+00
 GPNORM SNOW                 AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.000000000000000E+00 0.000000000000000E+00 0.000000000000000E+00
 GPNORM RAIN                 AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.000000000000000E+00 0.000000000000000E+00 0.000000000000000E+00
 GPNORM  CLOUD FRACTION      AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.000000000000000E+00 0.000000000000000E+00 0.000000000000000E+00
 GPNORM CLW                  AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.000000000000000E+00 0.000000000000000E+00 0.000000000000000E+00
 GPNORM CIW                  AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.000000000000000E+00 0.000000000000000E+00 0.000000000000000E+00
 GPNORM CC                   AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.000000000000000E+00 0.000000000000000E+00 0.000000000000000E+00
 GPNORM RFL                  AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.000000000000000E+00 0.000000000000000E+00 0.000000000000000E+00
 GPNORM SFL                  AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.000000000000000E+00 0.000000000000000E+00 0.000000000000000E+00
 GPNORM CRFL                 AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.000000000000000E+00 0.000000000000000E+00 0.000000000000000E+00
 GPNORM CSFL                 AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.000000000000000E+00 0.000000000000000E+00 0.000000000000000E+00
 GPNORM PFRC                 AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.000000000000000E+00 0.000000000000000E+00 0.000000000000000E+00
 NSTEP =     0 STEPO   A00000000
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
 NSTEP =     0 SCAN2M_HPOS  P
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ICMGGh7zg+000000 MODE=w
 IO-STREAM CLOSED - ICMGGh7zg+000000
 MPI-TASK:   1 -    593975626 BYTES IN      6 RECORDS TRANSFERRED IN    0.0002 SECONDS******** Mbytes/s, TOTAL TIME=    0.0896( 0.2%)
 NSTEP =     0 STEPO   0AA000000
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ./ICMSHh7zg+000000 MODE=a
 IO-STREAM CLOSED - ./ICMSHh7zg+000000
 MPI-TASK:   1 -    707765250 BYTES IN    202 RECORDS TRANSFERRED IN    0.0178 SECONDS 39762.1 Mbytes/s, TOTAL TIME=    0.6601( 2.7%)
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ./ICMUAh7zg+000000 MODE=a
 IO-STREAM CLOSED - ./ICMUAh7zg+000000
 MPI-TASK:   1 -   4820444823 BYTES IN    125 RECORDS TRANSFERRED IN    0.0061 SECONDS790236.9 Mbytes/s, TOTAL TIME=    0.3244( 1.9%)
 STEPO_FPOS WAS : B00000F00E
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ./ICMUAh7zg+000000 MODE=a
 IO-STREAM CLOSED - ./ICMUAh7zg+000000
 MPI-TASK:   1 -    292704169 BYTES IN      2 RECORDS TRANSFERRED IN    0.0003 SECONDS975680.6 Mbytes/s, TOTAL TIME=    0.0090( 3.3%)
 STEPO_FPOS WAS : V00000000Y
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ./ICMUAh7zg+000000 MODE=a
 IO-STREAM CLOSED - ./ICMUAh7zg+000000
 MPI-TASK:   1 -    301277802 BYTES IN      4 RECORDS TRANSFERRED IN    0.0005 SECONDS602555.6 Mbytes/s, TOTAL TIME=    0.0134( 3.7%)
 STEPO_FPOS WAS : T00000000M
 NSTEP =     0 STEPO   0AAA00AAA
------------------------------------------------
CNT0 not found; string returned was: 'STEPO'
>>> Printing last 7 lines from file: ifs.stat
 18:51:15 000000000 CNT3      -999     0.946    0.946    1.008      0:00      0:00 0.00000000000000E+00       1GB       0MB
 18:51:25 A00000000 STEPO        0     9.434    9.434   11.624      0:09      0:11 0.55515332309566E-07       2GB       0MB
 18:51:26 0AA000000 STEPO        0     0.231    0.231    0.252      0:09      0:11 0.55515332309566E-07       2GB       0MB
 18:51:28 FULLPOS-B DYNFPOS      0     2.675    2.675    2.872      0:12      0:14 0.55515332309566E-07       2GB       0MB
 18:51:38 FULLPOS-V DYNFPOS      0     7.628    7.628    9.690      0:19      0:24 0.55515332309566E-07       2GB       0MB
 18:51:41 FULLPOS-T DYNFPOS      0     1.182    1.182    3.205      0:21      0:27 0.55515332309566E-07       2GB       0MB
 18:51:43 0AAA00AAA STEPO        0     1.441    1.441    1.533      0:22      0:29 0.55515332309566E-07       2GB       0MB
------------------------------------------------
>>> Printing last 8 lines from file: /var/lib/boinc-client/slots/20/progress_file_12209727.xml

<running_values>
  <last_cpu_time>224.420000</last_cpu_time>
  <upload_file_number>0</upload_file_number>
  <last_iter>0</last_iter>
  <last_upload>0</last_upload>
  <model_completed>0</model_completed>
</running_values>
------------------------------------------------
..Failed, model did not complete successfully

</stderr_txt>
]]>
No trickles!


©2024 cpdn.org