<core_client_version>7.16.6</core_client_version>
<![CDATA[
<message>
Disk usage limit exceeded</message>
<stderr_txt>
356:1674371036:0.005] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:test-System-Product-Name:1:1:155972:155972] [20230122:010356:1674371036:0.005] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:test-System-Product-Name:1:1:155972:155972] [20230122:010356:1674371036:0.005] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:test-System-Product-Name:1:1:155972:155972] [20230122:010356:1674371036:0.005] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:test-System-Product-Name:1:1:155972:155972] [20230122:010356:1674371036:0.005] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:test-System-Product-Name:1:1:155972:155972] [20230122:010356:1674371036:0.005] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:test-System-Product-Name:1:1:155972:155972] [20230122:010356:1674371036:0.006] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:test-System-Product-Name:1:1:155972:155972] [20230122:010356:1674371036:0.006] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:test-System-Product-Name:1:1:155972:155972] [20230122:010356:1674371036:0.006] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:test-System-Product-Name:1:1:155972:155972] [20230122:010356:1674371036:0.006] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:test-System-Product-Name:1:1:155972:155972] [20230122:010356:1674371036:0.006] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:test-System-Product-Name:1:1:155972:155972] [20230122:010356:1674371036:0.006] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
01:07:58 STEP 288 H= 288:00 +CPU=216.078
01:09:57 STEP 289 H= 289:00 +CPU=118.136
01:12:24 STEP 290 H= 290:00 +CPU=146.180
01:15:36 STEP 291 H= 291:00 +CPU=190.226
01:17:42 STEP 292 H= 292:00 +CPU=124.627
01:20:03 STEP 293 H= 293:00 +CPU=140.150
01:23:29 STEP 294 H= 294:00 +CPU=204.953
01:25:31 STEP 295 H= 295:00 +CPU=120.538
01:27:36 STEP 296 H= 296:00 +CPU=123.976
01:30:52 STEP 297 H= 297:00 +CPU=194.520
01:33:05 STEP 298 H= 298:00 +CPU=132.395
01:35:06 STEP 299 H= 299:00 +CPU=119.897
01:38:49 STEP 300 H= 300:00 +CPU=221.477
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMGGhq0f+000300
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMSHhq0f+000300
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMUAhq0f+000300
01:40:52 STEP 301 H= 301:00 +CPU=121.642
01:42:58 STEP 302 H= 302:00 +CPU=125.850
01:46:12 STEP 303 H= 303:00 +CPU=191.864
01:48:12 STEP 304 H= 304:00 +CPU=119.819
01:50:19 STEP 305 H= 305:00 +CPU=125.770
01:53:23 STEP 306 H= 306:00 +CPU=182.665
01:55:35 STEP 307 H= 307:00 +CPU=130.767
01:57:46 STEP 308 H= 308:00 +CPU=130.432
02:00:46 STEP 309 H= 309:00 +CPU=177.848
02:03:02 STEP 310 H= 310:00 +CPU=134.902
02:05:02 STEP 311 H= 311:00 +CPU=119.407
02:08:38 STEP 312 H= 312:00 +CPU=214.067
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMGGhq0f+000312
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMSHhq0f+000312
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMUAhq0f+000312
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12170199/ICMGGhq0f+000288
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12170199/ICMSHhq0f+000288
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12170199/ICMUAhq0f+000288
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12170199/ICMGGhq0f+000300
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12170199/ICMSHhq0f+000300
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12170199/ICMUAhq0f+000300
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0555_1984050100_123_953_12170199_1_r600327352_12.zip
Uploading the intermediate file: upload_file_12.zip
02:10:59 STEP 313 H= 313:00 +CPU=140.572
Quit request received from BOINC client, ending the child process
..Failed, model did not complete successfully
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 1984050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0555
(argv4) batchid: 953
(argv5) wuid: 12170199
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/12
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12170199
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/12/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/12/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_bd1fdd38845ecd99cd334f92d97fd530 to: /var/lib/boinc-client/slots/12/oifs_43r3_ps_0555_1984050100_123_953_12170199.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/12/oifs_43r3_ps_0555_1984050100_123_953_12170199.zip
ic_ancil_file: ic_ancil_12170199
ifsdata_file: ifsdata_12170199
climate_data_file: clim_data_12170199
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_67961e968080678a3d119c18e75f553d to: /var/lib/boinc-client/slots/12/ic_ancil_12170199.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/12/ic_ancil_12170199.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_f30ea0e48dfaead6ea733c4b4d776e78 to: /var/lib/boinc-client/slots/12/ifsdata/ifsdata_12170199.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/12/ifsdata/ifsdata_12170199.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/12/159l_2/clim_data_12170199.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/12/159l_2/clim_data_12170199.zip
Checking for progress XML file: /var/lib/boinc-client/slots/12/progress_file_12170199.xml
Opened progress file ok : /var/lib/boinc-client/slots/12/progress_file_12170199.xml
-- Model is restarting --
Adjusting last_iter, 314, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/12/progress_file_12170199.xml
last_cpu_time: 64355
upload_file_number: 13
last_iter: 311
last_upload: 1123200
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0555_1984050100_123_953_12170199_1_r600327352
The child process has been launched with process id: 156285
Executing the command: /var/lib/boinc-client/slots/12/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:test-System-Product-Name:1:1:156285:156285] [20230122:021259:1674375179:0.002] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:test-System-Product-Name:1:1:156285:156285] [20230122:021259:1674375179:0.002] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:test-System-Product-Name:1:1:156285:156285] [20230122:021259:1674375179:0.002] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:test-System-Product-Name:1:1:156285:156285] [20230122:021259:1674375179:0.002] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:test-System-Product-Name:1:1:156285:156285] [20230122:021259:1674375179:0.002] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:test-System-Product-Name:1:1:156285:156285] [20230122:021259:1674375179:0.002] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647)
[EC_DRHOOK:test-System-Product-Name:1:1:156285:156285] [20230122:021259:1674375179:0.003] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:test-System-Product-Name:1:1:156285:156285] [20230122:021259:1674375179:0.003] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:test-System-Product-Name:1:1:156285:156285] [20230122:021259:1674375179:0.003] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:test-System-Product-Name:1:1:156285:156285] [20230122:021259:1674375179:0.003] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:test-System-Product-Name:1:1:156285:156285] [20230122:021259:1674375179:0.003] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:test-System-Product-Name:1:1:156285:156285] [20230122:021259:1674375179:0.004] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:test-System-Product-Name:1:1:156285:156285] [20230122:021259:1674375179:0.004] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:test-System-Product-Name:1:1:156285:156285] [20230122:021259:1674375179:0.004] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:test-System-Product-Name:1:1:156285:156285] [20230122:021259:1674375179:0.004] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:test-System-Product-Name:1:1:156285:156285] [20230122:021259:1674375179:0.004] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:test-System-Product-Name:1:1:156285:156285] [20230122:021259:1674375179:0.005] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:test-System-Product-Name:1:1:156285:156285] [20230122:021259:1674375179:0.005] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:test-System-Product-Name:1:1:156285:156285] [20230122:021259:1674375179:0.005] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:test-System-Product-Name:1:1:156285:156285] [20230122:021259:1674375179:0.005] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:test-System-Product-Name:1:1:156285:156285] [20230122:021259:1674375179:0.005] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:test-System-Product-Name:1:1:156285:156285] [20230122:021259:1674375179:0.005] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:test-System-Product-Name:1:1:156285:156285] [20230122:021259:1674375179:0.005] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:test-System-Product-Name:1:1:156285:156285] [20230122:021259:1674375179:0.006] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:test-System-Product-Name:1:1:156285:156285] [20230122:021259:1674375179:0.006] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:test-System-Product-Name:1:1:156285:156285] [20230122:021259:1674375179:0.006] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:test-System-Product-Name:1:1:156285:156285] [20230122:021259:1674375179:0.006] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:test-System-Product-Name:1:1:156285:156285] [20230122:021259:1674375179:0.006] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:test-System-Product-Name:1:1:156285:156285] [20230122:021259:1674375179:0.007] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:test-System-Product-Name:1:1:156285:156285] [20230122:021259:1674375179:0.007] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
02:17:12 STEP 312 H= 312:00 +CPU=226.882
02:19:30 STEP 313 H= 313:00 +CPU=136.802
02:21:41 STEP 314 H= 314:00 +CPU=130.030
02:24:44 STEP 315 H= 315:00 +CPU=181.729
02:26:59 STEP 316 H= 316:00 +CPU=134.259
02:29:20 STEP 317 H= 317:00 +CPU=139.523
02:32:32 STEP 318 H= 318:00 +CPU=190.749
02:34:37 STEP 319 H= 319:00 +CPU=124.302
02:37:02 STEP 320 H= 320:00 +CPU=143.506
02:40:17 STEP 321 H= 321:00 +CPU=193.452
02:42:26 STEP 322 H= 322:00 +CPU=128.043
02:44:36 STEP 323 H= 323:00 +CPU=129.824
02:48:10 STEP 324 H= 324:00 +CPU=212.355
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMGGhq0f+000324
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMSHhq0f+000324
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMUAhq0f+000324
02:50:18 STEP 325 H= 325:00 +CPU=126.286
02:52:41 STEP 326 H= 326:00 +CPU=141.769
02:55:41 STEP 327 H= 327:00 +CPU=178.727
02:57:55 STEP 328 H= 328:00 +CPU=132.789
03:00:04 STEP 329 H= 329:00 +CPU=128.864
03:03:22 STEP 330 H= 330:00 +CPU=196.534
03:05:43 STEP 331 H= 331:00 +CPU=139.443
03:08:00 STEP 332 H= 332:00 +CPU=135.789
03:11:11 STEP 333 H= 333:00 +CPU=190.215
03:13:34 STEP 334 H= 334:00 +CPU=141.822
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 1984050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0555
(argv4) batchid: 953
(argv5) wuid: 12170199
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/12
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12170199
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/12/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/12/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_bd1fdd38845ecd99cd334f92d97fd530 to: /var/lib/boinc-client/slots/12/oifs_43r3_ps_0555_1984050100_123_953_12170199.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/12/oifs_43r3_ps_0555_1984050100_123_953_12170199.zip
ic_ancil_file: ic_ancil_12170199
ifsdata_file: ifsdata_12170199
climate_data_file: clim_data_12170199
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_67961e968080678a3d119c18e75f553d to: /var/lib/boinc-client/slots/12/ic_ancil_12170199.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/12/ic_ancil_12170199.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_f30ea0e48dfaead6ea733c4b4d776e78 to: /var/lib/boinc-client/slots/12/ifsdata/ifsdata_12170199.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/12/ifsdata/ifsdata_12170199.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/12/159l_2/clim_data_12170199.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/12/159l_2/clim_data_12170199.zip
Checking for progress XML file: /var/lib/boinc-client/slots/12/progress_file_12170199.xml
Opened progress file ok : /var/lib/boinc-client/slots/12/progress_file_12170199.xml
-- Model is restarting --
Adjusting last_iter, 335, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/12/progress_file_12170199.xml
last_cpu_time: 68000
upload_file_number: 13
last_iter: 311
last_upload: 1123200
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0555_1984050100_123_953_12170199_1_r600327352
The child process has been launched with process id: 156555
Executing the command: /var/lib/boinc-client/slots/12/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:test-System-Product-Name:1:1:156555:156555] [20230122:031530:1674378930:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:test-System-Product-Name:1:1:156555:156555] [20230122:031530:1674378930:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:test-System-Product-Name:1:1:156555:156555] [20230122:031530:1674378930:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:test-System-Product-Name:1:1:156555:156555] [20230122:031530:1674378930:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:test-System-Product-Name:1:1:156555:156555] [20230122:031530:1674378930:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:test-System-Product-Name:1:1:156555:156555] [20230122:031530:1674378930:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647)
[EC_DRHOOK:test-System-Product-Name:1:1:156555:156555] [20230122:031530:1674378930:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:test-System-Product-Name:1:1:156555:156555] [20230122:031530:1674378930:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:test-System-Product-Name:1:1:156555:156555] [20230122:031530:1674378930:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:test-System-Product-Name:1:1:156555:156555] [20230122:031530:1674378930:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:test-System-Product-Name:1:1:156555:156555] [20230122:031530:1674378930:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:test-System-Product-Name:1:1:156555:156555] [20230122:031530:1674378930:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:test-System-Product-Name:1:1:156555:156555] [20230122:031530:1674378930:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:test-System-Product-Name:1:1:156555:156555] [20230122:031530:1674378930:0.001] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:test-System-Product-Name:1:1:156555:156555] [20230122:031530:1674378930:0.001] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:test-System-Product-Name:1:1:156555:156555] [20230122:031530:1674378930:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:test-System-Product-Name:1:1:156555:156555] [20230122:031530:1674378930:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:test-System-Product-Name:1:1:156555:156555] [20230122:031530:1674378930:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:test-System-Product-Name:1:1:156555:156555] [20230122:031530:1674378930:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:test-System-Product-Name:1:1:156555:156555] [20230122:031530:1674378930:0.001] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:test-System-Product-Name:1:1:156555:156555] [20230122:031530:1674378930:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:test-System-Product-Name:1:1:156555:156555] [20230122:031530:1674378930:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:test-System-Product-Name:1:1:156555:156555] [20230122:031530:1674378930:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:test-System-Product-Name:1:1:156555:156555] [20230122:031530:1674378930:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:test-System-Product-Name:1:1:156555:156555] [20230122:031530:1674378930:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:test-System-Product-Name:1:1:156555:156555] [20230122:031530:1674378930:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:test-System-Product-Name:1:1:156555:156555] [20230122:031530:1674378930:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:test-System-Product-Name:1:1:156555:156555] [20230122:031530:1674378930:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:test-System-Product-Name:1:1:156555:156555] [20230122:031530:1674378930:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:test-System-Product-Name:1:1:156555:156555] [20230122:031530:1674378930:0.001] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
03:19:44 STEP 312 H= 312:00 +CPU=230.442
03:22:09 STEP 313 H= 313:00 +CPU=144.354
03:24:30 STEP 314 H= 314:00 +CPU=139.372
03:27:49 STEP 315 H= 315:00 +CPU=197.642
03:29:57 STEP 316 H= 316:00 +CPU=127.386
03:32:14 STEP 317 H= 317:00 +CPU=136.177
03:35:34 STEP 318 H= 318:00 +CPU=198.238
03:37:39 STEP 319 H= 319:00 +CPU=123.910
03:39:54 STEP 320 H= 320:00 +CPU=133.855
03:43:09 STEP 321 H= 321:00 +CPU=192.893
03:45:22 STEP 322 H= 322:00 +CPU=132.045
03:47:32 STEP 323 H= 323:00 +CPU=129.197
03:51:04 STEP 324 H= 324:00 +CPU=210.783
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMGGhq0f+000324
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMSHhq0f+000324
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMUAhq0f+000324
03:53:08 STEP 325 H= 325:00 +CPU=122.995
03:55:21 STEP 326 H= 326:00 +CPU=131.958
03:58:29 STEP 327 H= 327:00 +CPU=186.752
04:00:30 STEP 328 H= 328:00 +CPU=119.794
04:02:44 STEP 329 H= 329:00 +CPU=133.320
04:06:05 STEP 330 H= 330:00 +CPU=199.455
04:08:22 STEP 331 H= 331:00 +CPU=135.301
04:10:32 STEP 332 H= 332:00 +CPU=129.197
04:13:44 STEP 333 H= 333:00 +CPU=190.707
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 1984050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0555
(argv4) batchid: 953
(argv5) wuid: 12170199
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/12
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12170199
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/12/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/12/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_bd1fdd38845ecd99cd334f92d97fd530 to: /var/lib/boinc-client/slots/12/oifs_43r3_ps_0555_1984050100_123_953_12170199.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/12/oifs_43r3_ps_0555_1984050100_123_953_12170199.zip
ic_ancil_file: ic_ancil_12170199
ifsdata_file: ifsdata_12170199
climate_data_file: clim_data_12170199
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_67961e968080678a3d119c18e75f553d to: /var/lib/boinc-client/slots/12/ic_ancil_12170199.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/12/ic_ancil_12170199.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_f30ea0e48dfaead6ea733c4b4d776e78 to: /var/lib/boinc-client/slots/12/ifsdata/ifsdata_12170199.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/12/ifsdata/ifsdata_12170199.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/12/159l_2/clim_data_12170199.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/12/159l_2/clim_data_12170199.zip
Checking for progress XML file: /var/lib/boinc-client/slots/12/progress_file_12170199.xml
Opened progress file ok : /var/lib/boinc-client/slots/12/progress_file_12170199.xml
-- Model is restarting --
Adjusting last_iter, 334, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/12/progress_file_12170199.xml
last_cpu_time: 71555
upload_file_number: 13
last_iter: 311
last_upload: 1123200
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0555_1984050100_123_953_12170199_1_r600327352
The child process has been launched with process id: 156837
Executing the command: /var/lib/boinc-client/slots/12/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:test-System-Product-Name:1:1:156837:156837] [20230122:041635:1674382595:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:test-System-Product-Name:1:1:156837:156837] [20230122:041635:1674382595:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:test-System-Product-Name:1:1:156837:156837] [20230122:041635:1674382595:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:test-System-Product-Name:1:1:156837:156837] [20230122:041635:1674382595:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:test-System-Product-Name:1:1:156837:156837] [20230122:041635:1674382595:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:test-System-Product-Name:1:1:156837:156837] [20230122:041635:1674382595:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647)
[EC_DRHOOK:test-System-Product-Name:1:1:156837:156837] [20230122:041635:1674382595:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:test-System-Product-Name:1:1:156837:156837] [20230122:041635:1674382595:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:test-System-Product-Name:1:1:156837:156837] [20230122:041635:1674382595:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:test-System-Product-Name:1:1:156837:156837] [20230122:041635:1674382595:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:test-System-Product-Name:1:1:156837:156837] [20230122:041635:1674382595:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:test-System-Product-Name:1:1:156837:156837] [20230122:041635:1674382595:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:test-System-Product-Name:1:1:156837:156837] [20230122:041635:1674382595:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:test-System-Product-Name:1:1:156837:156837] [20230122:041635:1674382595:0.001] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:test-System-Product-Name:1:1:156837:156837] [20230122:041635:1674382595:0.001] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:test-System-Product-Name:1:1:156837:156837] [20230122:041635:1674382595:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:test-System-Product-Name:1:1:156837:156837] [20230122:041635:1674382595:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:test-System-Product-Name:1:1:156837:156837] [20230122:041635:1674382595:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:test-System-Product-Name:1:1:156837:156837] [20230122:041635:1674382595:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:test-System-Product-Name:1:1:156837:156837] [20230122:041635:1674382595:0.001] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:test-System-Product-Name:1:1:156837:156837] [20230122:041635:1674382595:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:test-System-Product-Name:1:1:156837:156837] [20230122:041635:1674382595:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:test-System-Product-Name:1:1:156837:156837] [20230122:041635:1674382595:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:test-System-Product-Name:1:1:156837:156837] [20230122:041635:1674382595:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:test-System-Product-Name:1:1:156837:156837] [20230122:041635:1674382595:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:test-System-Product-Name:1:1:156837:156837] [20230122:041635:1674382595:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:test-System-Product-Name:1:1:156837:156837] [20230122:041635:1674382595:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:test-System-Product-Name:1:1:156837:156837] [20230122:041635:1674382595:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:test-System-Product-Name:1:1:156837:156837] [20230122:041635:1674382595:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:test-System-Product-Name:1:1:156837:156837] [20230122:041635:1674382595:0.001] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
04:21:03 STEP 312 H= 312:00 +CPU=244.750
04:23:25 STEP 313 H= 313:00 +CPU=140.481
04:25:49 STEP 314 H= 314:00 +CPU=143.298
04:29:05 STEP 315 H= 315:00 +CPU=194.020
04:31:12 STEP 316 H= 316:00 +CPU=126.591
04:33:28 STEP 317 H= 317:00 +CPU=135.193
04:36:40 STEP 318 H= 318:00 +CPU=190.729
04:38:57 STEP 319 H= 319:00 +CPU=135.833
04:41:19 STEP 320 H= 320:00 +CPU=140.297
04:44:37 STEP 321 H= 321:00 +CPU=197.114
04:46:55 STEP 322 H= 322:00 +CPU=136.425
04:49:01 STEP 323 H= 323:00 +CPU=125.168
04:52:34 STEP 324 H= 324:00 +CPU=211.622
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMGGhq0f+000324
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMSHhq0f+000324
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMUAhq0f+000324
04:54:59 STEP 325 H= 325:00 +CPU=144.159
04:56:59 STEP 326 H= 326:00 +CPU=118.253
05:00:06 STEP 327 H= 327:00 +CPU=186.231
05:02:20 STEP 328 H= 328:00 +CPU=132.688
05:04:17 STEP 329 H= 329:00 +CPU=116.092
05:07:28 STEP 330 H= 330:00 +CPU=189.591
05:09:34 STEP 331 H= 331:00 +CPU=125.497
05:11:48 STEP 332 H= 332:00 +CPU=132.620
05:14:45 STEP 333 H= 333:00 +CPU=175.609
05:17:00 STEP 334 H= 334:00 +CPU=134.383
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 1984050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0555
(argv4) batchid: 953
(argv5) wuid: 12170199
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/12
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12170199
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/12/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/12/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_bd1fdd38845ecd99cd334f92d97fd530 to: /var/lib/boinc-client/slots/12/oifs_43r3_ps_0555_1984050100_123_953_12170199.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/12/oifs_43r3_ps_0555_1984050100_123_953_12170199.zip
ic_ancil_file: ic_ancil_12170199
ifsdata_file: ifsdata_12170199
climate_data_file: clim_data_12170199
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_67961e968080678a3d119c18e75f553d to: /var/lib/boinc-client/slots/12/ic_ancil_12170199.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/12/ic_ancil_12170199.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_f30ea0e48dfaead6ea733c4b4d776e78 to: /var/lib/boinc-client/slots/12/ifsdata/ifsdata_12170199.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/12/ifsdata/ifsdata_12170199.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/12/159l_2/clim_data_12170199.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/12/159l_2/clim_data_12170199.zip
Checking for progress XML file: /var/lib/boinc-client/slots/12/progress_file_12170199.xml
Opened progress file ok : /var/lib/boinc-client/slots/12/progress_file_12170199.xml
-- Model is restarting --
Adjusting last_iter, 335, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/12/progress_file_12170199.xml
last_cpu_time: 75181
upload_file_number: 13
last_iter: 311
last_upload: 1123200
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0555_1984050100_123_953_12170199_1_r600327352
The child process has been launched with process id: 157117
Executing the command: /var/lib/boinc-client/slots/12/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:test-System-Product-Name:1:1:157117:157117] [20230122:051855:1674386335:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:test-System-Product-Name:1:1:157117:157117] [20230122:051855:1674386335:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:test-System-Product-Name:1:1:157117:157117] [20230122:051855:1674386335:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:test-System-Product-Name:1:1:157117:157117] [20230122:051855:1674386335:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:test-System-Product-Name:1:1:157117:157117] [20230122:051855:1674386335:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:test-System-Product-Name:1:1:157117:157117] [20230122:051855:1674386335:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647)
[EC_DRHOOK:test-System-Product-Name:1:1:157117:157117] [20230122:051855:1674386335:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:test-System-Product-Name:1:1:157117:157117] [20230122:051855:1674386335:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:test-System-Product-Name:1:1:157117:157117] [20230122:051855:1674386335:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:test-System-Product-Name:1:1:157117:157117] [20230122:051855:1674386335:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:test-System-Product-Name:1:1:157117:157117] [20230122:051855:1674386335:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:test-System-Product-Name:1:1:157117:157117] [20230122:051855:1674386335:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:test-System-Product-Name:1:1:157117:157117] [20230122:051855:1674386335:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:test-System-Product-Name:1:1:157117:157117] [20230122:051855:1674386335:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:test-System-Product-Name:1:1:157117:157117] [20230122:051855:1674386335:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:test-System-Product-Name:1:1:157117:157117] [20230122:051855:1674386335:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:test-System-Product-Name:1:1:157117:157117] [20230122:051855:1674386335:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:test-System-Product-Name:1:1:157117:157117] [20230122:051855:1674386335:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:test-System-Product-Name:1:1:157117:157117] [20230122:051855:1674386335:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:test-System-Product-Name:1:1:157117:157117] [20230122:051855:1674386335:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:test-System-Product-Name:1:1:157117:157117] [20230122:051855:1674386335:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:test-System-Product-Name:1:1:157117:157117] [20230122:051855:1674386335:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:test-System-Product-Name:1:1:157117:157117] [20230122:051855:1674386335:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:test-System-Product-Name:1:1:157117:157117] [20230122:051855:1674386335:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:test-System-Product-Name:1:1:157117:157117] [20230122:051855:1674386335:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:test-System-Product-Name:1:1:157117:157117] [20230122:051855:1674386335:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:test-System-Product-Name:1:1:157117:157117] [20230122:051855:1674386335:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:test-System-Product-Name:1:1:157117:157117] [20230122:051855:1674386335:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:test-System-Product-Name:1:1:157117:157117] [20230122:051855:1674386335:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:test-System-Product-Name:1:1:157117:157117] [20230122:051855:1674386335:0.001] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
05:22:53 STEP 312 H= 312:00 +CPU=214.977
05:25:08 STEP 313 H= 313:00 +CPU=134.458
05:27:27 STEP 314 H= 314:00 +CPU=137.293
05:30:22 STEP 315 H= 315:00 +CPU=173.582
05:32:34 STEP 316 H= 316:00 +CPU=131.146
05:34:37 STEP 317 H= 317:00 +CPU=122.272
05:37:38 STEP 318 H= 318:00 +CPU=180.119
05:39:49 STEP 319 H= 319:00 +CPU=130.036
05:42:11 STEP 320 H= 320:00 +CPU=140.665
05:45:27 STEP 321 H= 321:00 +CPU=193.932
05:47:18 STEP 322 H= 322:00 +CPU=110.609
05:49:20 STEP 323 H= 323:00 +CPU=120.730
05:52:16 STEP 324 H= 324:00 +CPU=174.735
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMGGhq0f+000324
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMSHhq0f+000324
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMUAhq0f+000324
05:53:52 STEP 325 H= 325:00 +CPU= 95.901
05:55:38 STEP 326 H= 326:00 +CPU=104.792
05:58:17 STEP 327 H= 327:00 +CPU=157.674
06:00:11 STEP 328 H= 328:00 +CPU=112.865
06:01:54 STEP 329 H= 329:00 +CPU=102.423
06:04:39 STEP 330 H= 330:00 +CPU=164.162
06:06:30 STEP 331 H= 331:00 +CPU=110.003
06:08:26 STEP 332 H= 332:00 +CPU=115.473
06:11:05 STEP 333 H= 333:00 +CPU=156.956
06:13:09 STEP 334 H= 334:00 +CPU=123.298
06:15:12 STEP 335 H= 335:00 +CPU=122.595
06:18:39 STEP 336 H= 336:00 +CPU=205.082
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMGGhq0f+000336
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMSHhq0f+000336
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMUAhq0f+000336
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12170199/ICMGGhq0f+000312
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12170199/ICMSHhq0f+000312
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12170199/ICMUAhq0f+000312
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12170199/ICMGGhq0f+000324
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12170199/ICMSHhq0f+000324
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12170199/ICMUAhq0f+000324
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0555_1984050100_123_953_12170199_1_r600327352_13.zip
Uploading the intermediate file: upload_file_13.zip
06:20:50 STEP 337 H= 337:00 +CPU=130.431
06:22:54 STEP 338 H= 338:00 +CPU=123.068
06:25:32 STEP 339 H= 339:00 +CPU=157.175
06:27:43 STEP 340 H= 340:00 +CPU=130.186
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 1984050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0555
(argv4) batchid: 953
(argv5) wuid: 12170199
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/12
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12170199
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/12/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/12/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_bd1fdd38845ecd99cd334f92d97fd530 to: /var/lib/boinc-client/slots/12/oifs_43r3_ps_0555_1984050100_123_953_12170199.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/12/oifs_43r3_ps_0555_1984050100_123_953_12170199.zip
ic_ancil_file: ic_ancil_12170199
ifsdata_file: ifsdata_12170199
climate_data_file: clim_data_12170199
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_67961e968080678a3d119c18e75f553d to: /var/lib/boinc-client/slots/12/ic_ancil_12170199.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/12/ic_ancil_12170199.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_f30ea0e48dfaead6ea733c4b4d776e78 to: /var/lib/boinc-client/slots/12/ifsdata/ifsdata_12170199.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/12/ifsdata/ifsdata_12170199.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/12/159l_2/clim_data_12170199.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/12/159l_2/clim_data_12170199.zip
Checking for progress XML file: /var/lib/boinc-client/slots/12/progress_file_12170199.xml
Opened progress file ok : /var/lib/boinc-client/slots/12/progress_file_12170199.xml
-- Model is restarting --
Adjusting last_iter, 341, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/12/progress_file_12170199.xml
last_cpu_time: 79337
upload_file_number: 14
last_iter: 335
last_upload: 1209600
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0555_1984050100_123_953_12170199_1_r600327352
The child process has been launched with process id: 157477
Executing the command: /var/lib/boinc-client/slots/12/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:test-System-Product-Name:1:1:157477:157477] [20230122:063008:1674390608:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:test-System-Product-Name:1:1:157477:157477] [20230122:063008:1674390608:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:test-System-Product-Name:1:1:157477:157477] [20230122:063008:1674390608:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:test-System-Product-Name:1:1:157477:157477] [20230122:063008:1674390608:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:test-System-Product-Name:1:1:157477:157477] [20230122:063008:1674390608:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:test-System-Product-Name:1:1:157477:157477] [20230122:063008:1674390608:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647)
[EC_DRHOOK:test-System-Product-Name:1:1:157477:157477] [20230122:063008:1674390608:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:test-System-Product-Name:1:1:157477:157477] [20230122:063008:1674390608:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:test-System-Product-Name:1:1:157477:157477] [20230122:063008:1674390608:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:test-System-Product-Name:1:1:157477:157477] [20230122:063008:1674390608:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:test-System-Product-Name:1:1:157477:157477] [20230122:063008:1674390608:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:test-System-Product-Name:1:1:157477:157477] [20230122:063008:1674390608:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:test-System-Product-Name:1:1:157477:157477] [20230122:063008:1674390608:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:test-System-Product-Name:1:1:157477:157477] [20230122:063008:1674390608:0.001] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:test-System-Product-Name:1:1:157477:157477] [20230122:063008:1674390608:0.001] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:test-System-Product-Name:1:1:157477:157477] [20230122:063008:1674390608:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:test-System-Product-Name:1:1:157477:157477] [20230122:063008:1674390608:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:test-System-Product-Name:1:1:157477:157477] [20230122:063008:1674390608:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:test-System-Product-Name:1:1:157477:157477] [20230122:063008:1674390608:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:test-System-Product-Name:1:1:157477:157477] [20230122:063008:1674390608:0.001] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:test-System-Product-Name:1:1:157477:157477] [20230122:063008:1674390608:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:test-System-Product-Name:1:1:157477:157477] [20230122:063008:1674390608:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:test-System-Product-Name:1:1:157477:157477] [20230122:063008:1674390608:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:test-System-Product-Name:1:1:157477:157477] [20230122:063008:1674390608:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:test-System-Product-Name:1:1:157477:157477] [20230122:063008:1674390608:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:test-System-Product-Name:1:1:157477:157477] [20230122:063008:1674390608:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:test-System-Product-Name:1:1:157477:157477] [20230122:063008:1674390608:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:test-System-Product-Name:1:1:157477:157477] [20230122:063008:1674390608:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:test-System-Product-Name:1:1:157477:157477] [20230122:063008:1674390608:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:test-System-Product-Name:1:1:157477:157477] [20230122:063008:1674390608:0.001] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
06:33:39 STEP 336 H= 336:00 +CPU=190.919
06:35:47 STEP 337 H= 337:00 +CPU=126.731
06:37:52 STEP 338 H= 338:00 +CPU=124.067
06:40:47 STEP 339 H= 339:00 +CPU=173.738
06:42:52 STEP 340 H= 340:00 +CPU=124.461
06:44:42 STEP 341 H= 341:00 +CPU=108.597
06:47:34 STEP 342 H= 342:00 +CPU=171.154
06:49:37 STEP 343 H= 343:00 +CPU=121.835
06:52:06 STEP 344 H= 344:00 +CPU=147.477
06:55:15 STEP 345 H= 345:00 +CPU=187.886
06:57:17 STEP 346 H= 346:00 +CPU=120.885
06:59:22 STEP 347 H= 347:00 +CPU=124.484
07:02:35 STEP 348 H= 348:00 +CPU=191.542
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMGGhq0f+000348
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMSHhq0f+000348
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMUAhq0f+000348
07:04:42 STEP 349 H= 349:00 +CPU=125.735
07:06:44 STEP 350 H= 350:00 +CPU=121.301
07:09:37 STEP 351 H= 351:00 +CPU=171.893
07:11:46 STEP 352 H= 352:00 +CPU=128.041
07:13:39 STEP 353 H= 353:00 +CPU=111.748
07:16:41 STEP 354 H= 354:00 +CPU=180.714
07:18:36 STEP 355 H= 355:00 +CPU=113.860
07:20:28 STEP 356 H= 356:00 +CPU=111.918
07:23:03 STEP 357 H= 357:00 +CPU=153.202
07:24:57 STEP 358 H= 358:00 +CPU=113.351
07:26:50 STEP 359 H= 359:00 +CPU=112.141
Abort request received from BOINC client, ending the child process
..Failed, model did not complete successfully
</stderr_txt>
]]>
