<core_client_version>7.16.6</core_client_version>
<![CDATA[
<message>
Disk usage limit exceeded</message>
<stderr_txt>
20:31:30 STEP 441 H= 441:00 +CPU= 34.031
20:31:56 STEP 442 H= 442:00 +CPU= 26.108
20:32:20 STEP 443 H= 443:00 +CPU= 24.515
20:32:59 STEP 444 H= 444:00 +CPU= 38.523
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000444
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000444
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000444
20:33:27 STEP 445 H= 445:00 +CPU= 27.938
20:33:51 STEP 446 H= 446:00 +CPU= 24.412
20:34:26 STEP 447 H= 447:00 +CPU= 34.046
20:34:53 STEP 448 H= 448:00 +CPU= 27.606
20:35:16 STEP 449 H= 449:00 +CPU= 22.546
20:35:51 STEP 450 H= 450:00 +CPU= 34.887
20:36:19 STEP 451 H= 451:00 +CPU= 28.050
20:36:43 STEP 452 H= 452:00 +CPU= 24.474
20:37:17 STEP 453 H= 453:00 +CPU= 33.949
20:37:44 STEP 454 H= 454:00 +CPU= 26.619
20:38:08 STEP 455 H= 455:00 +CPU= 24.064
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201498/ICMGGhq0f+000432
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201498/ICMSHhq0f+000432
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201498/ICMUAhq0f+000432
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201498/ICMGGhq0f+000444
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201498/ICMSHhq0f+000444
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201498/ICMUAhq0f+000444
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0854_2015050100_123_984_12201498_0_r328028627_17.zip
Uploading the intermediate file: upload_file_17.zip
20:38:49 STEP 456 H= 456:00 +CPU= 41.006
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000456
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000456
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000456
20:39:16 STEP 457 H= 457:00 +CPU= 27.003
20:39:39 STEP 458 H= 458:00 +CPU= 22.524
20:40:14 STEP 459 H= 459:00 +CPU= 35.177
20:40:40 STEP 460 H= 460:00 +CPU= 26.147
20:41:02 STEP 461 H= 461:00 +CPU= 22.002
20:41:40 STEP 462 H= 462:00 +CPU= 37.207
20:42:05 STEP 463 H= 463:00 +CPU= 24.271
20:42:27 STEP 464 H= 464:00 +CPU= 22.626
20:43:05 STEP 465 H= 465:00 +CPU= 37.894
20:43:29 STEP 466 H= 466:00 +CPU= 23.934
20:43:51 STEP 467 H= 467:00 +CPU= 22.093
20:44:34 STEP 468 H= 468:00 +CPU= 43.025
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000468
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000468
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000468
20:44:56 STEP 469 H= 469:00 +CPU= 21.899
20:45:19 STEP 470 H= 470:00 +CPU= 22.475
20:45:57 STEP 471 H= 471:00 +CPU= 37.959
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
20:47:01 STEP 472 H= 472:00 +CPU= 30.189
20:47:31 STEP 473 H= 473:00 +CPU= 27.852
[EC_DRHOOK:ruly-Inspiron-5558:1:1:8425:8425] [20230118:204807:1674103687:1989.219] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1538] Received signal#15 (SIGTERM) :: 2591MB (heap), 4419MB (maxrss), 0MB (maxstack), 0 (paging), nsigs = 1
[EC_DRHOOK:ruly-Inspiron-5558:1:1:8425:8425] [20230118:204807:1674103687:1989.219] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1542] Also activating Harakiri-alarm (SIGALRM=14) to expire after 500s elapsed to prevent hangs, nsigs = 1
[EC_DRHOOK:ruly-Inspiron-5558:1:1:8425:8425] [20230118:204807:1674103687:1989.219] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1544] Harakiri signal handler 'signal_harakiri' for signal#14 (SIGALRM) installed at 0x81f3c0 (old at (nil))
[EC_DRHOOK:ruly-Inspiron-5558:1:1:8425:8425] [20230118:204807:1674103687:1989.219] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1617] Signal#15 was caused by unrecognized si_code [memaddr=0x1], nsigs = 1
[EC_DRHOOK:ruly-Inspiron-5558:1:1:8425:8425] [20230118:204807:1674103687:1989.219] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1686] Starting DrHook backtrace for signal#15, nsigs = 1
[EC_DRHOOK:ruly-Inspiron-5558:1:1:8425:8425] [20230118:204807:1674103687:1989.219] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3843] 2598 MB (maxheap), 4419 MB (maxrss), 0 MB (maxstack)
[EC_DRHOOK:ruly-Inspiron-5558:1:1:8425:8425] [20230118:204807:1674103687:1989.219] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : MASTER
[EC_DRHOOK:ruly-Inspiron-5558:1:1:8425:8425] [20230118:204807:1674103687:1989.219] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT0
[EC_DRHOOK:ruly-Inspiron-5558:1:1:8425:8425] [20230118:204807:1674103687:1989.219] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT1
[EC_DRHOOK:ruly-Inspiron-5558:1:1:8425:8425] [20230118:204807:1674103687:1989.219] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT2
[EC_DRHOOK:ruly-Inspiron-5558:1:1:8425:8425] [20230118:204807:1674103687:1989.219] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT3
[EC_DRHOOK:ruly-Inspiron-5558:1:1:8425:8425] [20230118:204807:1674103687:1989.219] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT4
[EC_DRHOOK:ruly-Inspiron-5558:1:1:8425:8425] [20230118:204807:1674103687:1989.219] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : WVCOUPLE
[EC_DRHOOK:ruly-Inspiron-5558:1:1:8425:8425] [20230118:204807:1674103687:1989.219] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : WAVEMDL
[EC_DRHOOK:ruly-Inspiron-5558:1:1:8425:8425] [20230118:204807:1674103687:1989.219] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : WAMODEL
[EC_DRHOOK:ruly-Inspiron-5558:1:1:8425:8425] [20230118:204807:1674103687:1989.219] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : PROPAG_WAM
[EC_DRHOOK:ruly-Inspiron-5558:1:1:8425:8425] [20230118:204807:1674103687:1989.219] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : >OMP-WAMODEL 1 (1430)
[EC_DRHOOK:ruly-Inspiron-5558:1:1:8425:8425] [20230118:204807:1674103687:1989.219] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : PROPAGS2
[EC_DRHOOK:ruly-Inspiron-5558:1:1:8425:8425] [20230118:204808:1674103688:1990.219] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1734] DrHook backtrace done for signal#15, nsigs = 1
[EC_DRHOOK:ruly-Inspiron-5558:1:1:8425:8425] [20230118:204808:1674103688:1990.219] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1785] Calling previous signal handler at 0x1ce8bb0 for signal#15, nsigs = 1
forrtl: error (78): process killed (SIGTERM)
Image PC Routine Line Source
oifs_43r3_model.e 0000000001CE916B Unknown Unknown Unknown
oifs_43r3_model.e 000000000081FFF1 Unknown Unknown Unknown
oifs_43r3_model.e 0000000001DC9090 Unknown Unknown Unknown
oifs_43r3_model.e 0000000001A26BB7 propags2_ 132 propags2.F
oifs_43r3_model.e 000000000169F16E propag_wam_ 86 propag_wam.F
oifs_43r3_model.e 000000000168BBC6 wamodel_ 942 wamodel.F
oifs_43r3_model.e 0000000000872295 wavemdl_ 584 wavemdl.F
oifs_43r3_model.e 00000000007E0165 wvcouple_ 260 wvcouple.F90
oifs_43r3_model.e 000000000055E4AD cnt4_ 1283 cnt4.F90
oifs_43r3_model.e 00000000005471C9 cnt3_ 267 cnt3.F90
oifs_43r3_model.e 0000000000546412 cnt2_ 88 cnt2.F90
oifs_43r3_model.e 0000000000545E28 cnt1_ 92 cnt1.F90
oifs_43r3_model.e 000000000040708D cnt0_ 146 cnt0.F90
oifs_43r3_model.e 000000000040220F MAIN__ 96 master.F90
oifs_43r3_model.e 00000000004021A2 Unknown Unknown Unknown
oifs_43r3_model.e 0000000001DCA390 Unknown Unknown Unknown
oifs_43r3_model.e 000000000040206E Unknown Unknown Unknown
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 2015050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0854
(argv4) batchid: 984
(argv5) wuid: 12201498
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/0
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201498
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/0/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/0/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_8e9864471c4355a33866531b9302fcee to: /var/lib/boinc-client/slots/0/oifs_43r3_ps_0854_2015050100_123_984_12201498.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/0/oifs_43r3_ps_0854_2015050100_123_984_12201498.zip
ic_ancil_file: ic_ancil_12201498
ifsdata_file: ifsdata_12201498
climate_data_file: clim_data_12201498
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_b282c1bc7cd4c4972e3aca84d2006d0e to: /var/lib/boinc-client/slots/0/ic_ancil_12201498.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/0/ic_ancil_12201498.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_5c03420e2fdf00e5e19c769d3fcf71d9 to: /var/lib/boinc-client/slots/0/ifsdata/ifsdata_12201498.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/0/ifsdata/ifsdata_12201498.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/0/159l_2/clim_data_12201498.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/0/159l_2/clim_data_12201498.zip
Checking for progress XML file: /var/lib/boinc-client/slots/0/progress_file_12201498.xml
Opened progress file ok : /var/lib/boinc-client/slots/0/progress_file_12201498.xml
-- Model is restarting --
Adjusting last_iter, 474, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/0/progress_file_12201498.xml
last_cpu_time: 13794
upload_file_number: 18
last_iter: 455
last_upload: 1638000
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0854_2015050100_123_984_12201498_0_r328028627
The child process has been launched with process id: 2626
Executing the command: /var/lib/boinc-client/slots/0/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:ruly-Inspiron-5558:1:1:2626:2626] [20230120:155831:1674259111:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:ruly-Inspiron-5558:1:1:2626:2626] [20230120:155831:1674259111:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:ruly-Inspiron-5558:1:1:2626:2626] [20230120:155831:1674259111:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:ruly-Inspiron-5558:1:1:2626:2626] [20230120:155831:1674259111:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:ruly-Inspiron-5558:1:1:2626:2626] [20230120:155831:1674259111:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:ruly-Inspiron-5558:1:1:2626:2626] [20230120:155831:1674259111:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647)
[EC_DRHOOK:ruly-Inspiron-5558:1:1:2626:2626] [20230120:155831:1674259111:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:ruly-Inspiron-5558:1:1:2626:2626] [20230120:155831:1674259111:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:ruly-Inspiron-5558:1:1:2626:2626] [20230120:155831:1674259111:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:ruly-Inspiron-5558:1:1:2626:2626] [20230120:155831:1674259111:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:ruly-Inspiron-5558:1:1:2626:2626] [20230120:155831:1674259111:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:ruly-Inspiron-5558:1:1:2626:2626] [20230120:155831:1674259111:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:ruly-Inspiron-5558:1:1:2626:2626] [20230120:155831:1674259111:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:ruly-Inspiron-5558:1:1:2626:2626] [20230120:155831:1674259111:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:ruly-Inspiron-5558:1:1:2626:2626] [20230120:155831:1674259111:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:ruly-Inspiron-5558:1:1:2626:2626] [20230120:155831:1674259111:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ruly-Inspiron-5558:1:1:2626:2626] [20230120:155831:1674259111:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:ruly-Inspiron-5558:1:1:2626:2626] [20230120:155831:1674259111:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ruly-Inspiron-5558:1:1:2626:2626] [20230120:155831:1674259111:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:ruly-Inspiron-5558:1:1:2626:2626] [20230120:155831:1674259111:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:ruly-Inspiron-5558:1:1:2626:2626] [20230120:155831:1674259111:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ruly-Inspiron-5558:1:1:2626:2626] [20230120:155831:1674259111:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ruly-Inspiron-5558:1:1:2626:2626] [20230120:155831:1674259111:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:ruly-Inspiron-5558:1:1:2626:2626] [20230120:155831:1674259111:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ruly-Inspiron-5558:1:1:2626:2626] [20230120:155831:1674259111:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ruly-Inspiron-5558:1:1:2626:2626] [20230120:155831:1674259111:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ruly-Inspiron-5558:1:1:2626:2626] [20230120:155831:1674259111:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:ruly-Inspiron-5558:1:1:2626:2626] [20230120:155831:1674259111:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:ruly-Inspiron-5558:1:1:2626:2626] [20230120:155831:1674259111:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:ruly-Inspiron-5558:1:1:2626:2626] [20230120:155831:1674259111:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
16:04:49 STEP 456 H= 456:00 +CPU= 65.378
16:05:12 STEP 457 H= 457:00 +CPU= 22.344
16:05:35 STEP 458 H= 458:00 +CPU= 23.012
16:06:16 STEP 459 H= 459:00 +CPU= 39.951
16:06:39 STEP 460 H= 460:00 +CPU= 23.123
16:07:05 STEP 461 H= 461:00 +CPU= 24.605
16:07:48 STEP 462 H= 462:00 +CPU= 41.289
16:08:13 STEP 463 H= 463:00 +CPU= 24.451
16:08:43 STEP 464 H= 464:00 +CPU= 29.269
16:09:20 STEP 465 H= 465:00 +CPU= 37.284
16:09:44 STEP 466 H= 466:00 +CPU= 23.164
16:10:13 STEP 467 H= 467:00 +CPU= 28.236
16:10:53 STEP 468 H= 468:00 +CPU= 40.465
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000468
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000468
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000468
16:11:16 STEP 469 H= 469:00 +CPU= 22.118
16:11:46 STEP 470 H= 470:00 +CPU= 29.339
16:12:23 STEP 471 H= 471:00 +CPU= 36.414
16:12:48 STEP 472 H= 472:00 +CPU= 24.009
16:13:16 STEP 473 H= 473:00 +CPU= 27.761
16:13:54 STEP 474 H= 474:00 +CPU= 37.497
16:14:16 STEP 475 H= 475:00 +CPU= 22.064
16:14:43 STEP 476 H= 476:00 +CPU= 27.344
16:15:21 STEP 477 H= 477:00 +CPU= 37.225
16:15:43 STEP 478 H= 478:00 +CPU= 22.730
16:16:11 STEP 479 H= 479:00 +CPU= 27.330
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201498/ICMGGhq0f+000456
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201498/ICMSHhq0f+000456
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201498/ICMUAhq0f+000456
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201498/ICMGGhq0f+000468
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201498/ICMSHhq0f+000468
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201498/ICMUAhq0f+000468
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0854_2015050100_123_984_12201498_0_r328028627_18.zip
Uploading the intermediate file: upload_file_18.zip
16:16:56 STEP 480 H= 480:00 +CPU= 44.399
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000480
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000480
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000480
16:17:20 STEP 481 H= 481:00 +CPU= 23.002
16:17:49 STEP 482 H= 482:00 +CPU= 28.716
16:18:25 STEP 483 H= 483:00 +CPU= 36.034
16:18:45 STEP 484 H= 484:00 +CPU= 20.022
16:19:05 STEP 485 H= 485:00 +CPU= 20.163
16:19:40 STEP 486 H= 486:00 +CPU= 35.071
16:20:03 STEP 487 H= 487:00 +CPU= 22.494
16:20:26 STEP 488 H= 488:00 +CPU= 22.992
16:21:01 STEP 489 H= 489:00 +CPU= 34.487
16:21:24 STEP 490 H= 490:00 +CPU= 22.891
16:21:46 STEP 491 H= 491:00 +CPU= 22.280
16:22:25 STEP 492 H= 492:00 +CPU= 38.346
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000492
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000492
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000492
16:22:47 STEP 493 H= 493:00 +CPU= 22.026
16:23:10 STEP 494 H= 494:00 +CPU= 22.729
16:23:44 STEP 495 H= 495:00 +CPU= 34.147
16:24:06 STEP 496 H= 496:00 +CPU= 22.480
16:24:29 STEP 497 H= 497:00 +CPU= 22.076
16:25:04 STEP 498 H= 498:00 +CPU= 35.219
16:25:26 STEP 499 H= 499:00 +CPU= 22.158
16:25:49 STEP 500 H= 500:00 +CPU= 23.056
16:26:24 STEP 501 H= 501:00 +CPU= 34.379
16:26:47 STEP 502 H= 502:00 +CPU= 23.200
16:27:11 STEP 503 H= 503:00 +CPU= 23.147
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201498/ICMGGhq0f+000480
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201498/ICMSHhq0f+000480
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201498/ICMUAhq0f+000480
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201498/ICMGGhq0f+000492
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201498/ICMSHhq0f+000492
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201498/ICMUAhq0f+000492
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0854_2015050100_123_984_12201498_0_r328028627_19.zip
Uploading the intermediate file: upload_file_19.zip
16:27:52 STEP 504 H= 504:00 +CPU= 41.378
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000504
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000504
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000504
16:28:14 STEP 505 H= 505:00 +CPU= 22.114
16:28:38 STEP 506 H= 506:00 +CPU= 23.063
16:29:12 STEP 507 H= 507:00 +CPU= 34.106
16:29:35 STEP 508 H= 508:00 +CPU= 22.819
16:29:57 STEP 509 H= 509:00 +CPU= 22.058
16:30:32 STEP 510 H= 510:00 +CPU= 35.502
16:30:58 STEP 511 H= 511:00 +CPU= 25.413
16:31:21 STEP 512 H= 512:00 +CPU= 23.104
16:31:56 STEP 513 H= 513:00 +CPU= 34.430
16:32:19 STEP 514 H= 514:00 +CPU= 22.823
16:32:41 STEP 515 H= 515:00 +CPU= 22.194
16:33:20 STEP 516 H= 516:00 +CPU= 39.087
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000516
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000516
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000516
16:33:43 STEP 517 H= 517:00 +CPU= 22.157
16:34:06 STEP 518 H= 518:00 +CPU= 22.823
16:34:40 STEP 519 H= 519:00 +CPU= 34.360
16:35:03 STEP 520 H= 520:00 +CPU= 22.885
16:35:25 STEP 521 H= 521:00 +CPU= 22.181
16:36:01 STEP 522 H= 522:00 +CPU= 35.341
16:36:23 STEP 523 H= 523:00 +CPU= 22.085
16:36:46 STEP 524 H= 524:00 +CPU= 22.855
16:37:20 STEP 525 H= 525:00 +CPU= 34.203
16:37:43 STEP 526 H= 526:00 +CPU= 22.891
16:38:06 STEP 527 H= 527:00 +CPU= 22.870
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201498/ICMGGhq0f+000504
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201498/ICMSHhq0f+000504
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201498/ICMUAhq0f+000504
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201498/ICMGGhq0f+000516
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201498/ICMSHhq0f+000516
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201498/ICMUAhq0f+000516
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0854_2015050100_123_984_12201498_0_r328028627_20.zip
Uploading the intermediate file: upload_file_20.zip
16:38:47 STEP 528 H= 528:00 +CPU= 40.828
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000528
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000528
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000528
16:39:09 STEP 529 H= 529:00 +CPU= 22.127
16:39:33 STEP 530 H= 530:00 +CPU= 23.091
16:40:07 STEP 531 H= 531:00 +CPU= 34.356
16:40:30 STEP 532 H= 532:00 +CPU= 22.663
16:40:52 STEP 533 H= 533:00 +CPU= 22.187
16:41:27 STEP 534 H= 534:00 +CPU= 34.867
16:41:50 STEP 535 H= 535:00 +CPU= 22.403
16:42:13 STEP 536 H= 536:00 +CPU= 22.868
16:42:47 STEP 537 H= 537:00 +CPU= 34.195
16:43:10 STEP 538 H= 538:00 +CPU= 22.810
16:43:32 STEP 539 H= 539:00 +CPU= 22.595
16:44:11 STEP 540 H= 540:00 +CPU= 38.092
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000540
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000540
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000540
16:44:33 STEP 541 H= 541:00 +CPU= 22.066
16:44:56 STEP 542 H= 542:00 +CPU= 22.852
16:45:30 STEP 543 H= 543:00 +CPU= 34.054
16:45:53 STEP 544 H= 544:00 +CPU= 23.077
16:46:15 STEP 545 H= 545:00 +CPU= 22.206
16:46:50 STEP 546 H= 546:00 +CPU= 34.823
16:47:12 STEP 547 H= 547:00 +CPU= 22.103
16:47:35 STEP 548 H= 548:00 +CPU= 22.733
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
16:49:20 STEP 549 H= 549:00 +CPU= 40.683
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
16:50:11 STEP 550 H= 550:00 +CPU= 26.753
16:50:47 STEP 551 H= 551:00 +CPU= 33.286
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201498/ICMGGhq0f+000528
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201498/ICMSHhq0f+000528
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201498/ICMUAhq0f+000528
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201498/ICMGGhq0f+000540
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201498/ICMSHhq0f+000540
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201498/ICMUAhq0f+000540
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0854_2015050100_123_984_12201498_0_r328028627_21.zip
Uploading the intermediate file: upload_file_21.zip
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
16:53:33 STEP 552 H= 552:00 +CPU= 61.463
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000552
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000552
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000552
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
16:54:38 STEP 553 H= 553:00 +CPU= 36.175
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
16:56:00 STEP 554 H= 554:00 +CPU= 34.436
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
16:57:11 STEP 555 H= 555:00 +CPU= 46.357
16:57:38 STEP 556 H= 556:00 +CPU= 25.982
16:58:01 STEP 557 H= 557:00 +CPU= 23.146
16:58:37 STEP 558 H= 558:00 +CPU= 35.448
16:59:04 STEP 559 H= 559:00 +CPU= 27.430
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 2015050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0854
(argv4) batchid: 984
(argv5) wuid: 12201498
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/0
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201498
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/0/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/0/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_8e9864471c4355a33866531b9302fcee to: /var/lib/boinc-client/slots/0/oifs_43r3_ps_0854_2015050100_123_984_12201498.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/0/oifs_43r3_ps_0854_2015050100_123_984_12201498.zip
ic_ancil_file: ic_ancil_12201498
ifsdata_file: ifsdata_12201498
climate_data_file: clim_data_12201498
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_b282c1bc7cd4c4972e3aca84d2006d0e to: /var/lib/boinc-client/slots/0/ic_ancil_12201498.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/0/ic_ancil_12201498.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_5c03420e2fdf00e5e19c769d3fcf71d9 to: /var/lib/boinc-client/slots/0/ifsdata/ifsdata_12201498.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/0/ifsdata/ifsdata_12201498.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/0/159l_2/clim_data_12201498.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/0/159l_2/clim_data_12201498.zip
Checking for progress XML file: /var/lib/boinc-client/slots/0/progress_file_12201498.xml
Opened progress file ok : /var/lib/boinc-client/slots/0/progress_file_12201498.xml
-- Model is restarting --
Adjusting last_iter, 559, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/0/progress_file_12201498.xml
last_cpu_time: 16795
upload_file_number: 22
last_iter: 551
last_upload: 1983600
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0854_2015050100_123_984_12201498_0_r328028627
The child process has been launched with process id: 7191
Executing the command: /var/lib/boinc-client/slots/0/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:ruly-Inspiron-5558:1:1:7191:7191] [20230120:170011:1674262811:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:ruly-Inspiron-5558:1:1:7191:7191] [20230120:170011:1674262811:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:ruly-Inspiron-5558:1:1:7191:7191] [20230120:170011:1674262811:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:ruly-Inspiron-5558:1:1:7191:7191] [20230120:170011:1674262811:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:ruly-Inspiron-5558:1:1:7191:7191] [20230120:170011:1674262811:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:ruly-Inspiron-5558:1:1:7191:7191] [20230120:170011:1674262811:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647)
[EC_DRHOOK:ruly-Inspiron-5558:1:1:7191:7191] [20230120:170011:1674262811:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:ruly-Inspiron-5558:1:1:7191:7191] [20230120:170011:1674262811:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:ruly-Inspiron-5558:1:1:7191:7191] [20230120:170011:1674262811:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:ruly-Inspiron-5558:1:1:7191:7191] [20230120:170011:1674262811:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:ruly-Inspiron-5558:1:1:7191:7191] [20230120:170011:1674262811:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:ruly-Inspiron-5558:1:1:7191:7191] [20230120:170011:1674262811:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:ruly-Inspiron-5558:1:1:7191:7191] [20230120:170011:1674262811:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:ruly-Inspiron-5558:1:1:7191:7191] [20230120:170011:1674262811:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:ruly-Inspiron-5558:1:1:7191:7191] [20230120:170011:1674262811:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:ruly-Inspiron-5558:1:1:7191:7191] [20230120:170011:1674262811:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ruly-Inspiron-5558:1:1:7191:7191] [20230120:170011:1674262811:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:ruly-Inspiron-5558:1:1:7191:7191] [20230120:170011:1674262811:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ruly-Inspiron-5558:1:1:7191:7191] [20230120:170011:1674262811:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:ruly-Inspiron-5558:1:1:7191:7191] [20230120:170011:1674262811:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:ruly-Inspiron-5558:1:1:7191:7191] [20230120:170011:1674262811:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ruly-Inspiron-5558:1:1:7191:7191] [20230120:170011:1674262811:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ruly-Inspiron-5558:1:1:7191:7191] [20230120:170011:1674262811:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:ruly-Inspiron-5558:1:1:7191:7191] [20230120:170011:1674262811:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ruly-Inspiron-5558:1:1:7191:7191] [20230120:170011:1674262811:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ruly-Inspiron-5558:1:1:7191:7191] [20230120:170011:1674262811:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ruly-Inspiron-5558:1:1:7191:7191] [20230120:170011:1674262811:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:ruly-Inspiron-5558:1:1:7191:7191] [20230120:170011:1674262811:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:ruly-Inspiron-5558:1:1:7191:7191] [20230120:170011:1674262811:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:ruly-Inspiron-5558:1:1:7191:7191] [20230120:170011:1674262811:0.001] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
17:00:57 STEP 552 H= 552:00 +CPU= 42.517
17:01:19 STEP 553 H= 553:00 +CPU= 21.601
17:01:47 STEP 554 H= 554:00 +CPU= 27.796
17:02:21 STEP 555 H= 555:00 +CPU= 34.489
17:02:44 STEP 556 H= 556:00 +CPU= 22.355
17:03:12 STEP 557 H= 557:00 +CPU= 27.643
17:03:47 STEP 558 H= 558:00 +CPU= 34.964
17:04:09 STEP 559 H= 559:00 +CPU= 21.800
17:04:36 STEP 560 H= 560:00 +CPU= 27.216
17:05:10 STEP 561 H= 561:00 +CPU= 34.403
17:05:34 STEP 562 H= 562:00 +CPU= 23.932
17:06:01 STEP 563 H= 563:00 +CPU= 26.133
17:06:39 STEP 564 H= 564:00 +CPU= 38.233
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000564
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000564
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000564
17:07:05 STEP 565 H= 565:00 +CPU= 25.719
17:07:32 STEP 566 H= 566:00 +CPU= 26.759
17:08:06 STEP 567 H= 567:00 +CPU= 34.450
17:08:32 STEP 568 H= 568:00 +CPU= 25.396
17:08:56 STEP 569 H= 569:00 +CPU= 24.456
17:09:31 STEP 570 H= 570:00 +CPU= 35.012
17:09:56 STEP 571 H= 571:00 +CPU= 25.170
17:10:23 STEP 572 H= 572:00 +CPU= 26.736
17:10:58 STEP 573 H= 573:00 +CPU= 34.315
17:11:23 STEP 574 H= 574:00 +CPU= 25.558
17:11:48 STEP 575 H= 575:00 +CPU= 25.064
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201498/ICMGGhq0f+000552
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201498/ICMSHhq0f+000552
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201498/ICMUAhq0f+000552
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201498/ICMGGhq0f+000564
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201498/ICMSHhq0f+000564
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201498/ICMUAhq0f+000564
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0854_2015050100_123_984_12201498_0_r328028627_22.zip
Uploading the intermediate file: upload_file_22.zip
17:12:29 STEP 576 H= 576:00 +CPU= 40.612
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000576
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000576
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000576
17:12:56 STEP 577 H= 577:00 +CPU= 26.489
17:13:18 STEP 578 H= 578:00 +CPU= 22.623
17:13:53 STEP 579 H= 579:00 +CPU= 34.597
17:14:22 STEP 580 H= 580:00 +CPU= 28.463
17:14:44 STEP 581 H= 581:00 +CPU= 22.097
17:15:19 STEP 582 H= 582:00 +CPU= 35.547
17:15:46 STEP 583 H= 583:00 +CPU= 26.614
17:16:09 STEP 584 H= 584:00 +CPU= 22.654
17:16:45 STEP 585 H= 585:00 +CPU= 35.900
17:17:10 STEP 586 H= 586:00 +CPU= 25.118
17:17:32 STEP 587 H= 587:00 +CPU= 21.999
17:18:15 STEP 588 H= 588:00 +CPU= 42.803
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000588
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000588
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000588
17:18:40 STEP 589 H= 589:00 +CPU= 24.790
17:19:03 STEP 590 H= 590:00 +CPU= 22.805
17:19:41 STEP 591 H= 591:00 +CPU= 38.153
17:20:05 STEP 592 H= 592:00 +CPU= 24.057
17:20:27 STEP 593 H= 593:00 +CPU= 21.882
17:21:08 STEP 594 H= 594:00 +CPU= 40.603
17:21:30 STEP 595 H= 595:00 +CPU= 21.799
17:21:52 STEP 596 H= 596:00 +CPU= 22.533
17:22:32 STEP 597 H= 597:00 +CPU= 39.018
17:22:54 STEP 598 H= 598:00 +CPU= 22.422
17:23:17 STEP 599 H= 599:00 +CPU= 22.592
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201498/ICMGGhq0f+000576
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201498/ICMSHhq0f+000576
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201498/ICMUAhq0f+000576
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201498/ICMGGhq0f+000588
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201498/ICMSHhq0f+000588
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201498/ICMUAhq0f+000588
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0854_2015050100_123_984_12201498_0_r328028627_23.zip
Uploading the intermediate file: upload_file_23.zip
17:24:04 STEP 600 H= 600:00 +CPU= 46.875
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000600
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000600
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000600
17:24:26 STEP 601 H= 601:00 +CPU= 21.870
17:24:48 STEP 602 H= 602:00 +CPU= 22.596
17:25:27 STEP 603 H= 603:00 +CPU= 38.579
17:25:50 STEP 604 H= 604:00 +CPU= 22.659
17:26:12 STEP 605 H= 605:00 +CPU= 22.213
17:26:52 STEP 606 H= 606:00 +CPU= 39.978
17:27:14 STEP 607 H= 607:00 +CPU= 21.774
17:27:39 STEP 608 H= 608:00 +CPU= 24.726
17:28:16 STEP 609 H= 609:00 +CPU= 36.829
17:28:39 STEP 610 H= 610:00 +CPU= 22.866
17:29:03 STEP 611 H= 611:00 +CPU= 24.755
17:29:44 STEP 612 H= 612:00 +CPU= 39.947
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000612
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000612
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000612
17:30:05 STEP 613 H= 613:00 +CPU= 21.612
17:30:33 STEP 614 H= 614:00 +CPU= 27.746
17:31:10 STEP 615 H= 615:00 +CPU= 37.018
17:31:33 STEP 616 H= 616:00 +CPU= 22.393
17:31:59 STEP 617 H= 617:00 +CPU= 25.876
17:32:35 STEP 618 H= 618:00 +CPU= 36.707
17:32:57 STEP 619 H= 619:00 +CPU= 22.026
17:33:25 STEP 620 H= 620:00 +CPU= 27.774
17:34:02 STEP 621 H= 621:00 +CPU= 36.871
17:34:24 STEP 622 H= 622:00 +CPU= 22.272
17:34:51 STEP 623 H= 623:00 +CPU= 26.741
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201498/ICMGGhq0f+000600
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201498/ICMSHhq0f+000600
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201498/ICMUAhq0f+000600
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201498/ICMGGhq0f+000612
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201498/ICMSHhq0f+000612
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201498/ICMUAhq0f+000612
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0854_2015050100_123_984_12201498_0_r328028627_24.zip
Uploading the intermediate file: upload_file_24.zip
17:35:35 STEP 624 H= 624:00 +CPU= 43.782
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000624
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000624
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000624
17:35:58 STEP 625 H= 625:00 +CPU= 22.185
17:36:25 STEP 626 H= 626:00 +CPU= 27.057
17:37:02 STEP 627 H= 627:00 +CPU= 37.085
17:37:24 STEP 628 H= 628:00 +CPU= 22.410
17:37:48 STEP 629 H= 629:00 +CPU= 23.297
17:38:25 STEP 630 H= 630:00 +CPU= 37.541
17:38:47 STEP 631 H= 631:00 +CPU= 21.885
17:39:13 STEP 632 H= 632:00 +CPU= 26.251
17:39:52 STEP 633 H= 633:00 +CPU= 38.802
17:40:15 STEP 634 H= 634:00 +CPU= 22.537
17:40:39 STEP 635 H= 635:00 +CPU= 24.201
17:41:20 STEP 636 H= 636:00 +CPU= 41.161
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000636
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000636
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000636
17:41:42 STEP 637 H= 637:00 +CPU= 21.734
17:42:07 STEP 638 H= 638:00 +CPU= 25.177
17:42:42 STEP 639 H= 639:00 +CPU= 34.182
17:43:05 STEP 640 H= 640:00 +CPU= 23.722
17:43:31 STEP 641 H= 641:00 +CPU= 25.425
17:44:06 STEP 642 H= 642:00 +CPU= 34.710
17:44:30 STEP 643 H= 643:00 +CPU= 23.965
17:44:54 STEP 644 H= 644:00 +CPU= 24.605
17:45:28 STEP 645 H= 645:00 +CPU= 33.955
17:45:53 STEP 646 H= 646:00 +CPU= 24.459
17:46:18 STEP 647 H= 647:00 +CPU= 25.156
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201498/ICMGGhq0f+000624
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201498/ICMSHhq0f+000624
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201498/ICMUAhq0f+000624
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201498/ICMGGhq0f+000636
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201498/ICMSHhq0f+000636
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201498/ICMUAhq0f+000636
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0854_2015050100_123_984_12201498_0_r328028627_25.zip
Uploading the intermediate file: upload_file_25.zip
17:46:59 STEP 648 H= 648:00 +CPU= 40.684
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000648
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000648
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000648
17:47:24 STEP 649 H= 649:00 +CPU= 24.460
17:47:46 STEP 650 H= 650:00 +CPU= 22.502
17:48:22 STEP 651 H= 651:00 +CPU= 35.506
17:48:47 STEP 652 H= 652:00 +CPU= 25.626
17:49:09 STEP 653 H= 653:00 +CPU= 21.753
17:49:47 STEP 654 H= 654:00 +CPU= 37.777
17:50:11 STEP 655 H= 655:00 +CPU= 23.877
17:50:33 STEP 656 H= 656:00 +CPU= 22.491
17:51:12 STEP 657 H= 657:00 +CPU= 38.560
17:51:35 STEP 658 H= 658:00 +CPU= 23.062
17:51:57 STEP 659 H= 659:00 +CPU= 22.050
17:52:41 STEP 660 H= 660:00 +CPU= 43.453
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000660
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000660
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000660
17:53:03 STEP 661 H= 661:00 +CPU= 21.833
17:53:25 STEP 662 H= 662:00 +CPU= 22.528
17:54:05 STEP 663 H= 663:00 +CPU= 39.265
17:54:27 STEP 664 H= 664:00 +CPU= 22.548
17:54:49 STEP 665 H= 665:00 +CPU= 21.809
17:55:29 STEP 666 H= 666:00 +CPU= 39.736
17:55:51 STEP 667 H= 667:00 +CPU= 22.030
17:56:14 STEP 668 H= 668:00 +CPU= 22.499
17:56:52 STEP 669 H= 669:00 +CPU= 38.640
17:57:15 STEP 670 H= 670:00 +CPU= 22.221
17:57:37 STEP 671 H= 671:00 +CPU= 22.567
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201498/ICMGGhq0f+000648
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201498/ICMSHhq0f+000648
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201498/ICMUAhq0f+000648
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201498/ICMGGhq0f+000660
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201498/ICMSHhq0f+000660
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201498/ICMUAhq0f+000660
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0854_2015050100_123_984_12201498_0_r328028627_26.zip
Uploading the intermediate file: upload_file_26.zip
17:58:24 STEP 672 H= 672:00 +CPU= 46.143
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000672
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000672
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000672
17:58:46 STEP 673 H= 673:00 +CPU= 21.883
17:59:10 STEP 674 H= 674:00 +CPU= 24.381
17:59:47 STEP 675 H= 675:00 +CPU= 37.361
18:00:10 STEP 676 H= 676:00 +CPU= 22.376
18:00:35 STEP 677 H= 677:00 +CPU= 25.311
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 2015050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0854
(argv4) batchid: 984
(argv5) wuid: 12201498
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/0
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201498
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/0/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/0/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_8e9864471c4355a33866531b9302fcee to: /var/lib/boinc-client/slots/0/oifs_43r3_ps_0854_2015050100_123_984_12201498.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/0/oifs_43r3_ps_0854_2015050100_123_984_12201498.zip
ic_ancil_file: ic_ancil_12201498
ifsdata_file: ifsdata_12201498
climate_data_file: clim_data_12201498
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_b282c1bc7cd4c4972e3aca84d2006d0e to: /var/lib/boinc-client/slots/0/ic_ancil_12201498.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/0/ic_ancil_12201498.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_5c03420e2fdf00e5e19c769d3fcf71d9 to: /var/lib/boinc-client/slots/0/ifsdata/ifsdata_12201498.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/0/ifsdata/ifsdata_12201498.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/0/159l_2/clim_data_12201498.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/0/159l_2/clim_data_12201498.zip
Checking for progress XML file: /var/lib/boinc-client/slots/0/progress_file_12201498.xml
Opened progress file ok : /var/lib/boinc-client/slots/0/progress_file_12201498.xml
-- Model is restarting --
Adjusting last_iter, 678, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/0/progress_file_12201498.xml
last_cpu_time: 20416
upload_file_number: 27
last_iter: 671
last_upload: 2415600
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0854_2015050100_123_984_12201498_0_r328028627
The child process has been launched with process id: 8418
Executing the command: /var/lib/boinc-client/slots/0/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:ruly-Inspiron-5558:1:1:8418:8418] [20230120:180157:1674266517:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:ruly-Inspiron-5558:1:1:8418:8418] [20230120:180157:1674266517:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:ruly-Inspiron-5558:1:1:8418:8418] [20230120:180157:1674266517:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:ruly-Inspiron-5558:1:1:8418:8418] [20230120:180157:1674266517:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:ruly-Inspiron-5558:1:1:8418:8418] [20230120:180157:1674266517:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:ruly-Inspiron-5558:1:1:8418:8418] [20230120:180157:1674266517:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647)
[EC_DRHOOK:ruly-Inspiron-5558:1:1:8418:8418] [20230120:180157:1674266517:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:ruly-Inspiron-5558:1:1:8418:8418] [20230120:180157:1674266517:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:ruly-Inspiron-5558:1:1:8418:8418] [20230120:180157:1674266517:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:ruly-Inspiron-5558:1:1:8418:8418] [20230120:180157:1674266517:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:ruly-Inspiron-5558:1:1:8418:8418] [20230120:180157:1674266517:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:ruly-Inspiron-5558:1:1:8418:8418] [20230120:180157:1674266517:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:ruly-Inspiron-5558:1:1:8418:8418] [20230120:180157:1674266517:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:ruly-Inspiron-5558:1:1:8418:8418] [20230120:180157:1674266517:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:ruly-Inspiron-5558:1:1:8418:8418] [20230120:180157:1674266517:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:ruly-Inspiron-5558:1:1:8418:8418] [20230120:180157:1674266517:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ruly-Inspiron-5558:1:1:8418:8418] [20230120:180157:1674266517:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:ruly-Inspiron-5558:1:1:8418:8418] [20230120:180157:1674266517:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ruly-Inspiron-5558:1:1:8418:8418] [20230120:180157:1674266517:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:ruly-Inspiron-5558:1:1:8418:8418] [20230120:180157:1674266517:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:ruly-Inspiron-5558:1:1:8418:8418] [20230120:180157:1674266517:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ruly-Inspiron-5558:1:1:8418:8418] [20230120:180157:1674266517:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ruly-Inspiron-5558:1:1:8418:8418] [20230120:180157:1674266517:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:ruly-Inspiron-5558:1:1:8418:8418] [20230120:180157:1674266517:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ruly-Inspiron-5558:1:1:8418:8418] [20230120:180157:1674266517:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ruly-Inspiron-5558:1:1:8418:8418] [20230120:180157:1674266517:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ruly-Inspiron-5558:1:1:8418:8418] [20230120:180157:1674266517:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:ruly-Inspiron-5558:1:1:8418:8418] [20230120:180157:1674266517:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:ruly-Inspiron-5558:1:1:8418:8418] [20230120:180157:1674266517:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:ruly-Inspiron-5558:1:1:8418:8418] [20230120:180157:1674266517:0.001] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
18:02:43 STEP 672 H= 672:00 +CPU= 42.880
18:03:05 STEP 673 H= 673:00 +CPU= 21.905
18:03:33 STEP 674 H= 674:00 +CPU= 27.337
18:04:07 STEP 675 H= 675:00 +CPU= 33.840
18:04:29 STEP 676 H= 676:00 +CPU= 22.369
18:04:56 STEP 677 H= 677:00 +CPU= 26.858
18:05:31 STEP 678 H= 678:00 +CPU= 34.702
18:05:53 STEP 679 H= 679:00 +CPU= 21.953
18:06:20 STEP 680 H= 680:00 +CPU= 27.127
18:06:54 STEP 681 H= 681:00 +CPU= 33.831
18:07:17 STEP 682 H= 682:00 +CPU= 23.139
18:07:44 STEP 683 H= 683:00 +CPU= 26.687
18:08:22 STEP 684 H= 684:00 +CPU= 37.734
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000684
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000684
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000684
18:08:46 STEP 685 H= 685:00 +CPU= 24.242
18:09:11 STEP 686 H= 686:00 +CPU= 25.063
18:09:45 STEP 687 H= 687:00 +CPU= 34.112
18:10:11 STEP 688 H= 688:00 +CPU= 26.107
18:10:38 STEP 689 H= 689:00 +CPU= 26.126
18:11:12 STEP 690 H= 690:00 +CPU= 34.669
18:11:36 STEP 691 H= 691:00 +CPU= 23.523
18:12:02 STEP 692 H= 692:00 +CPU= 26.399
18:12:36 STEP 693 H= 693:00 +CPU= 33.702
18:13:01 STEP 694 H= 694:00 +CPU= 25.169
18:13:27 STEP 695 H= 695:00 +CPU= 25.579
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201498/ICMGGhq0f+000672
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201498/ICMSHhq0f+000672
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201498/ICMUAhq0f+000672
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201498/ICMGGhq0f+000684
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201498/ICMSHhq0f+000684
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201498/ICMUAhq0f+000684
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0854_2015050100_123_984_12201498_0_r328028627_27.zip
Uploading the intermediate file: upload_file_27.zip
Abort request received from BOINC client, ending the child process
</stderr_txt>
]]>
