climateprediction.net home page
Task 22263571

Task 22263571

Name oifs_43r3_ps_0053_1993050100_123_962_12178697_0
Workunit 12178697
Created 22 Dec 2022, 2:06:56 UTC
Sent 22 Dec 2022, 3:54:43 UTC
Report deadline 21 Jan 2023, 3:54:43 UTC
Received 19 Jan 2023, 0:52:35 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 196 (0x000000C4) EXIT_DISK_LIMIT_EXCEEDED
Computer ID 1509518
Run time 15 hours 7 min 27 sec
CPU time 8 hours 58 min 32 sec
Validate state Invalid
Credit 2,419.20
Device peak FLOPS 4.11 GFLOPS
Application version OpenIFS 43r3 Perturbed Surface v1.05
x86_64-pc-linux-gnu
Peak working set size 4,417.89 MB
Peak swap size 4,978.07 MB
Peak disk usage 7,704.46 MB
Stderr
<core_client_version>7.16.6</core_client_version>
<![CDATA[
<message>
Disk usage limit exceeded</message>
<stderr_txt>
s now 0 (0x0)
[EC_DRHOOK:ruly-Inspiron-5558:1:1:1926:1926] [20230117:200947:1674014987:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:ruly-Inspiron-5558:1:1:1926:1926] [20230117:200947:1674014987:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:ruly-Inspiron-5558:1:1:1926:1926] [20230117:200947:1674014987:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:ruly-Inspiron-5558:1:1:1926:1926] [20230117:200947:1674014987:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:ruly-Inspiron-5558:1:1:1926:1926] [20230117:200947:1674014987:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:ruly-Inspiron-5558:1:1:1926:1926] [20230117:200947:1674014987:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:ruly-Inspiron-5558:1:1:1926:1926] [20230117:200947:1674014987:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:ruly-Inspiron-5558:1:1:1926:1926] [20230117:200947:1674014987:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:ruly-Inspiron-5558:1:1:1926:1926] [20230117:200947:1674014987:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:ruly-Inspiron-5558:1:1:1926:1926] [20230117:200947:1674014987:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:ruly-Inspiron-5558:1:1:1926:1926] [20230117:200947:1674014987:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:ruly-Inspiron-5558:1:1:1926:1926] [20230117:200947:1674014987:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:ruly-Inspiron-5558:1:1:1926:1926] [20230117:200947:1674014987:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ruly-Inspiron-5558:1:1:1926:1926] [20230117:200947:1674014987:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:ruly-Inspiron-5558:1:1:1926:1926] [20230117:200947:1674014987:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ruly-Inspiron-5558:1:1:1926:1926] [20230117:200947:1674014987:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:ruly-Inspiron-5558:1:1:1926:1926] [20230117:200947:1674014987:0.001] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:ruly-Inspiron-5558:1:1:1926:1926] [20230117:200947:1674014987:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ruly-Inspiron-5558:1:1:1926:1926] [20230117:200947:1674014987:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ruly-Inspiron-5558:1:1:1926:1926] [20230117:200947:1674014987:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:ruly-Inspiron-5558:1:1:1926:1926] [20230117:200947:1674014987:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ruly-Inspiron-5558:1:1:1926:1926] [20230117:200947:1674014987:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ruly-Inspiron-5558:1:1:1926:1926] [20230117:200947:1674014987:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ruly-Inspiron-5558:1:1:1926:1926] [20230117:200947:1674014987:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:ruly-Inspiron-5558:1:1:1926:1926] [20230117:200947:1674014987:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:ruly-Inspiron-5558:1:1:1926:1926] [20230117:200947:1674014987:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:ruly-Inspiron-5558:1:1:1926:1926] [20230117:200947:1674014987:0.001] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:13:19 STEP  600 H= 600:00 +CPU= 60.179
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:13:58 STEP  601 H= 601:00 +CPU= 27.140
  04:14:21 STEP  602 H= 602:00 +CPU= 22.571
  04:14:57 STEP  603 H= 603:00 +CPU= 36.541
  04:15:24 STEP  604 H= 604:00 +CPU= 26.438
  04:15:46 STEP  605 H= 605:00 +CPU= 21.984
  04:16:23 STEP  606 H= 606:00 +CPU= 36.792
  04:16:47 STEP  607 H= 607:00 +CPU= 24.579
  04:17:10 STEP  608 H= 608:00 +CPU= 22.648
  04:17:51 STEP  609 H= 609:00 +CPU= 38.878
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:18:26 STEP  610 H= 610:00 +CPU= 23.335
  04:18:51 STEP  611 H= 611:00 +CPU= 24.315
  04:19:33 STEP  612 H= 612:00 +CPU= 41.820
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000612
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000612
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000612
  04:19:56 STEP  613 H= 613:00 +CPU= 22.369
  04:20:24 STEP  614 H= 614:00 +CPU= 28.042
  04:20:59 STEP  615 H= 615:00 +CPU= 34.347
  04:21:23 STEP  616 H= 616:00 +CPU= 23.573
  04:21:51 STEP  617 H= 617:00 +CPU= 27.248
  04:22:26 STEP  618 H= 618:00 +CPU= 34.867
  04:22:49 STEP  619 H= 619:00 +CPU= 22.891
  04:23:16 STEP  620 H= 620:00 +CPU= 26.804
  04:23:50 STEP  621 H= 621:00 +CPU= 34.023
  04:24:15 STEP  622 H= 622:00 +CPU= 24.950
  04:24:42 STEP  623 H= 623:00 +CPU= 26.240
  04:25:23 STEP  624 H= 624:00 +CPU= 40.677
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000624
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000624
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000624
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12178697/ICMGGhq0f+000600
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12178697/ICMSHhq0f+000600
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12178697/ICMUAhq0f+000600
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12178697/ICMGGhq0f+000612
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12178697/ICMSHhq0f+000612
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12178697/ICMUAhq0f+000612
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0053_1993050100_123_962_12178697_0_r379649415_25.zip
Uploading the intermediate file: upload_file_25.zip
  04:25:51 STEP  625 H= 625:00 +CPU= 28.260
  04:26:14 STEP  626 H= 626:00 +CPU= 22.712
  04:26:48 STEP  627 H= 627:00 +CPU= 34.306
  04:27:13 STEP  628 H= 628:00 +CPU= 24.543
  04:27:35 STEP  629 H= 629:00 +CPU= 22.191
  04:28:14 STEP  630 H= 630:00 +CPU= 38.659
  04:28:38 STEP  631 H= 631:00 +CPU= 23.763
  04:29:00 STEP  632 H= 632:00 +CPU= 22.764
  04:29:37 STEP  633 H= 633:00 +CPU= 36.366
  04:30:00 STEP  634 H= 634:00 +CPU= 22.922
  04:30:22 STEP  635 H= 635:00 +CPU= 22.297
  04:31:06 STEP  636 H= 636:00 +CPU= 43.586
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000636
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000636
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000636
  04:31:28 STEP  637 H= 637:00 +CPU= 22.107
  04:31:51 STEP  638 H= 638:00 +CPU= 22.825
  04:32:30 STEP  639 H= 639:00 +CPU= 38.812
  04:32:52 STEP  640 H= 640:00 +CPU= 22.733
  04:33:15 STEP  641 H= 641:00 +CPU= 22.919
  04:33:54 STEP  642 H= 642:00 +CPU= 38.537
  04:34:16 STEP  643 H= 643:00 +CPU= 22.030
  04:34:41 STEP  644 H= 644:00 +CPU= 24.983
  04:35:18 STEP  645 H= 645:00 +CPU= 37.214
  04:35:41 STEP  646 H= 646:00 +CPU= 22.861
  04:36:08 STEP  647 H= 647:00 +CPU= 26.605
  04:36:53 STEP  648 H= 648:00 +CPU= 44.663
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000648
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000648
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000648
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12178697/ICMGGhq0f+000624
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12178697/ICMSHhq0f+000624
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12178697/ICMUAhq0f+000624
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12178697/ICMGGhq0f+000636
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12178697/ICMSHhq0f+000636
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12178697/ICMUAhq0f+000636
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0053_1993050100_123_962_12178697_0_r379649415_26.zip
Uploading the intermediate file: upload_file_26.zip
  04:37:15 STEP  649 H= 649:00 +CPU= 22.162
  04:37:42 STEP  650 H= 650:00 +CPU= 27.116
  04:38:16 STEP  651 H= 651:00 +CPU= 34.110
  04:38:39 STEP  652 H= 652:00 +CPU= 22.475
  04:39:04 STEP  653 H= 653:00 +CPU= 24.874
  04:39:46 STEP  654 H= 654:00 +CPU= 39.784
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:40:34 STEP  655 H= 655:00 +CPU= 32.124
  04:41:02 STEP  656 H= 656:00 +CPU= 26.474
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:42:31 STEP  657 H= 657:00 +CPU= 50.214
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 1993050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0053
(argv4) batchid: 962
(argv5) wuid: 12178697
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/1
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12178697
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/1/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/1/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_f04f29ed294dfc1ab47988c881f6a14a to: /var/lib/boinc-client/slots/1/oifs_43r3_ps_0053_1993050100_123_962_12178697.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/1/oifs_43r3_ps_0053_1993050100_123_962_12178697.zip
ic_ancil_file: ic_ancil_12178697
ifsdata_file: ifsdata_12178697
climate_data_file: clim_data_12178697
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_0eaef503942db79873eccd178570a5d1 to: /var/lib/boinc-client/slots/1/ic_ancil_12178697.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/1/ic_ancil_12178697.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_ec8db43afe47977dbae7d900b2ce83aa to: /var/lib/boinc-client/slots/1/ifsdata/ifsdata_12178697.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/1/ifsdata/ifsdata_12178697.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/1/159l_2/clim_data_12178697.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/1/159l_2/clim_data_12178697.zip
Checking for progress XML file: /var/lib/boinc-client/slots/1/progress_file_12178697.xml
Opened progress file ok : /var/lib/boinc-client/slots/1/progress_file_12178697.xml
-- Model is restarting --
Adjusting last_iter, 658, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/1/progress_file_12178697.xml
last_cpu_time: 25327
upload_file_number: 27
last_iter: 647
last_upload: 2332800
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0053_1993050100_123_962_12178697_0_r379649415
The child process has been launched with process id: 5978
Executing the command: /var/lib/boinc-client/slots/1/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:ruly-Inspiron-5558:1:1:5978:5978] [20230118:044434:1674045874:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:ruly-Inspiron-5558:1:1:5978:5978] [20230118:044434:1674045874:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:ruly-Inspiron-5558:1:1:5978:5978] [20230118:044434:1674045874:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:ruly-Inspiron-5558:1:1:5978:5978] [20230118:044434:1674045874:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:ruly-Inspiron-5558:1:1:5978:5978] [20230118:044434:1674045874:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:ruly-Inspiron-5558:1:1:5978:5978] [20230118:044434:1674045874:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:ruly-Inspiron-5558:1:1:5978:5978] [20230118:044434:1674045874:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:ruly-Inspiron-5558:1:1:5978:5978] [20230118:044434:1674045874:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:ruly-Inspiron-5558:1:1:5978:5978] [20230118:044434:1674045874:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:ruly-Inspiron-5558:1:1:5978:5978] [20230118:044434:1674045874:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:ruly-Inspiron-5558:1:1:5978:5978] [20230118:044434:1674045874:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:ruly-Inspiron-5558:1:1:5978:5978] [20230118:044434:1674045874:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:ruly-Inspiron-5558:1:1:5978:5978] [20230118:044434:1674045874:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:ruly-Inspiron-5558:1:1:5978:5978] [20230118:044434:1674045874:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:ruly-Inspiron-5558:1:1:5978:5978] [20230118:044434:1674045874:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:ruly-Inspiron-5558:1:1:5978:5978] [20230118:044434:1674045874:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ruly-Inspiron-5558:1:1:5978:5978] [20230118:044434:1674045874:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:ruly-Inspiron-5558:1:1:5978:5978] [20230118:044434:1674045874:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ruly-Inspiron-5558:1:1:5978:5978] [20230118:044434:1674045874:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:ruly-Inspiron-5558:1:1:5978:5978] [20230118:044434:1674045874:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:ruly-Inspiron-5558:1:1:5978:5978] [20230118:044434:1674045874:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ruly-Inspiron-5558:1:1:5978:5978] [20230118:044434:1674045874:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ruly-Inspiron-5558:1:1:5978:5978] [20230118:044434:1674045874:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:ruly-Inspiron-5558:1:1:5978:5978] [20230118:044434:1674045874:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ruly-Inspiron-5558:1:1:5978:5978] [20230118:044434:1674045874:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ruly-Inspiron-5558:1:1:5978:5978] [20230118:044434:1674045874:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ruly-Inspiron-5558:1:1:5978:5978] [20230118:044434:1674045874:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:ruly-Inspiron-5558:1:1:5978:5978] [20230118:044434:1674045874:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:ruly-Inspiron-5558:1:1:5978:5978] [20230118:044434:1674045874:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:ruly-Inspiron-5558:1:1:5978:5978] [20230118:044434:1674045874:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  04:45:33 STEP  648 H= 648:00 +CPU= 50.424
  04:46:07 STEP  649 H= 649:00 +CPU= 30.327
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:46:53 STEP  650 H= 650:00 +CPU= 31.350
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:48:02 STEP  651 H= 651:00 +CPU= 44.010
  04:48:34 STEP  652 H= 652:00 +CPU= 29.812
  04:49:03 STEP  653 H= 653:00 +CPU= 27.228
  04:49:48 STEP  654 H= 654:00 +CPU= 43.694
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:51:06 STEP  655 H= 655:00 +CPU= 31.771
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:53:13 STEP  656 H= 656:00 +CPU= 30.170
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:54:22 STEP  657 H= 657:00 +CPU= 50.923
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:55:09 STEP  658 H= 658:00 +CPU= 32.405
  04:55:43 STEP  659 H= 659:00 +CPU= 31.004
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:57:27 STEP  660 H= 660:00 +CPU= 54.298
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000660
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000660
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000660
  04:58:03 STEP  661 H= 661:00 +CPU= 32.312
  04:58:32 STEP  662 H= 662:00 +CPU= 27.458
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:00:25 STEP  663 H= 663:00 +CPU= 46.619
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:01:17 STEP  664 H= 664:00 +CPU= 34.670
  05:01:46 STEP  665 H= 665:00 +CPU= 27.186
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:02:58 STEP  666 H= 666:00 +CPU= 52.534
  05:03:27 STEP  667 H= 667:00 +CPU= 27.457
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:04:14 STEP  668 H= 668:00 +CPU= 32.421
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:06:03 STEP  669 H= 669:00 +CPU= 50.346
  05:06:33 STEP  670 H= 670:00 +CPU= 28.094
  05:07:07 STEP  671 H= 671:00 +CPU= 31.536
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:08:23 STEP  672 H= 672:00 +CPU= 57.180
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000672
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000672
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000672
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12178697/ICMGGhq0f+000648
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12178697/ICMSHhq0f+000648
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12178697/ICMUAhq0f+000648
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12178697/ICMGGhq0f+000660
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12178697/ICMSHhq0f+000660
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12178697/ICMUAhq0f+000660
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0053_1993050100_123_962_12178697_0_r379649415_27.zip
Uploading the intermediate file: upload_file_27.zip
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:09:27 STEP  673 H= 673:00 +CPU= 34.797
  05:09:59 STEP  674 H= 674:00 +CPU= 29.595
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:11:48 STEP  675 H= 675:00 +CPU= 50.952
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:12:49 STEP  676 H= 676:00 +CPU= 34.188
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:14:10 STEP  677 H= 677:00 +CPU= 32.752
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:15:14 STEP  678 H= 678:00 +CPU= 47.961
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:15:57 STEP  679 H= 679:00 +CPU= 29.547
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:16:45 STEP  680 H= 680:00 +CPU= 31.941
  05:17:40 STEP  681 H= 681:00 +CPU= 49.065
  05:18:17 STEP  682 H= 682:00 +CPU= 32.874
  05:18:52 STEP  683 H= 683:00 +CPU= 31.580
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:21:15 STEP  684 H= 684:00 +CPU= 58.748
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000684
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000684
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000684
  05:21:52 STEP  685 H= 685:00 +CPU= 33.389
  05:22:27 STEP  686 H= 686:00 +CPU= 32.037
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:24:22 STEP  687 H= 687:00 +CPU= 53.970
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:25:07 STEP  688 H= 688:00 +CPU= 31.049
  05:25:30 STEP  689 H= 689:00 +CPU= 22.270
  05:26:07 STEP  690 H= 690:00 +CPU= 37.344
  05:26:29 STEP  691 H= 691:00 +CPU= 22.297
  05:26:56 STEP  692 H= 692:00 +CPU= 26.362
  05:27:32 STEP  693 H= 693:00 +CPU= 36.598
  05:27:55 STEP  694 H= 694:00 +CPU= 22.809
  05:28:23 STEP  695 H= 695:00 +CPU= 27.231
  05:29:06 STEP  696 H= 696:00 +CPU= 43.318
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000696
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000696
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000696
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12178697/ICMGGhq0f+000672
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12178697/ICMSHhq0f+000672
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12178697/ICMUAhq0f+000672
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12178697/ICMGGhq0f+000684
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12178697/ICMSHhq0f+000684
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12178697/ICMUAhq0f+000684
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0053_1993050100_123_962_12178697_0_r379649415_28.zip
Uploading the intermediate file: upload_file_28.zip
  05:29:29 STEP  697 H= 697:00 +CPU= 22.487
  05:29:58 STEP  698 H= 698:00 +CPU= 29.217
  05:30:33 STEP  699 H= 699:00 +CPU= 34.540
  05:30:56 STEP  700 H= 700:00 +CPU= 22.999
  05:31:22 STEP  701 H= 701:00 +CPU= 25.974
  05:31:57 STEP  702 H= 702:00 +CPU= 35.156
  05:32:20 STEP  703 H= 703:00 +CPU= 23.195
  05:32:47 STEP  704 H= 704:00 +CPU= 26.731
  05:33:22 STEP  705 H= 705:00 +CPU= 34.539
  05:33:48 STEP  706 H= 706:00 +CPU= 26.049
  05:34:14 STEP  707 H= 707:00 +CPU= 25.671
  05:34:52 STEP  708 H= 708:00 +CPU= 37.988
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000708
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000708
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000708
  05:35:20 STEP  709 H= 709:00 +CPU= 27.983
  05:35:43 STEP  710 H= 710:00 +CPU= 23.084
  05:36:17 STEP  711 H= 711:00 +CPU= 34.229
  05:36:46 STEP  712 H= 712:00 +CPU= 28.498
  05:37:08 STEP  713 H= 713:00 +CPU= 22.209
  05:37:44 STEP  714 H= 714:00 +CPU= 36.345
  05:38:10 STEP  715 H= 715:00 +CPU= 26.098
  05:38:33 STEP  716 H= 716:00 +CPU= 22.874
  05:39:10 STEP  717 H= 717:00 +CPU= 36.288
  05:39:36 STEP  718 H= 718:00 +CPU= 25.779
  05:39:58 STEP  719 H= 719:00 +CPU= 22.933
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:41:09 STEP  720 H= 720:00 +CPU= 53.576
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000720
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000720
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000720
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12178697/ICMGGhq0f+000696
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12178697/ICMSHhq0f+000696
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12178697/ICMUAhq0f+000696
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12178697/ICMGGhq0f+000708
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12178697/ICMSHhq0f+000708
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12178697/ICMUAhq0f+000708
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0053_1993050100_123_962_12178697_0_r379649415_29.zip
Uploading the intermediate file: upload_file_29.zip
  05:41:32 STEP  721 H= 721:00 +CPU= 22.486
  05:42:00 STEP  722 H= 722:00 +CPU= 27.585
  05:42:35 STEP  723 H= 723:00 +CPU= 35.375
  05:42:58 STEP  724 H= 724:00 +CPU= 22.908
  05:43:26 STEP  725 H= 725:00 +CPU= 27.916
  05:44:02 STEP  726 H= 726:00 +CPU= 35.869
  05:44:24 STEP  727 H= 727:00 +CPU= 22.380
  05:44:49 STEP  728 H= 728:00 +CPU= 24.927
  05:45:24 STEP  729 H= 729:00 +CPU= 34.382
  05:45:51 STEP  730 H= 730:00 +CPU= 27.257
  05:46:14 STEP  731 H= 731:00 +CPU= 23.224
  05:46:53 STEP  732 H= 732:00 +CPU= 38.144
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000732
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000732
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000732
  05:47:20 STEP  733 H= 733:00 +CPU= 27.737
  05:47:43 STEP  734 H= 734:00 +CPU= 22.847
  05:48:18 STEP  735 H= 735:00 +CPU= 34.555
  05:48:43 STEP  736 H= 736:00 +CPU= 24.768
  05:49:05 STEP  737 H= 737:00 +CPU= 22.224
  05:49:45 STEP  738 H= 738:00 +CPU= 40.297
  05:50:08 STEP  739 H= 739:00 +CPU= 22.744
  05:50:31 STEP  740 H= 740:00 +CPU= 22.872
  05:51:10 STEP  741 H= 741:00 +CPU= 39.371
  05:51:33 STEP  742 H= 742:00 +CPU= 22.744
  05:51:56 STEP  743 H= 743:00 +CPU= 23.111
  05:52:42 STEP  744 H= 744:00 +CPU= 45.864
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000744
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000744
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000744
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12178697/ICMGGhq0f+000720
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12178697/ICMSHhq0f+000720
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12178697/ICMUAhq0f+000720
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12178697/ICMGGhq0f+000732
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12178697/ICMSHhq0f+000732
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12178697/ICMUAhq0f+000732
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0053_1993050100_123_962_12178697_0_r379649415_30.zip
Uploading the intermediate file: upload_file_30.zip
  05:53:05 STEP  745 H= 745:00 +CPU= 22.422
  05:53:28 STEP  746 H= 746:00 +CPU= 22.906
  05:54:07 STEP  747 H= 747:00 +CPU= 39.014
  05:54:30 STEP  748 H= 748:00 +CPU= 22.976
  05:54:53 STEP  749 H= 749:00 +CPU= 22.849
  05:55:32 STEP  750 H= 750:00 +CPU= 38.846
  05:55:54 STEP  751 H= 751:00 +CPU= 22.465
  05:56:19 STEP  752 H= 752:00 +CPU= 24.838
  05:56:56 STEP  753 H= 753:00 +CPU= 36.828
  05:57:19 STEP  754 H= 754:00 +CPU= 22.829
  05:57:45 STEP  755 H= 755:00 +CPU= 25.779
  05:58:25 STEP  756 H= 756:00 +CPU= 40.583
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000756
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000756
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000756
  05:58:48 STEP  757 H= 757:00 +CPU= 22.250
  05:59:17 STEP  758 H= 758:00 +CPU= 29.009
  05:59:53 STEP  759 H= 759:00 +CPU= 36.635
  06:00:16 STEP  760 H= 760:00 +CPU= 22.806
  06:00:44 STEP  761 H= 761:00 +CPU= 27.529
  06:01:19 STEP  762 H= 762:00 +CPU= 34.964
  06:01:41 STEP  763 H= 763:00 +CPU= 22.121
  06:02:09 STEP  764 H= 764:00 +CPU= 27.218
  06:02:43 STEP  765 H= 765:00 +CPU= 34.332
  06:03:06 STEP  766 H= 766:00 +CPU= 23.176
  06:03:33 STEP  767 H= 767:00 +CPU= 26.507
  06:04:14 STEP  768 H= 768:00 +CPU= 41.130
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000768
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000768
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000768
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12178697/ICMGGhq0f+000744
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12178697/ICMSHhq0f+000744
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12178697/ICMUAhq0f+000744
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12178697/ICMGGhq0f+000756
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12178697/ICMSHhq0f+000756
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12178697/ICMUAhq0f+000756
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0053_1993050100_123_962_12178697_0_r379649415_31.zip
Uploading the intermediate file: upload_file_31.zip
  06:04:40 STEP  769 H= 769:00 +CPU= 26.312
  06:05:05 STEP  770 H= 770:00 +CPU= 24.723
  06:05:40 STEP  771 H= 771:00 +CPU= 34.357
  06:06:05 STEP  772 H= 772:00 +CPU= 24.969
  06:06:27 STEP  773 H= 773:00 +CPU= 22.330
  06:07:04 STEP  774 H= 774:00 +CPU= 36.901
  06:07:29 STEP  775 H= 775:00 +CPU= 25.416
  06:07:52 STEP  776 H= 776:00 +CPU= 22.839
  06:08:30 STEP  777 H= 777:00 +CPU= 37.512
  06:08:55 STEP  778 H= 778:00 +CPU= 25.455
  06:09:18 STEP  779 H= 779:00 +CPU= 22.203
  06:09:59 STEP  780 H= 780:00 +CPU= 40.857
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000780
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000780
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000780
  06:10:21 STEP  781 H= 781:00 +CPU= 22.454
  06:10:44 STEP  782 H= 782:00 +CPU= 23.046
  06:11:22 STEP  783 H= 783:00 +CPU= 37.609
  06:11:45 STEP  784 H= 784:00 +CPU= 22.747
  06:12:09 STEP  785 H= 785:00 +CPU= 24.356
  06:12:47 STEP  786 H= 786:00 +CPU= 37.319
  06:13:09 STEP  787 H= 787:00 +CPU= 22.035
  06:13:35 STEP  788 H= 788:00 +CPU= 26.578
  06:14:11 STEP  789 H= 789:00 +CPU= 35.957
  06:14:34 STEP  790 H= 790:00 +CPU= 22.607
  06:15:03 STEP  791 H= 791:00 +CPU= 28.540
  06:15:46 STEP  792 H= 792:00 +CPU= 43.016
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000792
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000792
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000792
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12178697/ICMGGhq0f+000768
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12178697/ICMSHhq0f+000768
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12178697/ICMUAhq0f+000768
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12178697/ICMGGhq0f+000780
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12178697/ICMSHhq0f+000780
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12178697/ICMUAhq0f+000780
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0053_1993050100_123_962_12178697_0_r379649415_32.zip
Uploading the intermediate file: upload_file_32.zip
  06:16:08 STEP  793 H= 793:00 +CPU= 22.436
  06:16:34 STEP  794 H= 794:00 +CPU= 25.331
  06:17:08 STEP  795 H= 795:00 +CPU= 34.262
  06:17:33 STEP  796 H= 796:00 +CPU= 24.852
  06:17:58 STEP  797 H= 797:00 +CPU= 24.646
  06:18:33 STEP  798 H= 798:00 +CPU= 34.895
  06:18:58 STEP  799 H= 799:00 +CPU= 25.076
  06:19:22 STEP  800 H= 800:00 +CPU= 24.392
  06:19:57 STEP  801 H= 801:00 +CPU= 34.389
  06:20:25 STEP  802 H= 802:00 +CPU= 28.205
  06:20:47 STEP  803 H= 803:00 +CPU= 22.504
  06:21:26 STEP  804 H= 804:00 +CPU= 38.095
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000804
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000804
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000804
  06:21:52 STEP  805 H= 805:00 +CPU= 26.733
  06:22:15 STEP  806 H= 806:00 +CPU= 22.882
  06:22:50 STEP  807 H= 807:00 +CPU= 35.122
  06:23:17 STEP  808 H= 808:00 +CPU= 26.725
  06:23:39 STEP  809 H= 809:00 +CPU= 22.226
  06:24:18 STEP  810 H= 810:00 +CPU= 38.199
  06:24:42 STEP  811 H= 811:00 +CPU= 24.172
  06:25:05 STEP  812 H= 812:00 +CPU= 22.919
  06:25:43 STEP  813 H= 813:00 +CPU= 38.081
  06:26:07 STEP  814 H= 814:00 +CPU= 24.170
  06:26:30 STEP  815 H= 815:00 +CPU= 22.863
  06:27:17 STEP  816 H= 816:00 +CPU= 46.342
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000816
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000816
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000816
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12178697/ICMGGhq0f+000792
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12178697/ICMSHhq0f+000792
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12178697/ICMUAhq0f+000792
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12178697/ICMGGhq0f+000804
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12178697/ICMSHhq0f+000804
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12178697/ICMUAhq0f+000804
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0053_1993050100_123_962_12178697_0_r379649415_33.zip
Uploading the intermediate file: upload_file_33.zip
  06:27:39 STEP  817 H= 817:00 +CPU= 22.265
  06:28:02 STEP  818 H= 818:00 +CPU= 22.736
  06:28:41 STEP  819 H= 819:00 +CPU= 39.310
  06:29:04 STEP  820 H= 820:00 +CPU= 22.880
  06:29:26 STEP  821 H= 821:00 +CPU= 21.971
  06:30:04 STEP  822 H= 822:00 +CPU= 37.625
  06:30:26 STEP  823 H= 823:00 +CPU= 22.136
  06:30:51 STEP  824 H= 824:00 +CPU= 24.950
  06:31:28 STEP  825 H= 825:00 +CPU= 36.985
  06:31:51 STEP  826 H= 826:00 +CPU= 22.811
  06:32:17 STEP  827 H= 827:00 +CPU= 25.754
  06:32:59 STEP  828 H= 828:00 +CPU= 42.670
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000828
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000828
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000828
  06:33:22 STEP  829 H= 829:00 +CPU= 22.254
  06:33:48 STEP  830 H= 830:00 +CPU= 25.826
  06:34:24 STEP  831 H= 831:00 +CPU= 36.061
  06:34:46 STEP  832 H= 832:00 +CPU= 22.778
  06:35:20 STEP  833 H= 833:00 +CPU= 30.638
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:36:20 STEP  834 H= 834:00 +CPU= 45.361
  06:36:52 STEP  835 H= 835:00 +CPU= 29.325
  06:37:17 STEP  836 H= 836:00 +CPU= 24.245
  06:38:06 STEP  837 H= 837:00 +CPU= 44.555
  06:38:33 STEP  838 H= 838:00 +CPU= 25.495
  06:39:03 STEP  839 H= 839:00 +CPU= 28.461
  06:39:51 STEP  840 H= 840:00 +CPU= 47.200
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000840
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000840
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000840
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12178697/ICMGGhq0f+000816
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12178697/ICMSHhq0f+000816
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12178697/ICMUAhq0f+000816
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12178697/ICMGGhq0f+000828
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12178697/ICMSHhq0f+000828
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12178697/ICMUAhq0f+000828
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0053_1993050100_123_962_12178697_0_r379649415_34.zip
Uploading the intermediate file: upload_file_34.zip
  06:40:21 STEP  841 H= 841:00 +CPU= 27.778
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:41:28 STEP  842 H= 842:00 +CPU= 30.959
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:43:23 STEP  843 H= 843:00 +CPU= 47.248
  06:43:49 STEP  844 H= 844:00 +CPU= 24.891
  06:44:20 STEP  845 H= 845:00 +CPU= 29.173
[EC_DRHOOK:ruly-Inspiron-5558:1:1:5978:5978] [20230118:064425:1674053065:7190.389] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1538] Received signal#15 (SIGTERM) :: 3900MB (heap), 4411MB (maxrss), 0MB (maxstack), 0 (paging), nsigs = 1
[EC_DRHOOK:ruly-Inspiron-5558:1:1:5978:5978] [20230118:064425:1674053065:7190.392] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1542] Also activating Harakiri-alarm (SIGALRM=14) to expire after 500s elapsed to prevent hangs, nsigs = 1
[EC_DRHOOK:ruly-Inspiron-5558:1:1:5978:5978] [20230118:064425:1674053065:7190.392] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1544] Harakiri signal handler 'signal_harakiri' for signal#14 (SIGALRM) installed at 0x81f3c0 (old at (nil))
[EC_DRHOOK:ruly-Inspiron-5558:1:1:5978:5978] [20230118:064425:1674053065:7190.392] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1617] Signal#15 was caused by unrecognized si_code [memaddr=0x1], nsigs = 1
[EC_DRHOOK:ruly-Inspiron-5558:1:1:5978:5978] [20230118:064425:1674053065:7190.392] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1686] Starting DrHook backtrace for signal#15, nsigs = 1
[EC_DRHOOK:ruly-Inspiron-5558:1:1:5978:5978] [20230118:064425:1674053065:7190.392] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3843] 3900 MB (maxheap), 4411 MB (maxrss), 0 MB (maxstack)
[EC_DRHOOK:ruly-Inspiron-5558:1:1:5978:5978] [20230118:064425:1674053065:7190.392] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :  MASTER 
[EC_DRHOOK:ruly-Inspiron-5558:1:1:5978:5978] [20230118:064425:1674053065:7190.392] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :   CNT0 
[EC_DRHOOK:ruly-Inspiron-5558:1:1:5978:5978] [20230118:064425:1674053065:7190.392] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :    CNT1 
[EC_DRHOOK:ruly-Inspiron-5558:1:1:5978:5978] [20230118:064425:1674053065:7190.392] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :     CNT2 
[EC_DRHOOK:ruly-Inspiron-5558:1:1:5978:5978] [20230118:064425:1674053065:7190.392] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :      CNT3 
[EC_DRHOOK:ruly-Inspiron-5558:1:1:5978:5978] [20230118:064425:1674053065:7190.392] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :       CNT4 
[EC_DRHOOK:ruly-Inspiron-5558:1:1:5978:5978] [20230118:064425:1674053065:7190.392] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :        STEPO 
[EC_DRHOOK:ruly-Inspiron-5558:1:1:5978:5978] [20230118:064425:1674053065:7190.392] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :         SCAN2M 
[EC_DRHOOK:ruly-Inspiron-5558:1:1:5978:5978] [20230118:064425:1674053065:7190.392] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :          GP_MODEL_HEAP 
[EC_DRHOOK:ruly-Inspiron-5558:1:1:5978:5978] [20230118:064425:1674053065:7190.392] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :           GP_MODEL 
[EC_DRHOOK:ruly-Inspiron-5558:1:1:5978:5978] [20230118:064425:1674053065:7190.392] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :            RADDRV 
[EC_DRHOOK:ruly-Inspiron-5558:1:1:5978:5978] [20230118:064425:1674053065:7190.392] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :             >OMP-RADDRV      2        (1202) 
[EC_DRHOOK:ruly-Inspiron-5558:1:1:5978:5978] [20230118:064426:1674053066:7191.392] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1734] DrHook backtrace done for signal#15, nsigs = 1
[EC_DRHOOK:ruly-Inspiron-5558:1:1:5978:5978] [20230118:064426:1674053066:7191.392] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1785] Calling previous signal handler at 0x1ce8bb0 for signal#15, nsigs = 1
forrtl: error (78): process killed (SIGTERM)
Image              PC                Routine            Line        Source             
oifs_43r3_model.e  0000000001CE916B  Unknown               Unknown  Unknown
oifs_43r3_model.e  000000000081FFF1  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001DC9090  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001DAFCCA  Unknown               Unknown  Unknown
oifs_43r3_model.e  000000000126F37D  raddrv_                   504  raddrv.F90
oifs_43r3_model.e  0000000000BFD839  gp_model_                 482  gp_model.F90
oifs_43r3_model.e  00000000012E9482  gp_model_heap_             74  gp_model_heap.F90
oifs_43r3_model.e  0000000000BBC1CE  scan2m_                   535  scan2m.F90
oifs_43r3_model.e  000000000057A664  stepo_                    327  stepo.F90
oifs_43r3_model.e  000000000055DEEC  cnt4_                    1133  cnt4.F90
oifs_43r3_model.e  00000000005471C9  cnt3_                     267  cnt3.F90
oifs_43r3_model.e  0000000000546412  cnt2_                      88  cnt2.F90
oifs_43r3_model.e  0000000000545E28  cnt1_                      92  cnt1.F90
oifs_43r3_model.e  000000000040708D  cnt0_                     146  cnt0.F90
oifs_43r3_model.e  000000000040220F  MAIN__                     96  master.F90
oifs_43r3_model.e  00000000004021A2  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001DCA390  Unknown               Unknown  Unknown
oifs_43r3_model.e  000000000040206E  Unknown               Unknown  Unknown
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 1993050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0053
(argv4) batchid: 962
(argv5) wuid: 12178697
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/1
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12178697
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/1/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/1/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_f04f29ed294dfc1ab47988c881f6a14a to: /var/lib/boinc-client/slots/1/oifs_43r3_ps_0053_1993050100_123_962_12178697.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/1/oifs_43r3_ps_0053_1993050100_123_962_12178697.zip
ic_ancil_file: ic_ancil_12178697
ifsdata_file: ifsdata_12178697
climate_data_file: clim_data_12178697
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_0eaef503942db79873eccd178570a5d1 to: /var/lib/boinc-client/slots/1/ic_ancil_12178697.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/1/ic_ancil_12178697.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_ec8db43afe47977dbae7d900b2ce83aa to: /var/lib/boinc-client/slots/1/ifsdata/ifsdata_12178697.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/1/ifsdata/ifsdata_12178697.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/1/159l_2/clim_data_12178697.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/1/159l_2/clim_data_12178697.zip
Checking for progress XML file: /var/lib/boinc-client/slots/1/progress_file_12178697.xml
Opened progress file ok : /var/lib/boinc-client/slots/1/progress_file_12178697.xml
-- Model is restarting --
Adjusting last_iter, 846, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/1/progress_file_12178697.xml
last_cpu_time: 31471
upload_file_number: 35
last_iter: 839
last_upload: 3024000
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0053_1993050100_123_962_12178697_0_r379649415
The child process has been launched with process id: 1906
Executing the command: /var/lib/boinc-client/slots/1/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:ruly-Inspiron-5558:1:1:1906:1906] [20230118:163050:1674088250:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:ruly-Inspiron-5558:1:1:1906:1906] [20230118:163050:1674088250:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:ruly-Inspiron-5558:1:1:1906:1906] [20230118:163050:1674088250:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:ruly-Inspiron-5558:1:1:1906:1906] [20230118:163050:1674088250:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:ruly-Inspiron-5558:1:1:1906:1906] [20230118:163050:1674088250:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:ruly-Inspiron-5558:1:1:1906:1906] [20230118:163050:1674088250:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:ruly-Inspiron-5558:1:1:1906:1906] [20230118:163050:1674088250:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:ruly-Inspiron-5558:1:1:1906:1906] [20230118:163050:1674088250:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:ruly-Inspiron-5558:1:1:1906:1906] [20230118:163050:1674088250:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:ruly-Inspiron-5558:1:1:1906:1906] [20230118:163050:1674088250:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:ruly-Inspiron-5558:1:1:1906:1906] [20230118:163050:1674088250:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:ruly-Inspiron-5558:1:1:1906:1906] [20230118:163050:1674088250:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:ruly-Inspiron-5558:1:1:1906:1906] [20230118:163050:1674088250:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:ruly-Inspiron-5558:1:1:1906:1906] [20230118:163050:1674088250:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:ruly-Inspiron-5558:1:1:1906:1906] [20230118:163050:1674088250:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:ruly-Inspiron-5558:1:1:1906:1906] [20230118:163050:1674088250:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ruly-Inspiron-5558:1:1:1906:1906] [20230118:163050:1674088250:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:ruly-Inspiron-5558:1:1:1906:1906] [20230118:163050:1674088250:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ruly-Inspiron-5558:1:1:1906:1906] [20230118:163050:1674088250:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:ruly-Inspiron-5558:1:1:1906:1906] [20230118:163050:1674088250:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:ruly-Inspiron-5558:1:1:1906:1906] [20230118:163050:1674088250:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ruly-Inspiron-5558:1:1:1906:1906] [20230118:163050:1674088250:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ruly-Inspiron-5558:1:1:1906:1906] [20230118:163050:1674088250:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:ruly-Inspiron-5558:1:1:1906:1906] [20230118:163050:1674088250:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ruly-Inspiron-5558:1:1:1906:1906] [20230118:163050:1674088250:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ruly-Inspiron-5558:1:1:1906:1906] [20230118:163050:1674088250:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ruly-Inspiron-5558:1:1:1906:1906] [20230118:163050:1674088250:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:ruly-Inspiron-5558:1:1:1906:1906] [20230118:163050:1674088250:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:ruly-Inspiron-5558:1:1:1906:1906] [20230118:163050:1674088250:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:ruly-Inspiron-5558:1:1:1906:1906] [20230118:163050:1674088250:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  16:31:40 STEP  840 H= 840:00 +CPU= 45.877
  16:32:03 STEP  841 H= 841:00 +CPU= 22.213
  16:32:31 STEP  842 H= 842:00 +CPU= 26.763
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:33:30 STEP  843 H= 843:00 +CPU= 44.744
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:35:20 STEP  844 H= 844:00 +CPU= 33.005
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:38:00 STEP  845 H= 845:00 +CPU= 35.363
  16:38:49 STEP  846 H= 846:00 +CPU= 45.395
  16:39:22 STEP  847 H= 847:00 +CPU= 29.406
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:40:16 STEP  848 H= 848:00 +CPU= 30.359
  16:41:05 STEP  849 H= 849:00 +CPU= 44.982
  16:41:33 STEP  850 H= 850:00 +CPU= 26.983
  16:42:07 STEP  851 H= 851:00 +CPU= 30.309
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:43:37 STEP  852 H= 852:00 +CPU= 51.180
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000852
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000852
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000852
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:44:25 STEP  853 H= 853:00 +CPU= 32.720
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:45:07 STEP  854 H= 854:00 +CPU= 29.115
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:46:13 STEP  855 H= 855:00 +CPU= 48.174
  16:46:46 STEP  856 H= 856:00 +CPU= 31.577
  16:47:12 STEP  857 H= 857:00 +CPU= 24.929
  16:47:56 STEP  858 H= 858:00 +CPU= 42.433
  16:48:24 STEP  859 H= 859:00 +CPU= 27.289
  16:48:52 STEP  860 H= 860:00 +CPU= 26.172
  16:49:40 STEP  861 H= 861:00 +CPU= 44.187
  16:50:06 STEP  862 H= 862:00 +CPU= 26.042
  16:50:35 STEP  863 H= 863:00 +CPU= 28.115
Abort request received from BOINC client, ending the child process
..Failed, model did not complete successfully

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 May 2023 16:21:45 1509518 22263571 oifs_43r3_ps_0053_1993050100_123_962_12178697_0 1,728,000 19,675 0.0114


©2024 cpdn.org