climateprediction.net home page
Task 22246890

Task 22246890

Name oifs_43r3_ps_1797_2021050100_123_946_12164886_0
Workunit 12164886
Created 29 Nov 2022, 12:19:58 UTC
Sent 29 Nov 2022, 12:30:33 UTC
Report deadline 29 Dec 2022, 12:30:33 UTC
Received 18 Mar 2023, 7:46:32 UTC
Server state Over
Outcome Computation error
Client state Aborted by user
Exit status 203 (0x000000CB) EXIT_ABORTED_VIA_GUI
Computer ID 1524992
Run time 14 hours 29 min 56 sec
CPU time 41 min 45 sec
Validate state Invalid
Credit 11,370.24
Device peak FLOPS 6.24 GFLOPS
Application version OpenIFS 43r3 Perturbed Surface v1.01
x86_64-pc-linux-gnu
Peak working set size 4,779.25 MB
Peak swap size 4,978.08 MB
Peak disk usage 8,684.41 MB
Stderr
<core_client_version>7.22.0</core_client_version>
<![CDATA[
<message>
aborted by user</message>
<stderr_txt>
846:00 +CPU= 17.603
  19:16:51 STEP 1847 H=1847:00 +CPU= 19.029
  19:17:29 STEP 1848 H=1848:00 +CPU= 32.199
Moving to projects directory: /var/lib/boinc/slots/47/ICMGGhpi1+001848
Moving to projects directory: /var/lib/boinc/slots/47/ICMSHhpi1+001848
Moving to projects directory: /var/lib/boinc/slots/47/ICMUAhpi1+001848
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMGGhpi1+001824
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMSHhpi1+001824
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMUAhpi1+001824
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMGGhpi1+001836
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMSHhpi1+001836
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMUAhpi1+001836
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_1797_2021050100_123_946_12164886_0_r985316780_13.zip
  19:17:46 STEP 1849 H=1849:00 +CPU= 16.979
Uploading the intermediate file: upload_file_13.zip
  19:18:05 STEP 1850 H=1850:00 +CPU= 18.933
Uploading trickle at timestep: 6652800
  19:18:32 STEP 1851 H=1851:00 +CPU= 27.259
  19:18:51 STEP 1852 H=1852:00 +CPU= 18.384
  19:19:09 STEP 1853 H=1853:00 +CPU= 18.410
  19:19:39 STEP 1854 H=1854:00 +CPU= 29.498
  19:19:57 STEP 1855 H=1855:00 +CPU= 17.458
  19:20:14 STEP 1856 H=1856:00 +CPU= 17.580
  19:20:44 STEP 1857 H=1857:00 +CPU= 29.332
  19:21:03 STEP 1858 H=1858:00 +CPU= 18.609
  19:21:20 STEP 1859 H=1859:00 +CPU= 16.663
  19:21:51 STEP 1860 H=1860:00 +CPU= 30.665
Moving to projects directory: /var/lib/boinc/slots/47/ICMGGhpi1+001860
Moving to projects directory: /var/lib/boinc/slots/47/ICMSHhpi1+001860
Moving to projects directory: /var/lib/boinc/slots/47/ICMUAhpi1+001860
  19:22:09 STEP 1861 H=1861:00 +CPU= 18.037
  19:22:27 STEP 1862 H=1862:00 +CPU= 17.947
  19:22:54 STEP 1863 H=1863:00 +CPU= 26.655
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.01_x86_64-pc-linux-gnu
(argv1) start_date: 2021050100
(argv2) exptid: hpi1
(argv3) unique_member_id: 1797
(argv4) batchid: 946
(argv5) wuid: 12164886
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc/slots/47
Project directory is: /var/lib/boinc/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.01
Location of temp folder: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886
..mkdir for temp folder for results failed
Copying: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_app_1.01_x86_64-pc-linux-gnu.zip to: /var/lib/boinc/slots/47/oifs_43r3_ps_app_1.01_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc/slots/47/oifs_43r3_ps_app_1.01_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_3659bad2faf61568334f51b9a9d6e564 to: /var/lib/boinc/slots/47/oifs_43r3_ps_1797_2021050100_123_946_12164886.zip
Unzipping the namelist zip file: /var/lib/boinc/slots/47/oifs_43r3_ps_1797_2021050100_123_946_12164886.zip
ic_ancil_file: ic_ancil_12164886
ifsdata_file: ifsdata_12164886
climate_data_file: clim_data_12164886
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
Copying IC ancils from: ../../projects/climateprediction.net/jf_43854f33bbd5e4731ac504454f342a2c to: /var/lib/boinc/slots/47/ic_ancil_12164886.zip
Unzipping the IC ancils zip file: /var/lib/boinc/slots/47/ic_ancil_12164886.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_ecd5050cc3ea7ff9cff6d96bf8917ef9 to: /var/lib/boinc/slots/47/ifsdata/ifsdata_12164886.zip
Unzipping the ifsdata_zip file: /var/lib/boinc/slots/47/ifsdata/ifsdata_12164886.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc/slots/47/159l_2/clim_data_12164886.zip
Unzipping the climate data zip file: /var/lib/boinc/slots/47/159l_2/clim_data_12164886.zip
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_1797_2021050100_123_946_12164886_0_r985316780
The child process has been launched with process id: 271991
Executing the command: /var/lib/boinc/slots/47/./master.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:guoj-desktop:1:1:271991:271991] [20221203:192451:1670120691:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:guoj-desktop:1:1:271991:271991] [20221203:192451:1670120691:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:guoj-desktop:1:1:271991:271991] [20221203:192451:1670120691:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:guoj-desktop:1:1:271991:271991] [20221203:192451:1670120691:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:guoj-desktop:1:1:271991:271991] [20221203:192451:1670120691:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:guoj-desktop:1:1:271991:271991] [20221203:192451:1670120691:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:guoj-desktop:1:1:271991:271991] [20221203:192451:1670120691:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:guoj-desktop:1:1:271991:271991] [20221203:192451:1670120691:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:guoj-desktop:1:1:271991:271991] [20221203:192451:1670120691:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:guoj-desktop:1:1:271991:271991] [20221203:192451:1670120691:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:guoj-desktop:1:1:271991:271991] [20221203:192451:1670120691:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:guoj-desktop:1:1:271991:271991] [20221203:192451:1670120691:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:guoj-desktop:1:1:271991:271991] [20221203:192451:1670120691:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:guoj-desktop:1:1:271991:271991] [20221203:192451:1670120691:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:guoj-desktop:1:1:271991:271991] [20221203:192451:1670120691:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:guoj-desktop:1:1:271991:271991] [20221203:192451:1670120691:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:guoj-desktop:1:1:271991:271991] [20221203:192451:1670120691:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:guoj-desktop:1:1:271991:271991] [20221203:192451:1670120691:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:guoj-desktop:1:1:271991:271991] [20221203:192451:1670120691:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:guoj-desktop:1:1:271991:271991] [20221203:192451:1670120691:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:guoj-desktop:1:1:271991:271991] [20221203:192451:1670120691:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:guoj-desktop:1:1:271991:271991] [20221203:192451:1670120691:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:guoj-desktop:1:1:271991:271991] [20221203:192451:1670120691:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:guoj-desktop:1:1:271991:271991] [20221203:192451:1670120691:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:guoj-desktop:1:1:271991:271991] [20221203:192451:1670120691:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:guoj-desktop:1:1:271991:271991] [20221203:192451:1670120691:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:guoj-desktop:1:1:271991:271991] [20221203:192451:1670120691:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:guoj-desktop:1:1:271991:271991] [20221203:192451:1670120691:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:guoj-desktop:1:1:271991:271991] [20221203:192451:1670120691:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:guoj-desktop:1:1:271991:271991] [20221203:192451:1670120691:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  19:26:50 STEP 1848 H=1848:00 +CPU= 31.473
  19:27:07 STEP 1849 H=1849:00 +CPU= 17.727
  19:27:24 STEP 1850 H=1850:00 +CPU= 16.719
  19:27:52 STEP 1851 H=1851:00 +CPU= 27.504
  19:28:10 STEP 1852 H=1852:00 +CPU= 18.056
  19:28:27 STEP 1853 H=1853:00 +CPU= 17.220
  19:28:55 STEP 1854 H=1854:00 +CPU= 27.810
  19:29:11 STEP 1855 H=1855:00 +CPU= 15.508
  19:29:27 STEP 1856 H=1856:00 +CPU= 15.583
  19:29:55 STEP 1857 H=1857:00 +CPU= 27.752
  19:30:13 STEP 1858 H=1858:00 +CPU= 18.435
  19:30:29 STEP 1859 H=1859:00 +CPU= 15.820
  19:31:05 STEP 1860 H=1860:00 +CPU= 31.208
Moving to projects directory: /var/lib/boinc/slots/47/ICMGGhpi1+001860
Moving to projects directory: /var/lib/boinc/slots/47/ICMSHhpi1+001860
Moving to projects directory: /var/lib/boinc/slots/47/ICMUAhpi1+001860
  19:31:22 STEP 1861 H=1861:00 +CPU= 17.290
  19:31:40 STEP 1862 H=1862:00 +CPU= 17.700
  19:32:07 STEP 1863 H=1863:00 +CPU= 26.987
  19:32:26 STEP 1864 H=1864:00 +CPU= 18.546
  19:32:43 STEP 1865 H=1865:00 +CPU= 17.482
  19:33:14 STEP 1866 H=1866:00 +CPU= 29.407
  19:33:31 STEP 1867 H=1867:00 +CPU= 17.061
  19:33:48 STEP 1868 H=1868:00 +CPU= 16.939
  19:34:16 STEP 1869 H=1869:00 +CPU= 27.981
  19:34:35 STEP 1870 H=1870:00 +CPU= 18.772
  19:34:59 STEP 1871 H=1871:00 +CPU= 19.529
  19:35:34 STEP 1872 H=1872:00 +CPU= 31.468
Moving to projects directory: /var/lib/boinc/slots/47/ICMGGhpi1+001872
Moving to projects directory: /var/lib/boinc/slots/47/ICMSHhpi1+001872
Moving to projects directory: /var/lib/boinc/slots/47/ICMUAhpi1+001872
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMGGhpi1+001848
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMSHhpi1+001848
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMUAhpi1+001848
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMGGhpi1+001860
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMSHhpi1+001860
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMUAhpi1+001860
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_1797_2021050100_123_946_12164886_0_r985316780_1.zip
  19:35:52 STEP 1873 H=1873:00 +CPU= 17.706
Uploading the intermediate file: upload_file_1.zip
  19:36:09 STEP 1874 H=1874:00 +CPU= 17.410
Uploading trickle at timestep: 6739200
  19:36:36 STEP 1875 H=1875:00 +CPU= 26.515
  19:36:54 STEP 1876 H=1876:00 +CPU= 17.468
  19:37:11 STEP 1877 H=1877:00 +CPU= 17.253
  19:37:40 STEP 1878 H=1878:00 +CPU= 28.294
  19:37:58 STEP 1879 H=1879:00 +CPU= 17.723
  19:38:16 STEP 1880 H=1880:00 +CPU= 16.956
  19:38:42 STEP 1881 H=1881:00 +CPU= 26.767
  19:39:00 STEP 1882 H=1882:00 +CPU= 17.663
  19:39:16 STEP 1883 H=1883:00 +CPU= 15.812
  19:39:48 STEP 1884 H=1884:00 +CPU= 31.443
Moving to projects directory: /var/lib/boinc/slots/47/ICMGGhpi1+001884
Moving to projects directory: /var/lib/boinc/slots/47/ICMSHhpi1+001884
Moving to projects directory: /var/lib/boinc/slots/47/ICMUAhpi1+001884
  19:40:06 STEP 1885 H=1885:00 +CPU= 17.244
  19:40:23 STEP 1886 H=1886:00 +CPU= 17.637
  19:40:51 STEP 1887 H=1887:00 +CPU= 26.885
  19:41:09 STEP 1888 H=1888:00 +CPU= 17.912
  19:41:26 STEP 1889 H=1889:00 +CPU= 16.902
  19:41:55 STEP 1890 H=1890:00 +CPU= 28.553
  19:42:13 STEP 1891 H=1891:00 +CPU= 17.719
  19:42:29 STEP 1892 H=1892:00 +CPU= 16.673
  19:42:56 STEP 1893 H=1893:00 +CPU= 26.787
  19:43:14 STEP 1894 H=1894:00 +CPU= 17.550
  19:43:33 STEP 1895 H=1895:00 +CPU= 18.729
  19:44:05 STEP 1896 H=1896:00 +CPU= 31.863
Moving to projects directory: /var/lib/boinc/slots/47/ICMGGhpi1+001896
Moving to projects directory: /var/lib/boinc/slots/47/ICMSHhpi1+001896
Moving to projects directory: /var/lib/boinc/slots/47/ICMUAhpi1+001896
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMGGhpi1+001872
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMSHhpi1+001872
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMUAhpi1+001872
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMGGhpi1+001884
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMSHhpi1+001884
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMUAhpi1+001884
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_1797_2021050100_123_946_12164886_0_r985316780_2.zip
Uploading the intermediate file: upload_file_2.zip
  19:44:23 STEP 1897 H=1897:00 +CPU= 17.625
Uploading trickle at timestep: 6825600
  19:44:41 STEP 1898 H=1898:00 +CPU= 17.805
  19:45:09 STEP 1899 H=1899:00 +CPU= 27.402
  19:45:26 STEP 1900 H=1900:00 +CPU= 17.600
  19:45:42 STEP 1901 H=1901:00 +CPU= 15.847
  19:46:09 STEP 1902 H=1902:00 +CPU= 26.777
  19:46:27 STEP 1903 H=1903:00 +CPU= 17.275
  19:46:43 STEP 1904 H=1904:00 +CPU= 15.941
  19:47:09 STEP 1905 H=1905:00 +CPU= 25.953
  19:47:28 STEP 1906 H=1906:00 +CPU= 17.573
  19:47:43 STEP 1907 H=1907:00 +CPU= 15.769
  19:48:15 STEP 1908 H=1908:00 +CPU= 29.617
Moving to projects directory: /var/lib/boinc/slots/47/ICMGGhpi1+001908
Moving to projects directory: /var/lib/boinc/slots/47/ICMSHhpi1+001908
Moving to projects directory: /var/lib/boinc/slots/47/ICMUAhpi1+001908
  19:48:31 STEP 1909 H=1909:00 +CPU= 16.584
  19:48:50 STEP 1910 H=1910:00 +CPU= 18.030
  19:49:17 STEP 1911 H=1911:00 +CPU= 26.661
  19:49:34 STEP 1912 H=1912:00 +CPU= 16.918
  19:49:51 STEP 1913 H=1913:00 +CPU= 17.389
  19:50:20 STEP 1914 H=1914:00 +CPU= 28.598
  19:50:37 STEP 1915 H=1915:00 +CPU= 16.376
  19:50:54 STEP 1916 H=1916:00 +CPU= 17.857
  19:51:24 STEP 1917 H=1917:00 +CPU= 28.965
  19:51:42 STEP 1918 H=1918:00 +CPU= 17.876
  19:51:58 STEP 1919 H=1919:00 +CPU= 16.465
  19:52:33 STEP 1920 H=1920:00 +CPU= 32.432
Moving to projects directory: /var/lib/boinc/slots/47/ICMGGhpi1+001920
Moving to projects directory: /var/lib/boinc/slots/47/ICMSHhpi1+001920
Moving to projects directory: /var/lib/boinc/slots/47/ICMUAhpi1+001920
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMGGhpi1+001896
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMSHhpi1+001896
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMUAhpi1+001896
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMGGhpi1+001908
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMSHhpi1+001908
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMUAhpi1+001908
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_1797_2021050100_123_946_12164886_0_r985316780_3.zip
  19:52:51 STEP 1921 H=1921:00 +CPU= 18.299
Uploading the intermediate file: upload_file_3.zip
  19:53:09 STEP 1922 H=1922:00 +CPU= 17.283
Uploading trickle at timestep: 6912000
  19:53:35 STEP 1923 H=1923:00 +CPU= 26.448
  19:53:53 STEP 1924 H=1924:00 +CPU= 17.452
  19:54:11 STEP 1925 H=1925:00 +CPU= 18.177
  19:54:35 STEP 1926 H=1926:00 +CPU= 23.508
  19:54:49 STEP 1927 H=1927:00 +CPU= 13.128
  19:55:05 STEP 1928 H=1928:00 +CPU= 16.253
  19:55:33 STEP 1929 H=1929:00 +CPU= 27.562
  19:55:52 STEP 1930 H=1930:00 +CPU= 19.133
  19:56:09 STEP 1931 H=1931:00 +CPU= 16.894
  19:56:40 STEP 1932 H=1932:00 +CPU= 30.518
Moving to projects directory: /var/lib/boinc/slots/47/ICMGGhpi1+001932
Moving to projects directory: /var/lib/boinc/slots/47/ICMSHhpi1+001932
Moving to projects directory: /var/lib/boinc/slots/47/ICMUAhpi1+001932
  19:56:58 STEP 1933 H=1933:00 +CPU= 18.031
  19:57:16 STEP 1934 H=1934:00 +CPU= 17.694
  19:57:42 STEP 1935 H=1935:00 +CPU= 26.177
  19:57:58 STEP 1936 H=1936:00 +CPU= 16.101
  19:58:16 STEP 1937 H=1937:00 +CPU= 16.876
  19:58:42 STEP 1938 H=1938:00 +CPU= 26.406
  19:59:00 STEP 1939 H=1939:00 +CPU= 17.452
  19:59:17 STEP 1940 H=1940:00 +CPU= 16.801
  19:59:44 STEP 1941 H=1941:00 +CPU= 27.202
  20:00:01 STEP 1942 H=1942:00 +CPU= 16.784
  20:00:18 STEP 1943 H=1943:00 +CPU= 17.377
  20:00:52 STEP 1944 H=1944:00 +CPU= 30.725
Moving to projects directory: /var/lib/boinc/slots/47/ICMGGhpi1+001944
Moving to projects directory: /var/lib/boinc/slots/47/ICMSHhpi1+001944
Moving to projects directory: /var/lib/boinc/slots/47/ICMUAhpi1+001944
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMGGhpi1+001920
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMSHhpi1+001920
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMUAhpi1+001920
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMGGhpi1+001932
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMSHhpi1+001932
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMUAhpi1+001932
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_1797_2021050100_123_946_12164886_0_r985316780_4.zip
Uploading the intermediate file: upload_file_4.zip
  20:01:08 STEP 1945 H=1945:00 +CPU= 16.469
Uploading trickle at timestep: 6998400
  20:01:25 STEP 1946 H=1946:00 +CPU= 16.748
  20:01:52 STEP 1947 H=1947:00 +CPU= 26.965
  20:02:10 STEP 1948 H=1948:00 +CPU= 18.051
  20:02:26 STEP 1949 H=1949:00 +CPU= 15.564
  20:02:53 STEP 1950 H=1950:00 +CPU= 26.877
  20:03:10 STEP 1951 H=1951:00 +CPU= 16.734
  20:03:28 STEP 1952 H=1952:00 +CPU= 17.752
  20:03:55 STEP 1953 H=1953:00 +CPU= 27.680
  20:04:13 STEP 1954 H=1954:00 +CPU= 17.250
  20:04:29 STEP 1955 H=1955:00 +CPU= 15.664
  20:04:57 STEP 1956 H=1956:00 +CPU= 28.496
Moving to projects directory: /var/lib/boinc/slots/47/ICMGGhpi1+001956
Moving to projects directory: /var/lib/boinc/slots/47/ICMSHhpi1+001956
Moving to projects directory: /var/lib/boinc/slots/47/ICMUAhpi1+001956
  20:05:13 STEP 1957 H=1957:00 +CPU= 15.453
  20:05:29 STEP 1958 H=1958:00 +CPU= 16.715
  20:05:56 STEP 1959 H=1959:00 +CPU= 26.595
  20:06:14 STEP 1960 H=1960:00 +CPU= 17.570
  20:06:32 STEP 1961 H=1961:00 +CPU= 17.561
  20:07:00 STEP 1962 H=1962:00 +CPU= 28.038
  20:07:17 STEP 1963 H=1963:00 +CPU= 17.245
  20:07:35 STEP 1964 H=1964:00 +CPU= 17.834
  20:08:03 STEP 1965 H=1965:00 +CPU= 27.332
  20:08:20 STEP 1966 H=1966:00 +CPU= 17.413
  20:08:36 STEP 1967 H=1967:00 +CPU= 15.585
  20:09:09 STEP 1968 H=1968:00 +CPU= 32.233
Moving to projects directory: /var/lib/boinc/slots/47/ICMGGhpi1+001968
Moving to projects directory: /var/lib/boinc/slots/47/ICMSHhpi1+001968
Moving to projects directory: /var/lib/boinc/slots/47/ICMUAhpi1+001968
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMGGhpi1+001944
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMSHhpi1+001944
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMUAhpi1+001944
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMGGhpi1+001956
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMSHhpi1+001956
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMUAhpi1+001956
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_1797_2021050100_123_946_12164886_0_r985316780_5.zip
Uploading the intermediate file: upload_file_5.zip
  20:09:25 STEP 1969 H=1969:00 +CPU= 16.598
  20:09:39 STEP 1970 H=1970:00 +CPU= 14.093
Uploading trickle at timestep: 7084800
  20:10:04 STEP 1971 H=1971:00 +CPU= 24.463
  20:10:21 STEP 1972 H=1972:00 +CPU= 17.000
  20:10:37 STEP 1973 H=1973:00 +CPU= 16.040
  20:11:04 STEP 1974 H=1974:00 +CPU= 26.925
  20:11:21 STEP 1975 H=1975:00 +CPU= 16.857
  20:11:39 STEP 1976 H=1976:00 +CPU= 18.016
  20:12:06 STEP 1977 H=1977:00 +CPU= 26.367
  20:12:22 STEP 1978 H=1978:00 +CPU= 16.142
  20:12:38 STEP 1979 H=1979:00 +CPU= 15.710
  20:13:08 STEP 1980 H=1980:00 +CPU= 30.610
Moving to projects directory: /var/lib/boinc/slots/47/ICMGGhpi1+001980
Moving to projects directory: /var/lib/boinc/slots/47/ICMSHhpi1+001980
Moving to projects directory: /var/lib/boinc/slots/47/ICMUAhpi1+001980
  20:13:24 STEP 1981 H=1981:00 +CPU= 15.545
  20:13:39 STEP 1982 H=1982:00 +CPU= 14.846
  20:14:03 STEP 1983 H=1983:00 +CPU= 23.731
  20:14:20 STEP 1984 H=1984:00 +CPU= 17.004
  20:14:38 STEP 1985 H=1985:00 +CPU= 17.119
  20:15:07 STEP 1986 H=1986:00 +CPU= 28.723
  20:15:24 STEP 1987 H=1987:00 +CPU= 17.711
  20:15:42 STEP 1988 H=1988:00 +CPU= 17.268
  20:16:09 STEP 1989 H=1989:00 +CPU= 27.345
  20:16:24 STEP 1990 H=1990:00 +CPU= 15.154
  20:16:42 STEP 1991 H=1991:00 +CPU= 17.416
  20:17:15 STEP 1992 H=1992:00 +CPU= 32.180
Moving to projects directory: /var/lib/boinc/slots/47/ICMGGhpi1+001992
Moving to projects directory: /var/lib/boinc/slots/47/ICMSHhpi1+001992
Moving to projects directory: /var/lib/boinc/slots/47/ICMUAhpi1+001992
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMGGhpi1+001968
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMSHhpi1+001968
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMUAhpi1+001968
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMGGhpi1+001980
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMSHhpi1+001980
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMUAhpi1+001980
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_1797_2021050100_123_946_12164886_0_r985316780_6.zip
Uploading the intermediate file: upload_file_6.zip
  20:17:33 STEP 1993 H=1993:00 +CPU= 17.524
Uploading trickle at timestep: 7171200
  20:17:50 STEP 1994 H=1994:00 +CPU= 17.285
  20:18:18 STEP 1995 H=1995:00 +CPU= 27.672
  20:18:36 STEP 1996 H=1996:00 +CPU= 18.241
  20:18:53 STEP 1997 H=1997:00 +CPU= 16.937
  20:19:22 STEP 1998 H=1998:00 +CPU= 28.380
  20:19:38 STEP 1999 H=1999:00 +CPU= 16.563
  20:19:55 STEP 2000 H=2000:00 +CPU= 16.974
  20:20:24 STEP 2001 H=2001:00 +CPU= 28.804
  20:20:42 STEP 2002 H=2002:00 +CPU= 17.472
  20:20:58 STEP 2003 H=2003:00 +CPU= 15.839
  20:21:26 STEP 2004 H=2004:00 +CPU= 28.160
Moving to projects directory: /var/lib/boinc/slots/47/ICMGGhpi1+002004
Moving to projects directory: /var/lib/boinc/slots/47/ICMSHhpi1+002004
Moving to projects directory: /var/lib/boinc/slots/47/ICMUAhpi1+002004
  20:21:44 STEP 2005 H=2005:00 +CPU= 17.325
  20:22:02 STEP 2006 H=2006:00 +CPU= 18.185
  20:22:29 STEP 2007 H=2007:00 +CPU= 27.126
  20:22:46 STEP 2008 H=2008:00 +CPU= 16.640
  20:23:03 STEP 2009 H=2009:00 +CPU= 16.691
  20:23:31 STEP 2010 H=2010:00 +CPU= 27.499
  20:23:47 STEP 2011 H=2011:00 +CPU= 15.412
  20:24:03 STEP 2012 H=2012:00 +CPU= 16.055
  20:24:31 STEP 2013 H=2013:00 +CPU= 27.751
  20:24:49 STEP 2014 H=2014:00 +CPU= 17.490
  20:25:06 STEP 2015 H=2015:00 +CPU= 17.206
  20:25:40 STEP 2016 H=2016:00 +CPU= 32.062
Moving to projects directory: /var/lib/boinc/slots/47/ICMGGhpi1+002016
Moving to projects directory: /var/lib/boinc/slots/47/ICMSHhpi1+002016
Moving to projects directory: /var/lib/boinc/slots/47/ICMUAhpi1+002016
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMGGhpi1+001992
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMSHhpi1+001992
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMUAhpi1+001992
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMGGhpi1+002004
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMSHhpi1+002004
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMUAhpi1+002004
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_1797_2021050100_123_946_12164886_0_r985316780_7.zip
Uploading the intermediate file: upload_file_7.zip
  20:25:58 STEP 2017 H=2017:00 +CPU= 17.831
Uploading trickle at timestep: 7257600
  20:26:15 STEP 2018 H=2018:00 +CPU= 16.925
  20:26:43 STEP 2019 H=2019:00 +CPU= 28.140
  20:27:01 STEP 2020 H=2020:00 +CPU= 17.999
  20:27:18 STEP 2021 H=2021:00 +CPU= 16.887
  20:27:48 STEP 2022 H=2022:00 +CPU= 29.070
  20:28:06 STEP 2023 H=2023:00 +CPU= 18.036
  20:28:22 STEP 2024 H=2024:00 +CPU= 15.879
  20:28:49 STEP 2025 H=2025:00 +CPU= 27.137
  20:29:06 STEP 2026 H=2026:00 +CPU= 17.016
  20:29:23 STEP 2027 H=2027:00 +CPU= 16.664
  20:29:54 STEP 2028 H=2028:00 +CPU= 29.221
Moving to projects directory: /var/lib/boinc/slots/47/ICMGGhpi1+002028
Moving to projects directory: /var/lib/boinc/slots/47/ICMSHhpi1+002028
Moving to projects directory: /var/lib/boinc/slots/47/ICMUAhpi1+002028
  20:30:10 STEP 2029 H=2029:00 +CPU= 15.982
  20:30:28 STEP 2030 H=2030:00 +CPU= 17.900
  20:30:55 STEP 2031 H=2031:00 +CPU= 27.068
  20:31:13 STEP 2032 H=2032:00 +CPU= 17.684
  20:31:29 STEP 2033 H=2033:00 +CPU= 15.774
  20:31:54 STEP 2034 H=2034:00 +CPU= 25.283
  20:32:11 STEP 2035 H=2035:00 +CPU= 16.600
  20:32:26 STEP 2036 H=2036:00 +CPU= 15.402
  20:32:55 STEP 2037 H=2037:00 +CPU= 28.619
  20:33:13 STEP 2038 H=2038:00 +CPU= 17.832
  20:33:30 STEP 2039 H=2039:00 +CPU= 17.354
  20:34:07 STEP 2040 H=2040:00 +CPU= 32.923
Moving to projects directory: /var/lib/boinc/slots/47/ICMGGhpi1+002040
Moving to projects directory: /var/lib/boinc/slots/47/ICMSHhpi1+002040
Moving to projects directory: /var/lib/boinc/slots/47/ICMUAhpi1+002040
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMGGhpi1+002016
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMSHhpi1+002016
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMUAhpi1+002016
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMGGhpi1+002028
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMSHhpi1+002028
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMUAhpi1+002028
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_1797_2021050100_123_946_12164886_0_r985316780_8.zip
  20:34:24 STEP 2041 H=2041:00 +CPU= 17.407
Uploading the intermediate file: upload_file_8.zip
  20:34:42 STEP 2042 H=2042:00 +CPU= 17.690
Uploading trickle at timestep: 7344000
  20:35:11 STEP 2043 H=2043:00 +CPU= 28.132
  20:35:29 STEP 2044 H=2044:00 +CPU= 18.435
  20:35:47 STEP 2045 H=2045:00 +CPU= 17.282
  20:36:17 STEP 2046 H=2046:00 +CPU= 29.387
  20:36:36 STEP 2047 H=2047:00 +CPU= 18.220
  20:36:54 STEP 2048 H=2048:00 +CPU= 17.949
  20:37:21 STEP 2049 H=2049:00 +CPU= 27.320
  20:37:39 STEP 2050 H=2050:00 +CPU= 17.859
  20:37:56 STEP 2051 H=2051:00 +CPU= 16.970
  20:38:28 STEP 2052 H=2052:00 +CPU= 31.445
Moving to projects directory: /var/lib/boinc/slots/47/ICMGGhpi1+002052
Moving to projects directory: /var/lib/boinc/slots/47/ICMSHhpi1+002052
Moving to projects directory: /var/lib/boinc/slots/47/ICMUAhpi1+002052
  20:38:45 STEP 2053 H=2053:00 +CPU= 16.507
  20:39:00 STEP 2054 H=2054:00 +CPU= 15.205
  20:39:26 STEP 2055 H=2055:00 +CPU= 26.129
  20:39:44 STEP 2056 H=2056:00 +CPU= 17.702
  20:40:02 STEP 2057 H=2057:00 +CPU= 17.533
  20:40:30 STEP 2058 H=2058:00 +CPU= 27.979
  20:40:47 STEP 2059 H=2059:00 +CPU= 17.574
  20:41:05 STEP 2060 H=2060:00 +CPU= 17.960
  20:41:33 STEP 2061 H=2061:00 +CPU= 27.672
  20:41:51 STEP 2062 H=2062:00 +CPU= 17.691
  20:42:10 STEP 2063 H=2063:00 +CPU= 17.664
  20:43:22 STEP 2064 H=2064:00 +CPU= 33.287
Moving to projects directory: /var/lib/boinc/slots/47/ICMGGhpi1+002064
Moving to projects directory: /var/lib/boinc/slots/47/ICMSHhpi1+002064
Moving to projects directory: /var/lib/boinc/slots/47/ICMUAhpi1+002064
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMGGhpi1+002040
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMSHhpi1+002040
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMUAhpi1+002040
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMGGhpi1+002052
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMSHhpi1+002052
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMUAhpi1+002052
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_1797_2021050100_123_946_12164886_0_r985316780_9.zip
Uploading the intermediate file: upload_file_9.zip
  20:43:40 STEP 2065 H=2065:00 +CPU= 17.521
Uploading trickle at timestep: 7430400
  20:43:59 STEP 2066 H=2066:00 +CPU= 17.973
  20:44:27 STEP 2067 H=2067:00 +CPU= 27.990
  20:44:44 STEP 2068 H=2068:00 +CPU= 16.625
  20:45:00 STEP 2069 H=2069:00 +CPU= 16.079
  20:45:27 STEP 2070 H=2070:00 +CPU= 26.689
  20:45:42 STEP 2071 H=2071:00 +CPU= 15.364
  20:46:00 STEP 2072 H=2072:00 +CPU= 17.656
  20:46:27 STEP 2073 H=2073:00 +CPU= 26.166
  20:46:43 STEP 2074 H=2074:00 +CPU= 16.599
  20:47:00 STEP 2075 H=2075:00 +CPU= 17.066
  20:47:32 STEP 2076 H=2076:00 +CPU= 30.994
Moving to projects directory: /var/lib/boinc/slots/47/ICMGGhpi1+002076
Moving to projects directory: /var/lib/boinc/slots/47/ICMSHhpi1+002076
Moving to projects directory: /var/lib/boinc/slots/47/ICMUAhpi1+002076
  20:47:49 STEP 2077 H=2077:00 +CPU= 16.778
  20:48:05 STEP 2078 H=2078:00 +CPU= 16.318
  20:48:33 STEP 2079 H=2079:00 +CPU= 27.690
  20:48:52 STEP 2080 H=2080:00 +CPU= 19.004
  20:49:10 STEP 2081 H=2081:00 +CPU= 17.198
  20:49:35 STEP 2082 H=2082:00 +CPU= 24.976
  20:49:53 STEP 2083 H=2083:00 +CPU= 17.865
  20:50:11 STEP 2084 H=2084:00 +CPU= 18.006
  20:50:39 STEP 2085 H=2085:00 +CPU= 27.328
  20:50:57 STEP 2086 H=2086:00 +CPU= 18.134
  20:51:17 STEP 2087 H=2087:00 +CPU= 18.952
  20:52:04 STEP 2088 H=2088:00 +CPU= 28.241
Moving to projects directory: /var/lib/boinc/slots/47/ICMGGhpi1+002088
Moving to projects directory: /var/lib/boinc/slots/47/ICMSHhpi1+002088
Moving to projects directory: /var/lib/boinc/slots/47/ICMUAhpi1+002088
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMGGhpi1+002064
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMSHhpi1+002064
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMUAhpi1+002064
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMGGhpi1+002076
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMSHhpi1+002076
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMUAhpi1+002076
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_1797_2021050100_123_946_12164886_0_r985316780_10.zip
  20:52:22 STEP 2089 H=2089:00 +CPU= 17.915
Uploading the intermediate file: upload_file_10.zip
  20:52:40 STEP 2090 H=2090:00 +CPU= 18.169
Uploading trickle at timestep: 7516800
  20:53:07 STEP 2091 H=2091:00 +CPU= 26.897
  20:53:26 STEP 2092 H=2092:00 +CPU= 18.721
  20:53:44 STEP 2093 H=2093:00 +CPU= 17.452
  20:54:11 STEP 2094 H=2094:00 +CPU= 27.576
  20:54:29 STEP 2095 H=2095:00 +CPU= 17.491
  20:54:45 STEP 2096 H=2096:00 +CPU= 16.106
  20:55:13 STEP 2097 H=2097:00 +CPU= 27.277
  20:55:29 STEP 2098 H=2098:00 +CPU= 16.545
  20:55:45 STEP 2099 H=2099:00 +CPU= 15.689
  20:56:16 STEP 2100 H=2100:00 +CPU= 30.535
Moving to projects directory: /var/lib/boinc/slots/47/ICMGGhpi1+002100
Moving to projects directory: /var/lib/boinc/slots/47/ICMSHhpi1+002100
Moving to projects directory: /var/lib/boinc/slots/47/ICMUAhpi1+002100
  20:56:34 STEP 2101 H=2101:00 +CPU= 18.335
  20:56:52 STEP 2102 H=2102:00 +CPU= 17.308
  20:57:19 STEP 2103 H=2103:00 +CPU= 27.459
  20:57:38 STEP 2104 H=2104:00 +CPU= 18.145
  20:57:53 STEP 2105 H=2105:00 +CPU= 15.013
  20:58:21 STEP 2106 H=2106:00 +CPU= 27.868
  20:58:38 STEP 2107 H=2107:00 +CPU= 16.771
  20:58:56 STEP 2108 H=2108:00 +CPU= 18.379
  20:59:25 STEP 2109 H=2109:00 +CPU= 28.432
  20:59:42 STEP 2110 H=2110:00 +CPU= 16.675
  21:00:00 STEP 2111 H=2111:00 +CPU= 17.867
  21:00:41 STEP 2112 H=2112:00 +CPU= 33.846
Moving to projects directory: /var/lib/boinc/slots/47/ICMGGhpi1+002112
Moving to projects directory: /var/lib/boinc/slots/47/ICMSHhpi1+002112
Moving to projects directory: /var/lib/boinc/slots/47/ICMUAhpi1+002112
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMGGhpi1+002088
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMSHhpi1+002088
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMUAhpi1+002088
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMGGhpi1+002100
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMSHhpi1+002100
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMUAhpi1+002100
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_1797_2021050100_123_946_12164886_0_r985316780_11.zip
  21:00:59 STEP 2113 H=2113:00 +CPU= 17.585
Uploading the intermediate file: upload_file_11.zip
  21:01:15 STEP 2114 H=2114:00 +CPU= 15.727
Uploading trickle at timestep: 7603200
  21:01:44 STEP 2115 H=2115:00 +CPU= 28.631
  21:02:01 STEP 2116 H=2116:00 +CPU= 17.600
  21:02:19 STEP 2117 H=2117:00 +CPU= 17.533
  21:02:48 STEP 2118 H=2118:00 +CPU= 28.460
  21:03:05 STEP 2119 H=2119:00 +CPU= 17.007
  21:03:22 STEP 2120 H=2120:00 +CPU= 16.828
  21:03:50 STEP 2121 H=2121:00 +CPU= 27.876
  21:04:08 STEP 2122 H=2122:00 +CPU= 18.258
  21:04:25 STEP 2123 H=2123:00 +CPU= 16.784
  21:04:54 STEP 2124 H=2124:00 +CPU= 29.040
Moving to projects directory: /var/lib/boinc/slots/47/ICMGGhpi1+002124
Moving to projects directory: /var/lib/boinc/slots/47/ICMSHhpi1+002124
Moving to projects directory: /var/lib/boinc/slots/47/ICMUAhpi1+002124
  21:05:10 STEP 2125 H=2125:00 +CPU= 16.112
  21:05:29 STEP 2126 H=2126:00 +CPU= 18.466
  21:05:56 STEP 2127 H=2127:00 +CPU= 26.712
  21:06:15 STEP 2128 H=2128:00 +CPU= 18.747
  21:06:32 STEP 2129 H=2129:00 +CPU= 17.691
  21:06:57 STEP 2130 H=2130:00 +CPU= 24.675
  21:07:14 STEP 2131 H=2131:00 +CPU= 16.751
  21:07:33 STEP 2132 H=2132:00 +CPU= 18.899
  21:08:01 STEP 2133 H=2133:00 +CPU= 27.830
  21:08:20 STEP 2134 H=2134:00 +CPU= 19.036
  21:08:38 STEP 2135 H=2135:00 +CPU= 17.425
  21:09:12 STEP 2136 H=2136:00 +CPU= 30.358
Moving to projects directory: /var/lib/boinc/slots/47/ICMGGhpi1+002136
Moving to projects directory: /var/lib/boinc/slots/47/ICMSHhpi1+002136
Moving to projects directory: /var/lib/boinc/slots/47/ICMUAhpi1+002136
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMGGhpi1+002112
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMSHhpi1+002112
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMUAhpi1+002112
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMGGhpi1+002124
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMSHhpi1+002124
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMUAhpi1+002124
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_1797_2021050100_123_946_12164886_0_r985316780_12.zip
  21:09:29 STEP 2137 H=2137:00 +CPU= 16.763
  21:09:47 STEP 2138 H=2138:00 +CPU= 18.055
Uploading the intermediate file: upload_file_12.zip
Uploading trickle at timestep: 7689600
  21:10:15 STEP 2139 H=2139:00 +CPU= 27.380
  21:10:34 STEP 2140 H=2140:00 +CPU= 18.879
  21:10:54 STEP 2141 H=2141:00 +CPU= 16.836
  21:11:23 STEP 2142 H=2142:00 +CPU= 28.762
  21:11:41 STEP 2143 H=2143:00 +CPU= 17.356
  21:11:59 STEP 2144 H=2144:00 +CPU= 17.683
  21:12:27 STEP 2145 H=2145:00 +CPU= 27.638
  21:12:46 STEP 2146 H=2146:00 +CPU= 18.788
  21:13:02 STEP 2147 H=2147:00 +CPU= 16.411
  21:13:33 STEP 2148 H=2148:00 +CPU= 30.865
Moving to projects directory: /var/lib/boinc/slots/47/ICMGGhpi1+002148
Moving to projects directory: /var/lib/boinc/slots/47/ICMSHhpi1+002148
Moving to projects directory: /var/lib/boinc/slots/47/ICMUAhpi1+002148
  21:13:51 STEP 2149 H=2149:00 +CPU= 17.973
  21:14:10 STEP 2150 H=2150:00 +CPU= 18.623
  21:14:37 STEP 2151 H=2151:00 +CPU= 27.133
  21:14:55 STEP 2152 H=2152:00 +CPU= 18.067
  21:15:13 STEP 2153 H=2153:00 +CPU= 17.080
  21:15:42 STEP 2154 H=2154:00 +CPU= 28.758
  21:15:59 STEP 2155 H=2155:00 +CPU= 17.131
  21:16:17 STEP 2156 H=2156:00 +CPU= 18.014
  21:16:46 STEP 2157 H=2157:00 +CPU= 28.474
  21:17:05 STEP 2158 H=2158:00 +CPU= 18.633
  21:17:23 STEP 2159 H=2159:00 +CPU= 18.602
  21:18:00 STEP 2160 H=2160:00 +CPU= 33.305
Moving to projects directory: /var/lib/boinc/slots/47/ICMGGhpi1+002160
Moving to projects directory: /var/lib/boinc/slots/47/ICMSHhpi1+002160
Moving to projects directory: /var/lib/boinc/slots/47/ICMUAhpi1+002160
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMGGhpi1+002136
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMSHhpi1+002136
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMUAhpi1+002136
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMGGhpi1+002148
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMSHhpi1+002148
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMUAhpi1+002148
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_1797_2021050100_123_946_12164886_0_r985316780_13.zip
  21:18:18 STEP 2161 H=2161:00 +CPU= 17.340
Uploading the intermediate file: upload_file_13.zip
  21:18:37 STEP 2162 H=2162:00 +CPU= 18.181
Uploading trickle at timestep: 7776000
  21:19:04 STEP 2163 H=2163:00 +CPU= 26.894
  21:19:23 STEP 2164 H=2164:00 +CPU= 18.621
  21:19:38 STEP 2165 H=2165:00 +CPU= 15.489
  21:20:07 STEP 2166 H=2166:00 +CPU= 28.633
  21:20:26 STEP 2167 H=2167:00 +CPU= 18.160
  21:20:44 STEP 2168 H=2168:00 +CPU= 17.982
  21:21:12 STEP 2169 H=2169:00 +CPU= 28.122
  21:21:30 STEP 2170 H=2170:00 +CPU= 17.422
  21:21:47 STEP 2171 H=2171:00 +CPU= 16.894
  21:22:19 STEP 2172 H=2172:00 +CPU= 31.946
Moving to projects directory: /var/lib/boinc/slots/47/ICMGGhpi1+002172
Moving to projects directory: /var/lib/boinc/slots/47/ICMSHhpi1+002172
Moving to projects directory: /var/lib/boinc/slots/47/ICMUAhpi1+002172
  21:22:37 STEP 2173 H=2173:00 +CPU= 17.868
  21:22:55 STEP 2174 H=2174:00 +CPU= 18.203
  21:23:22 STEP 2175 H=2175:00 +CPU= 27.148
  21:23:43 STEP 2176 H=2176:00 +CPU= 20.262
  21:24:01 STEP 2177 H=2177:00 +CPU= 18.531
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.01_x86_64-pc-linux-gnu
(argv1) start_date: 2021050100
(argv2) exptid: hpi1
(argv3) unique_member_id: 1797
(argv4) batchid: 946
(argv5) wuid: 12164886
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc/slots/47
Project directory is: /var/lib/boinc/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.01
Location of temp folder: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886
..mkdir for temp folder for results failed
Copying: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_app_1.01_x86_64-pc-linux-gnu.zip to: /var/lib/boinc/slots/47/oifs_43r3_ps_app_1.01_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc/slots/47/oifs_43r3_ps_app_1.01_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_3659bad2faf61568334f51b9a9d6e564 to: /var/lib/boinc/slots/47/oifs_43r3_ps_1797_2021050100_123_946_12164886.zip
Unzipping the namelist zip file: /var/lib/boinc/slots/47/oifs_43r3_ps_1797_2021050100_123_946_12164886.zip
ic_ancil_file: ic_ancil_12164886
ifsdata_file: ifsdata_12164886
climate_data_file: clim_data_12164886
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
Copying IC ancils from: ../../projects/climateprediction.net/jf_43854f33bbd5e4731ac504454f342a2c to: /var/lib/boinc/slots/47/ic_ancil_12164886.zip
Unzipping the IC ancils zip file: /var/lib/boinc/slots/47/ic_ancil_12164886.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_ecd5050cc3ea7ff9cff6d96bf8917ef9 to: /var/lib/boinc/slots/47/ifsdata/ifsdata_12164886.zip
Unzipping the ifsdata_zip file: /var/lib/boinc/slots/47/ifsdata/ifsdata_12164886.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc/slots/47/159l_2/clim_data_12164886.zip
Unzipping the climate data zip file: /var/lib/boinc/slots/47/159l_2/clim_data_12164886.zip
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_1797_2021050100_123_946_12164886_0_r985316780
The child process has been launched with process id: 275209
Executing the command: /var/lib/boinc/slots/47/./master.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:guoj-desktop:1:1:275209:275209] [20221203:212612:1670127972:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:guoj-desktop:1:1:275209:275209] [20221203:212612:1670127972:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:guoj-desktop:1:1:275209:275209] [20221203:212612:1670127972:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:guoj-desktop:1:1:275209:275209] [20221203:212612:1670127972:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:guoj-desktop:1:1:275209:275209] [20221203:212612:1670127972:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:guoj-desktop:1:1:275209:275209] [20221203:212612:1670127972:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:guoj-desktop:1:1:275209:275209] [20221203:212612:1670127972:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:guoj-desktop:1:1:275209:275209] [20221203:212612:1670127972:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:guoj-desktop:1:1:275209:275209] [20221203:212612:1670127972:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:guoj-desktop:1:1:275209:275209] [20221203:212612:1670127972:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:guoj-desktop:1:1:275209:275209] [20221203:212612:1670127972:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:guoj-desktop:1:1:275209:275209] [20221203:212612:1670127972:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:guoj-desktop:1:1:275209:275209] [20221203:212612:1670127972:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:guoj-desktop:1:1:275209:275209] [20221203:212612:1670127972:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:guoj-desktop:1:1:275209:275209] [20221203:212612:1670127972:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:guoj-desktop:1:1:275209:275209] [20221203:212612:1670127972:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:guoj-desktop:1:1:275209:275209] [20221203:212612:1670127972:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:guoj-desktop:1:1:275209:275209] [20221203:212612:1670127972:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:guoj-desktop:1:1:275209:275209] [20221203:212612:1670127972:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:guoj-desktop:1:1:275209:275209] [20221203:212612:1670127972:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:guoj-desktop:1:1:275209:275209] [20221203:212612:1670127972:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:guoj-desktop:1:1:275209:275209] [20221203:212612:1670127972:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:guoj-desktop:1:1:275209:275209] [20221203:212612:1670127972:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:guoj-desktop:1:1:275209:275209] [20221203:212612:1670127972:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:guoj-desktop:1:1:275209:275209] [20221203:212612:1670127972:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:guoj-desktop:1:1:275209:275209] [20221203:212612:1670127972:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:guoj-desktop:1:1:275209:275209] [20221203:212612:1670127972:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:guoj-desktop:1:1:275209:275209] [20221203:212612:1670127972:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:guoj-desktop:1:1:275209:275209] [20221203:212612:1670127972:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:guoj-desktop:1:1:275209:275209] [20221203:212612:1670127972:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  21:26:46 STEP 2160 H=2160:00 +CPU= 31.171
  21:27:03 STEP 2161 H=2161:00 +CPU= 16.857
  21:27:21 STEP 2162 H=2162:00 +CPU= 17.739
  21:27:50 STEP 2163 H=2163:00 +CPU= 28.421
  21:28:08 STEP 2164 H=2164:00 +CPU= 17.813
  21:28:25 STEP 2165 H=2165:00 +CPU= 16.740
  21:28:53 STEP 2166 H=2166:00 +CPU= 27.301
  21:29:11 STEP 2167 H=2167:00 +CPU= 17.516
  21:29:30 STEP 2168 H=2168:00 +CPU= 18.321
  21:29:56 STEP 2169 H=2169:00 +CPU= 26.153
  21:30:11 STEP 2170 H=2170:00 +CPU= 15.067
  21:30:28 STEP 2171 H=2171:00 +CPU= 16.694
  21:31:00 STEP 2172 H=2172:00 +CPU= 31.013
Moving to projects directory: /var/lib/boinc/slots/47/ICMGGhpi1+002172
Moving to projects directory: /var/lib/boinc/slots/47/ICMSHhpi1+002172
Moving to projects directory: /var/lib/boinc/slots/47/ICMUAhpi1+002172
  21:31:18 STEP 2173 H=2173:00 +CPU= 17.138
  21:31:36 STEP 2174 H=2174:00 +CPU= 17.585
  21:32:03 STEP 2175 H=2175:00 +CPU= 27.034
  21:32:21 STEP 2176 H=2176:00 +CPU= 17.630
  21:32:38 STEP 2177 H=2177:00 +CPU= 16.752
  21:33:06 STEP 2178 H=2178:00 +CPU= 27.351
  21:33:23 STEP 2179 H=2179:00 +CPU= 16.733
  21:33:40 STEP 2180 H=2180:00 +CPU= 16.971
  21:34:08 STEP 2181 H=2181:00 +CPU= 26.802
  21:34:25 STEP 2182 H=2182:00 +CPU= 17.268
  21:34:43 STEP 2183 H=2183:00 +CPU= 17.071
  21:35:15 STEP 2184 H=2184:00 +CPU= 30.964
Moving to projects directory: /var/lib/boinc/slots/47/ICMGGhpi1+002184
Moving to projects directory: /var/lib/boinc/slots/47/ICMSHhpi1+002184
Moving to projects directory: /var/lib/boinc/slots/47/ICMUAhpi1+002184
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMGGhpi1+002160
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMSHhpi1+002160
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMUAhpi1+002160
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMGGhpi1+002172
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMSHhpi1+002172
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMUAhpi1+002172
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_1797_2021050100_123_946_12164886_0_r985316780_1.zip
Uploading the intermediate file: upload_file_1.zip
  21:35:33 STEP 2185 H=2185:00 +CPU= 17.220
Uploading trickle at timestep: 7862400
  21:35:52 STEP 2186 H=2186:00 +CPU= 17.717
  21:36:21 STEP 2187 H=2187:00 +CPU= 27.738
  21:36:39 STEP 2188 H=2188:00 +CPU= 17.624
  21:36:58 STEP 2189 H=2189:00 +CPU= 17.706
  21:37:27 STEP 2190 H=2190:00 +CPU= 28.124
  21:37:46 STEP 2191 H=2191:00 +CPU= 18.028
  21:38:03 STEP 2192 H=2192:00 +CPU= 17.014
  21:38:26 STEP 2193 H=2193:00 +CPU= 22.481
  21:38:41 STEP 2194 H=2194:00 +CPU= 15.150
  21:38:55 STEP 2195 H=2195:00 +CPU= 13.111
  21:39:22 STEP 2196 H=2196:00 +CPU= 26.593
Moving to projects directory: /var/lib/boinc/slots/47/ICMGGhpi1+002196
Moving to projects directory: /var/lib/boinc/slots/47/ICMSHhpi1+002196
Moving to projects directory: /var/lib/boinc/slots/47/ICMUAhpi1+002196
  21:39:37 STEP 2197 H=2197:00 +CPU= 15.119
  21:39:55 STEP 2198 H=2198:00 +CPU= 17.707
  21:40:21 STEP 2199 H=2199:00 +CPU= 25.605
  21:40:38 STEP 2200 H=2200:00 +CPU= 16.636
  21:40:55 STEP 2201 H=2201:00 +CPU= 16.179
  21:41:22 STEP 2202 H=2202:00 +CPU= 27.401
  21:41:39 STEP 2203 H=2203:00 +CPU= 16.208
  21:41:57 STEP 2204 H=2204:00 +CPU= 18.468
  21:42:25 STEP 2205 H=2205:00 +CPU= 27.227
  21:42:43 STEP 2206 H=2206:00 +CPU= 17.564
  21:43:01 STEP 2207 H=2207:00 +CPU= 17.296
  21:43:45 STEP 2208 H=2208:00 +CPU= 32.269
Moving to projects directory: /var/lib/boinc/slots/47/ICMGGhpi1+002208
Moving to projects directory: /var/lib/boinc/slots/47/ICMSHhpi1+002208
Moving to projects directory: /var/lib/boinc/slots/47/ICMUAhpi1+002208
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMGGhpi1+002184
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMSHhpi1+002184
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMUAhpi1+002184
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMGGhpi1+002196
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMSHhpi1+002196
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMUAhpi1+002196
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_1797_2021050100_123_946_12164886_0_r985316780_2.zip
Uploading the intermediate file: upload_file_2.zip
  21:44:01 STEP 2209 H=2209:00 +CPU= 16.123
Uploading trickle at timestep: 7948800
  21:44:18 STEP 2210 H=2210:00 +CPU= 16.364
  21:44:46 STEP 2211 H=2211:00 +CPU= 27.589
  21:45:03 STEP 2212 H=2212:00 +CPU= 17.237
  21:45:20 STEP 2213 H=2213:00 +CPU= 16.307
  21:45:47 STEP 2214 H=2214:00 +CPU= 27.293
  21:46:04 STEP 2215 H=2215:00 +CPU= 16.695
  21:46:21 STEP 2216 H=2216:00 +CPU= 16.691
  21:46:47 STEP 2217 H=2217:00 +CPU= 25.704
  21:47:04 STEP 2218 H=2218:00 +CPU= 16.881
  21:47:22 STEP 2219 H=2219:00 +CPU= 17.309
  21:47:52 STEP 2220 H=2220:00 +CPU= 29.310
Moving to projects directory: /var/lib/boinc/slots/47/ICMGGhpi1+002220
Moving to projects directory: /var/lib/boinc/slots/47/ICMSHhpi1+002220
Moving to projects directory: /var/lib/boinc/slots/47/ICMUAhpi1+002220
  21:48:08 STEP 2221 H=2221:00 +CPU= 15.637
  21:48:24 STEP 2222 H=2222:00 +CPU= 16.428
  21:48:50 STEP 2223 H=2223:00 +CPU= 25.248
  21:49:08 STEP 2224 H=2224:00 +CPU= 17.750
  21:49:25 STEP 2225 H=2225:00 +CPU= 16.521
  21:49:52 STEP 2226 H=2226:00 +CPU= 27.079
  21:50:08 STEP 2227 H=2227:00 +CPU= 15.985
  21:50:27 STEP 2228 H=2228:00 +CPU= 18.771
  21:50:54 STEP 2229 H=2229:00 +CPU= 26.810
  21:51:12 STEP 2230 H=2230:00 +CPU= 17.713
  21:51:34 STEP 2231 H=2231:00 +CPU= 18.901
  21:52:13 STEP 2232 H=2232:00 +CPU= 30.501
Moving to projects directory: /var/lib/boinc/slots/47/ICMGGhpi1+002232
Moving to projects directory: /var/lib/boinc/slots/47/ICMSHhpi1+002232
Moving to projects directory: /var/lib/boinc/slots/47/ICMUAhpi1+002232
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMGGhpi1+002208
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMSHhpi1+002208
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMUAhpi1+002208
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMGGhpi1+002220
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMSHhpi1+002220
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMUAhpi1+002220
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_1797_2021050100_123_946_12164886_0_r985316780_3.zip
  21:52:29 STEP 2233 H=2233:00 +CPU= 16.201
  21:52:46 STEP 2234 H=2234:00 +CPU= 16.677
Uploading the intermediate file: upload_file_3.zip
  21:53:13 STEP 2235 H=2235:00 +CPU= 26.784
Uploading trickle at timestep: 8035200
  21:53:30 STEP 2236 H=2236:00 +CPU= 16.439
  21:53:44 STEP 2237 H=2237:00 +CPU= 14.687
  21:54:12 STEP 2238 H=2238:00 +CPU= 27.307
  21:54:30 STEP 2239 H=2239:00 +CPU= 17.940
  21:54:48 STEP 2240 H=2240:00 +CPU= 18.075
  21:55:16 STEP 2241 H=2241:00 +CPU= 27.642
  21:55:35 STEP 2242 H=2242:00 +CPU= 18.461
  21:55:52 STEP 2243 H=2243:00 +CPU= 16.864
  21:56:18 STEP 2244 H=2244:00 +CPU= 25.491
Moving to projects directory: /var/lib/boinc/slots/47/ICMGGhpi1+002244
Moving to projects directory: /var/lib/boinc/slots/47/ICMSHhpi1+002244
Moving to projects directory: /var/lib/boinc/slots/47/ICMUAhpi1+002244
  21:56:34 STEP 2245 H=2245:00 +CPU= 16.388
  21:56:52 STEP 2246 H=2246:00 +CPU= 17.543
  21:57:19 STEP 2247 H=2247:00 +CPU= 26.712
  21:57:36 STEP 2248 H=2248:00 +CPU= 17.299
  21:57:53 STEP 2249 H=2249:00 +CPU= 17.112
  21:58:21 STEP 2250 H=2250:00 +CPU= 27.599
  21:58:39 STEP 2251 H=2251:00 +CPU= 17.541
  21:58:58 STEP 2252 H=2252:00 +CPU= 18.355
  21:59:24 STEP 2253 H=2253:00 +CPU= 26.287
  21:59:42 STEP 2254 H=2254:00 +CPU= 17.789
  22:00:01 STEP 2255 H=2255:00 +CPU= 18.310
  22:00:41 STEP 2256 H=2256:00 +CPU= 32.851
Moving to projects directory: /var/lib/boinc/slots/47/ICMGGhpi1+002256
Moving to projects directory: /var/lib/boinc/slots/47/ICMSHhpi1+002256
Moving to projects directory: /var/lib/boinc/slots/47/ICMUAhpi1+002256
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMGGhpi1+002232
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMSHhpi1+002232
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMUAhpi1+002232
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMGGhpi1+002244
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMSHhpi1+002244
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886/ICMUAhpi1+002244
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_1797_2021050100_123_946_12164886_0_r985316780_4.zip
Uploading the intermediate file: upload_file_4.zip
  22:00:59 STEP 2257 H=2257:00 +CPU= 17.931
  22:01:18 STEP 2258 H=2258:00 +CPU= 18.265
Uploading trickle at timestep: 8121600
  22:01:45 STEP 2259 H=2259:00 +CPU= 26.968
  22:02:02 STEP 2260 H=2260:00 +CPU= 17.238
  22:02:19 STEP 2261 H=2261:00 +CPU= 16.031
  22:02:45 STEP 2262 H=2262:00 +CPU= 25.917
  22:03:02 STEP 2263 H=2263:00 +CPU= 17.003
  22:03:20 STEP 2264 H=2264:00 +CPU= 17.575
  22:03:46 STEP 2265 H=2265:00 +CPU= 26.723
  22:04:03 STEP 2266 H=2266:00 +CPU= 16.729
  22:04:21 STEP 2267 H=2267:00 +CPU= 17.173
  22:04:51 STEP 2268 H=2268:00 +CPU= 29.911
Moving to projects directory: /var/lib/boinc/slots/47/ICMGGhpi1+002268
Moving to projects directory: /var/lib/boinc/slots/47/ICMSHhpi1+002268
Moving to projects directory: /var/lib/boinc/slots/47/ICMUAhpi1+002268
  22:05:07 STEP 2269 H=2269:00 +CPU= 16.520
  22:05:26 STEP 2270 H=2270:00 +CPU= 18.151
  22:05:54 STEP 2271 H=2271:00 +CPU= 27.912
  22:06:11 STEP 2272 H=2272:00 +CPU= 17.053
  22:06:29 STEP 2273 H=2273:00 +CPU= 17.858
  22:06:57 STEP 2274 H=2274:00 +CPU= 28.066
  22:07:16 STEP 2275 H=2275:00 +CPU= 18.587
  22:07:35 STEP 2276 H=2276:00 +CPU= 18.292
  22:08:02 STEP 2277 H=2277:00 +CPU= 26.854
  22:08:20 STEP 2278 H=2278:00 +CPU= 18.218
  22:08:39 STEP 2279 H=2279:00 +CPU= 18.110
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.01_x86_64-pc-linux-gnu
(argv1) start_date: 2021050100
(argv2) exptid: hpi1
(argv3) unique_member_id: 1797
(argv4) batchid: 946
(argv5) wuid: 12164886
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc/slots/47
Project directory is: /var/lib/boinc/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.01
Location of temp folder: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164886
..mkdir for temp folder for results failed
Copying: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_app_1.01_x86_64-pc-linux-gnu.zip to: /var/lib/boinc/slots/47/oifs_43r3_ps_app_1.01_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc/slots/47/oifs_43r3_ps_app_1.01_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_3659bad2faf61568334f51b9a9d6e564 to: /var/lib/boinc/slots/47/oifs_43r3_ps_1797_2021050100_123_946_12164886.zip
Unzipping the namelist zip file: /var/lib/boinc/slots/47/oifs_43r3_ps_1797_2021050100_123_946_12164886.zip
ic_ancil_file: ic_ancil_12164886
ifsdata_file: ifsdata_12164886
climate_data_file: clim_data_12164886
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
Copying IC ancils from: ../../projects/climateprediction.net/jf_43854f33bbd5e4731ac504454f342a2c to: /var/lib/boinc/slots/47/ic_ancil_12164886.zip
Unzipping the IC ancils zip file: /var/lib/boinc/slots/47/ic_ancil_12164886.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_ecd5050cc3ea7ff9cff6d96bf8917ef9 to: /var/lib/boinc/slots/47/ifsdata/ifsdata_12164886.zip

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 May 2023 15:32:54 1524992 22246890 oifs_43r3_ps_1797_2021050100_123_946_12164886_0 8,121,600 2,009 0.0002
09 May 2023 15:32:54 1524992 22246890 oifs_43r3_ps_1797_2021050100_123_946_12164886_0 8,035,200 1,515 0.0002
09 May 2023 15:32:53 1524992 22246890 oifs_43r3_ps_1797_2021050100_123_946_12164886_0 7,948,800 1,018 0.0001
09 May 2023 15:32:53 1524992 22246890 oifs_43r3_ps_1797_2021050100_123_946_12164886_0 7,862,400 534 0.0001
09 May 2023 15:32:46 1524992 22246890 oifs_43r3_ps_1797_2021050100_123_946_12164886_0 7,776,000 6,555 0.0008
09 May 2023 15:32:46 1524992 22246890 oifs_43r3_ps_1797_2021050100_123_946_12164886_0 7,689,600 6,042 0.0008
09 May 2023 15:32:46 1524992 22246890 oifs_43r3_ps_1797_2021050100_123_946_12164886_0 7,603,200 5,538 0.0007
09 May 2023 15:32:46 1524992 22246890 oifs_43r3_ps_1797_2021050100_123_946_12164886_0 7,516,800 5,022 0.0007
09 May 2023 15:32:46 1524992 22246890 oifs_43r3_ps_1797_2021050100_123_946_12164886_0 7,430,400 4,527 0.0006
09 May 2023 15:32:46 1524992 22246890 oifs_43r3_ps_1797_2021050100_123_946_12164886_0 7,344,000 4,019 0.0005
09 May 2023 15:32:46 1524992 22246890 oifs_43r3_ps_1797_2021050100_123_946_12164886_0 7,257,600 3,515 0.0005
09 May 2023 15:32:37 1524992 22246890 oifs_43r3_ps_1797_2021050100_123_946_12164886_0 7,171,200 3,017 0.0004
09 May 2023 15:32:36 1524992 22246890 oifs_43r3_ps_1797_2021050100_123_946_12164886_0 7,084,800 2,541 0.0004
09 May 2023 15:32:37 1524992 22246890 oifs_43r3_ps_1797_2021050100_123_946_12164886_0 6,998,400 2,042 0.0003
09 May 2023 15:32:37 1524992 22246890 oifs_43r3_ps_1797_2021050100_123_946_12164886_0 6,912,000 1,553 0.0002
09 May 2023 15:32:36 1524992 22246890 oifs_43r3_ps_1797_2021050100_123_946_12164886_0 6,825,600 1,049 0.0002
09 May 2023 15:32:36 1524992 22246890 oifs_43r3_ps_1797_2021050100_123_946_12164886_0 6,739,200 545 0.0001
09 May 2023 15:32:29 1524992 22246890 oifs_43r3_ps_1797_2021050100_123_946_12164886_0 6,652,800 6,630 0.0010
09 May 2023 15:32:29 1524992 22246890 oifs_43r3_ps_1797_2021050100_123_946_12164886_0 6,566,400 6,113 0.0009
09 May 2023 15:32:29 1524992 22246890 oifs_43r3_ps_1797_2021050100_123_946_12164886_0 6,480,000 5,602 0.0009


©2024 cpdn.org