climateprediction.net home page
Task 22246771

Task 22246771

Name oifs_43r3_ps_1683_2021050100_123_946_12164772_0
Workunit 12164772
Created 29 Nov 2022, 12:02:39 UTC
Sent 29 Nov 2022, 12:30:33 UTC
Report deadline 29 Dec 2022, 12:30:33 UTC
Received 18 Mar 2023, 7:46:32 UTC
Server state Over
Outcome Computation error
Client state Aborted by user
Exit status 203 (0x000000CB) EXIT_ABORTED_VIA_GUI
Computer ID 1524992
Run time 14 hours 29 min 24 sec
CPU time 39 min 30 sec
Validate state Invalid
Credit 11,249.28
Device peak FLOPS 6.24 GFLOPS
Application version OpenIFS 43r3 Perturbed Surface v1.01
x86_64-pc-linux-gnu
Peak working set size 4,771.82 MB
Peak swap size 4,978.08 MB
Peak disk usage 9,502.24 MB
Stderr
<core_client_version>7.22.0</core_client_version>
<![CDATA[
<message>
aborted by user</message>
<stderr_txt>
845:00 +CPU= 28.976
  19:18:11 STEP 1846 H=1846:00 +CPU= 17.288
  19:18:30 STEP 1847 H=1847:00 +CPU= 18.798
  19:19:05 STEP 1848 H=1848:00 +CPU= 32.857
Moving to projects directory: /var/lib/boinc/slots/40/ICMGGhpi1+001848
Moving to projects directory: /var/lib/boinc/slots/40/ICMSHhpi1+001848
Moving to projects directory: /var/lib/boinc/slots/40/ICMUAhpi1+001848
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMGGhpi1+001824
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMSHhpi1+001824
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMUAhpi1+001824
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMGGhpi1+001836
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMSHhpi1+001836
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMUAhpi1+001836
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_1683_2021050100_123_946_12164772_0_r592969078_13.zip
  19:19:22 STEP 1849 H=1849:00 +CPU= 17.071
Uploading the intermediate file: upload_file_13.zip
  19:19:41 STEP 1850 H=1850:00 +CPU= 18.937
Uploading trickle at timestep: 6652800
  19:20:08 STEP 1851 H=1851:00 +CPU= 27.172
  19:20:27 STEP 1852 H=1852:00 +CPU= 18.672
  19:20:46 STEP 1853 H=1853:00 +CPU= 19.157
  19:21:14 STEP 1854 H=1854:00 +CPU= 27.495
  19:21:33 STEP 1855 H=1855:00 +CPU= 18.325
  19:21:50 STEP 1856 H=1856:00 +CPU= 17.789
  19:22:19 STEP 1857 H=1857:00 +CPU= 28.080
  19:22:36 STEP 1858 H=1858:00 +CPU= 17.421
  19:22:53 STEP 1859 H=1859:00 +CPU= 16.206
  19:23:23 STEP 1860 H=1860:00 +CPU= 30.476
Moving to projects directory: /var/lib/boinc/slots/40/ICMGGhpi1+001860
Moving to projects directory: /var/lib/boinc/slots/40/ICMSHhpi1+001860
Moving to projects directory: /var/lib/boinc/slots/40/ICMUAhpi1+001860
  19:23:41 STEP 1861 H=1861:00 +CPU= 17.369
  19:23:59 STEP 1862 H=1862:00 +CPU= 18.714
  19:24:27 STEP 1863 H=1863:00 +CPU= 27.187
  19:24:45 STEP 1864 H=1864:00 +CPU= 17.897
  19:25:02 STEP 1865 H=1865:00 +CPU= 17.166
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.01_x86_64-pc-linux-gnu
(argv1) start_date: 2021050100
(argv2) exptid: hpi1
(argv3) unique_member_id: 1683
(argv4) batchid: 946
(argv5) wuid: 12164772
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc/slots/40
Project directory is: /var/lib/boinc/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.01
Location of temp folder: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772
..mkdir for temp folder for results failed
Copying: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_app_1.01_x86_64-pc-linux-gnu.zip to: /var/lib/boinc/slots/40/oifs_43r3_ps_app_1.01_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc/slots/40/oifs_43r3_ps_app_1.01_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_5ba1e9f8b9fbf1bea9196537bb2387f1 to: /var/lib/boinc/slots/40/oifs_43r3_ps_1683_2021050100_123_946_12164772.zip
Unzipping the namelist zip file: /var/lib/boinc/slots/40/oifs_43r3_ps_1683_2021050100_123_946_12164772.zip
ic_ancil_file: ic_ancil_12164772
ifsdata_file: ifsdata_12164772
climate_data_file: clim_data_12164772
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
Copying IC ancils from: ../../projects/climateprediction.net/jf_86d8a9bb11002fce89519a65e26a69c6 to: /var/lib/boinc/slots/40/ic_ancil_12164772.zip
Unzipping the IC ancils zip file: /var/lib/boinc/slots/40/ic_ancil_12164772.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_922d18559d90fc947765e7eadc776432 to: /var/lib/boinc/slots/40/ifsdata/ifsdata_12164772.zip
Unzipping the ifsdata_zip file: /var/lib/boinc/slots/40/ifsdata/ifsdata_12164772.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc/slots/40/159l_2/clim_data_12164772.zip
Unzipping the climate data zip file: /var/lib/boinc/slots/40/159l_2/clim_data_12164772.zip
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_1683_2021050100_123_946_12164772_0_r592969078
The child process has been launched with process id: 272017
Executing the command: /var/lib/boinc/slots/40/./master.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:guoj-desktop:1:1:272017:272017] [20221203:192639:1670120799:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:guoj-desktop:1:1:272017:272017] [20221203:192639:1670120799:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:guoj-desktop:1:1:272017:272017] [20221203:192639:1670120799:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:guoj-desktop:1:1:272017:272017] [20221203:192639:1670120799:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:guoj-desktop:1:1:272017:272017] [20221203:192639:1670120799:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:guoj-desktop:1:1:272017:272017] [20221203:192639:1670120799:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:guoj-desktop:1:1:272017:272017] [20221203:192639:1670120799:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:guoj-desktop:1:1:272017:272017] [20221203:192639:1670120799:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:guoj-desktop:1:1:272017:272017] [20221203:192639:1670120799:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:guoj-desktop:1:1:272017:272017] [20221203:192639:1670120799:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:guoj-desktop:1:1:272017:272017] [20221203:192639:1670120799:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:guoj-desktop:1:1:272017:272017] [20221203:192639:1670120799:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:guoj-desktop:1:1:272017:272017] [20221203:192639:1670120799:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:guoj-desktop:1:1:272017:272017] [20221203:192639:1670120799:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:guoj-desktop:1:1:272017:272017] [20221203:192639:1670120799:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:guoj-desktop:1:1:272017:272017] [20221203:192639:1670120799:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:guoj-desktop:1:1:272017:272017] [20221203:192639:1670120799:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:guoj-desktop:1:1:272017:272017] [20221203:192639:1670120799:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:guoj-desktop:1:1:272017:272017] [20221203:192639:1670120799:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:guoj-desktop:1:1:272017:272017] [20221203:192639:1670120799:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:guoj-desktop:1:1:272017:272017] [20221203:192639:1670120799:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:guoj-desktop:1:1:272017:272017] [20221203:192639:1670120799:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:guoj-desktop:1:1:272017:272017] [20221203:192639:1670120799:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:guoj-desktop:1:1:272017:272017] [20221203:192639:1670120799:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:guoj-desktop:1:1:272017:272017] [20221203:192639:1670120799:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:guoj-desktop:1:1:272017:272017] [20221203:192639:1670120799:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:guoj-desktop:1:1:272017:272017] [20221203:192639:1670120799:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:guoj-desktop:1:1:272017:272017] [20221203:192639:1670120799:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:guoj-desktop:1:1:272017:272017] [20221203:192639:1670120799:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:guoj-desktop:1:1:272017:272017] [20221203:192639:1670120799:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  19:27:28 STEP 1848 H=1848:00 +CPU= 30.909
  19:27:45 STEP 1849 H=1849:00 +CPU= 16.967
  19:28:03 STEP 1850 H=1850:00 +CPU= 17.159
  19:28:29 STEP 1851 H=1851:00 +CPU= 26.608
  19:28:48 STEP 1852 H=1852:00 +CPU= 18.253
  19:29:05 STEP 1853 H=1853:00 +CPU= 17.439
  19:29:29 STEP 1854 H=1854:00 +CPU= 23.902
  19:29:46 STEP 1855 H=1855:00 +CPU= 16.969
  19:30:04 STEP 1856 H=1856:00 +CPU= 18.078
  19:30:31 STEP 1857 H=1857:00 +CPU= 26.887
  19:30:49 STEP 1858 H=1858:00 +CPU= 17.057
  19:31:07 STEP 1859 H=1859:00 +CPU= 17.887
  19:31:37 STEP 1860 H=1860:00 +CPU= 29.799
Moving to projects directory: /var/lib/boinc/slots/40/ICMGGhpi1+001860
Moving to projects directory: /var/lib/boinc/slots/40/ICMSHhpi1+001860
Moving to projects directory: /var/lib/boinc/slots/40/ICMUAhpi1+001860
  19:31:54 STEP 1861 H=1861:00 +CPU= 17.468
  19:32:13 STEP 1862 H=1862:00 +CPU= 18.824
  19:32:41 STEP 1863 H=1863:00 +CPU= 27.427
  19:32:59 STEP 1864 H=1864:00 +CPU= 18.151
  19:33:17 STEP 1865 H=1865:00 +CPU= 17.444
  19:33:46 STEP 1866 H=1866:00 +CPU= 28.927
  19:34:03 STEP 1867 H=1867:00 +CPU= 17.114
  19:34:23 STEP 1868 H=1868:00 +CPU= 19.028
  19:34:52 STEP 1869 H=1869:00 +CPU= 29.494
  19:35:10 STEP 1870 H=1870:00 +CPU= 17.871
  19:35:33 STEP 1871 H=1871:00 +CPU= 17.508
  19:36:14 STEP 1872 H=1872:00 +CPU= 31.754
Moving to projects directory: /var/lib/boinc/slots/40/ICMGGhpi1+001872
Moving to projects directory: /var/lib/boinc/slots/40/ICMSHhpi1+001872
Moving to projects directory: /var/lib/boinc/slots/40/ICMUAhpi1+001872
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMGGhpi1+001848
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMSHhpi1+001848
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMUAhpi1+001848
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMGGhpi1+001860
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMSHhpi1+001860
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMUAhpi1+001860
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_1683_2021050100_123_946_12164772_0_r592969078_1.zip
Uploading the intermediate file: upload_file_1.zip
  19:36:31 STEP 1873 H=1873:00 +CPU= 17.084
Uploading trickle at timestep: 6739200
  19:36:49 STEP 1874 H=1874:00 +CPU= 17.924
  19:37:16 STEP 1875 H=1875:00 +CPU= 27.096
  19:37:34 STEP 1876 H=1876:00 +CPU= 18.011
  19:37:52 STEP 1877 H=1877:00 +CPU= 17.213
  19:38:19 STEP 1878 H=1878:00 +CPU= 27.094
  19:38:35 STEP 1879 H=1879:00 +CPU= 15.867
  19:38:52 STEP 1880 H=1880:00 +CPU= 16.627
  19:39:19 STEP 1881 H=1881:00 +CPU= 26.466
  19:39:37 STEP 1882 H=1882:00 +CPU= 18.725
  19:39:54 STEP 1883 H=1883:00 +CPU= 16.999
  19:40:25 STEP 1884 H=1884:00 +CPU= 30.280
Moving to projects directory: /var/lib/boinc/slots/40/ICMGGhpi1+001884
Moving to projects directory: /var/lib/boinc/slots/40/ICMSHhpi1+001884
Moving to projects directory: /var/lib/boinc/slots/40/ICMUAhpi1+001884
  19:40:42 STEP 1885 H=1885:00 +CPU= 16.920
  19:41:00 STEP 1886 H=1886:00 +CPU= 18.211
  19:41:28 STEP 1887 H=1887:00 +CPU= 27.232
  19:41:47 STEP 1888 H=1888:00 +CPU= 19.095
  19:42:05 STEP 1889 H=1889:00 +CPU= 17.745
  19:42:32 STEP 1890 H=1890:00 +CPU= 27.248
  19:42:50 STEP 1891 H=1891:00 +CPU= 17.030
  19:43:08 STEP 1892 H=1892:00 +CPU= 18.279
  19:43:36 STEP 1893 H=1893:00 +CPU= 27.612
  19:43:54 STEP 1894 H=1894:00 +CPU= 17.720
  19:44:12 STEP 1895 H=1895:00 +CPU= 17.865
  19:44:45 STEP 1896 H=1896:00 +CPU= 32.101
Moving to projects directory: /var/lib/boinc/slots/40/ICMGGhpi1+001896
Moving to projects directory: /var/lib/boinc/slots/40/ICMSHhpi1+001896
Moving to projects directory: /var/lib/boinc/slots/40/ICMUAhpi1+001896
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMGGhpi1+001872
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMSHhpi1+001872
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMUAhpi1+001872
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMGGhpi1+001884
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMSHhpi1+001884
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMUAhpi1+001884
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_1683_2021050100_123_946_12164772_0_r592969078_2.zip
  19:45:02 STEP 1897 H=1897:00 +CPU= 17.414
Uploading the intermediate file: upload_file_2.zip
  19:45:21 STEP 1898 H=1898:00 +CPU= 18.937
Uploading trickle at timestep: 6825600
  19:45:47 STEP 1899 H=1899:00 +CPU= 25.491
  19:46:03 STEP 1900 H=1900:00 +CPU= 16.056
  19:46:20 STEP 1901 H=1901:00 +CPU= 16.269
  19:46:46 STEP 1902 H=1902:00 +CPU= 25.980
  19:47:03 STEP 1903 H=1903:00 +CPU= 15.481
  19:47:22 STEP 1904 H=1904:00 +CPU= 18.054
  19:47:46 STEP 1905 H=1905:00 +CPU= 24.060
  19:48:03 STEP 1906 H=1906:00 +CPU= 17.275
  19:48:20 STEP 1907 H=1907:00 +CPU= 16.428
  19:48:50 STEP 1908 H=1908:00 +CPU= 29.848
Moving to projects directory: /var/lib/boinc/slots/40/ICMGGhpi1+001908
Moving to projects directory: /var/lib/boinc/slots/40/ICMSHhpi1+001908
Moving to projects directory: /var/lib/boinc/slots/40/ICMUAhpi1+001908
  19:49:07 STEP 1909 H=1909:00 +CPU= 17.018
  19:49:24 STEP 1910 H=1910:00 +CPU= 17.073
  19:49:52 STEP 1911 H=1911:00 +CPU= 27.798
  19:50:10 STEP 1912 H=1912:00 +CPU= 17.827
  19:50:27 STEP 1913 H=1913:00 +CPU= 17.473
  19:50:56 STEP 1914 H=1914:00 +CPU= 28.619
  19:51:15 STEP 1915 H=1915:00 +CPU= 18.555
  19:51:32 STEP 1916 H=1916:00 +CPU= 16.962
  19:51:59 STEP 1917 H=1917:00 +CPU= 27.124
  19:52:18 STEP 1918 H=1918:00 +CPU= 18.349
  19:52:35 STEP 1919 H=1919:00 +CPU= 17.395
  19:53:13 STEP 1920 H=1920:00 +CPU= 32.684
Moving to projects directory: /var/lib/boinc/slots/40/ICMGGhpi1+001920
Moving to projects directory: /var/lib/boinc/slots/40/ICMSHhpi1+001920
Moving to projects directory: /var/lib/boinc/slots/40/ICMUAhpi1+001920
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMGGhpi1+001896
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMSHhpi1+001896
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMUAhpi1+001896
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMGGhpi1+001908
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMSHhpi1+001908
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMUAhpi1+001908
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_1683_2021050100_123_946_12164772_0_r592969078_3.zip
Uploading the intermediate file: upload_file_3.zip
  19:53:29 STEP 1921 H=1921:00 +CPU= 16.609
Uploading trickle at timestep: 6912000
  19:53:48 STEP 1922 H=1922:00 +CPU= 18.089
  19:54:15 STEP 1923 H=1923:00 +CPU= 27.034
  19:54:30 STEP 1924 H=1924:00 +CPU= 14.794
  19:54:44 STEP 1925 H=1925:00 +CPU= 14.412
  19:55:09 STEP 1926 H=1926:00 +CPU= 25.233
  19:55:26 STEP 1927 H=1927:00 +CPU= 16.782
  19:55:45 STEP 1928 H=1928:00 +CPU= 18.718
  19:56:12 STEP 1929 H=1929:00 +CPU= 26.968
  19:56:29 STEP 1930 H=1930:00 +CPU= 17.157
  19:56:47 STEP 1931 H=1931:00 +CPU= 17.513
  19:57:16 STEP 1932 H=1932:00 +CPU= 29.165
Moving to projects directory: /var/lib/boinc/slots/40/ICMGGhpi1+001932
Moving to projects directory: /var/lib/boinc/slots/40/ICMSHhpi1+001932
Moving to projects directory: /var/lib/boinc/slots/40/ICMUAhpi1+001932
  19:57:33 STEP 1933 H=1933:00 +CPU= 17.150
  19:57:51 STEP 1934 H=1934:00 +CPU= 16.977
  19:58:18 STEP 1935 H=1935:00 +CPU= 27.086
  19:58:35 STEP 1936 H=1936:00 +CPU= 16.955
  19:58:52 STEP 1937 H=1937:00 +CPU= 16.668
  19:59:20 STEP 1938 H=1938:00 +CPU= 27.905
  19:59:37 STEP 1939 H=1939:00 +CPU= 17.230
  19:59:56 STEP 1940 H=1940:00 +CPU= 18.463
  20:00:23 STEP 1941 H=1941:00 +CPU= 27.327
  20:00:41 STEP 1942 H=1942:00 +CPU= 17.551
  20:00:57 STEP 1943 H=1943:00 +CPU= 16.042
  20:01:30 STEP 1944 H=1944:00 +CPU= 31.614
Moving to projects directory: /var/lib/boinc/slots/40/ICMGGhpi1+001944
Moving to projects directory: /var/lib/boinc/slots/40/ICMSHhpi1+001944
Moving to projects directory: /var/lib/boinc/slots/40/ICMUAhpi1+001944
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMGGhpi1+001920
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMSHhpi1+001920
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMUAhpi1+001920
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMGGhpi1+001932
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMSHhpi1+001932
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMUAhpi1+001932
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_1683_2021050100_123_946_12164772_0_r592969078_4.zip
Uploading the intermediate file: upload_file_4.zip
  20:01:48 STEP 1945 H=1945:00 +CPU= 17.822
Uploading trickle at timestep: 6998400
  20:02:06 STEP 1946 H=1946:00 +CPU= 17.377
  20:02:33 STEP 1947 H=1947:00 +CPU= 27.335
  20:02:50 STEP 1948 H=1948:00 +CPU= 17.069
  20:03:07 STEP 1949 H=1949:00 +CPU= 17.154
  20:03:36 STEP 1950 H=1950:00 +CPU= 28.301
  20:03:53 STEP 1951 H=1951:00 +CPU= 17.041
  20:04:10 STEP 1952 H=1952:00 +CPU= 17.402
  20:04:37 STEP 1953 H=1953:00 +CPU= 26.791
  20:04:56 STEP 1954 H=1954:00 +CPU= 18.126
  20:05:13 STEP 1955 H=1955:00 +CPU= 16.816
  20:05:44 STEP 1956 H=1956:00 +CPU= 30.968
Moving to projects directory: /var/lib/boinc/slots/40/ICMGGhpi1+001956
Moving to projects directory: /var/lib/boinc/slots/40/ICMSHhpi1+001956
Moving to projects directory: /var/lib/boinc/slots/40/ICMUAhpi1+001956
  20:06:00 STEP 1957 H=1957:00 +CPU= 16.476
  20:06:18 STEP 1958 H=1958:00 +CPU= 17.517
  20:06:44 STEP 1959 H=1959:00 +CPU= 26.654
  20:07:00 STEP 1960 H=1960:00 +CPU= 15.681
  20:07:18 STEP 1961 H=1961:00 +CPU= 17.352
  20:07:45 STEP 1962 H=1962:00 +CPU= 27.432
  20:08:03 STEP 1963 H=1963:00 +CPU= 17.286
  20:08:21 STEP 1964 H=1964:00 +CPU= 17.935
  20:08:48 STEP 1965 H=1965:00 +CPU= 27.126
  20:09:06 STEP 1966 H=1966:00 +CPU= 18.426
  20:09:23 STEP 1967 H=1967:00 +CPU= 16.559
  20:09:53 STEP 1968 H=1968:00 +CPU= 30.186
Moving to projects directory: /var/lib/boinc/slots/40/ICMGGhpi1+001968
Moving to projects directory: /var/lib/boinc/slots/40/ICMSHhpi1+001968
Moving to projects directory: /var/lib/boinc/slots/40/ICMUAhpi1+001968
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMGGhpi1+001944
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMSHhpi1+001944
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMUAhpi1+001944
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMGGhpi1+001956
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMSHhpi1+001956
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMUAhpi1+001956
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_1683_2021050100_123_946_12164772_0_r592969078_5.zip
  20:10:08 STEP 1969 H=1969:00 +CPU= 14.806
  20:10:24 STEP 1970 H=1970:00 +CPU= 15.564
Uploading the intermediate file: upload_file_5.zip
  20:10:50 STEP 1971 H=1971:00 +CPU= 25.981
Uploading trickle at timestep: 7084800
  20:11:07 STEP 1972 H=1972:00 +CPU= 17.049
  20:11:24 STEP 1973 H=1973:00 +CPU= 17.026
  20:11:51 STEP 1974 H=1974:00 +CPU= 26.853
  20:12:09 STEP 1975 H=1975:00 +CPU= 18.107
  20:12:26 STEP 1976 H=1976:00 +CPU= 16.635
  20:12:52 STEP 1977 H=1977:00 +CPU= 25.769
  20:13:09 STEP 1978 H=1978:00 +CPU= 16.865
  20:13:25 STEP 1979 H=1979:00 +CPU= 16.683
  20:13:52 STEP 1980 H=1980:00 +CPU= 26.276
Moving to projects directory: /var/lib/boinc/slots/40/ICMGGhpi1+001980
Moving to projects directory: /var/lib/boinc/slots/40/ICMSHhpi1+001980
Moving to projects directory: /var/lib/boinc/slots/40/ICMUAhpi1+001980
  20:14:09 STEP 1981 H=1981:00 +CPU= 16.973
  20:14:27 STEP 1982 H=1982:00 +CPU= 18.279
  20:14:56 STEP 1983 H=1983:00 +CPU= 28.579
  20:15:14 STEP 1984 H=1984:00 +CPU= 18.408
  20:15:32 STEP 1985 H=1985:00 +CPU= 17.290
  20:16:01 STEP 1986 H=1986:00 +CPU= 28.820
  20:16:18 STEP 1987 H=1987:00 +CPU= 16.830
  20:16:35 STEP 1988 H=1988:00 +CPU= 16.456
  20:17:03 STEP 1989 H=1989:00 +CPU= 28.023
  20:17:21 STEP 1990 H=1990:00 +CPU= 17.866
  20:17:40 STEP 1991 H=1991:00 +CPU= 18.878
  20:18:13 STEP 1992 H=1992:00 +CPU= 31.399
Moving to projects directory: /var/lib/boinc/slots/40/ICMGGhpi1+001992
Moving to projects directory: /var/lib/boinc/slots/40/ICMSHhpi1+001992
Moving to projects directory: /var/lib/boinc/slots/40/ICMUAhpi1+001992
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMGGhpi1+001968
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMSHhpi1+001968
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMUAhpi1+001968
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMGGhpi1+001980
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMSHhpi1+001980
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMUAhpi1+001980
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_1683_2021050100_123_946_12164772_0_r592969078_6.zip
Uploading the intermediate file: upload_file_6.zip
  20:18:30 STEP 1993 H=1993:00 +CPU= 16.169
Uploading trickle at timestep: 7171200
  20:18:48 STEP 1994 H=1994:00 +CPU= 18.178
  20:19:16 STEP 1995 H=1995:00 +CPU= 28.372
  20:19:34 STEP 1996 H=1996:00 +CPU= 17.404
  20:19:50 STEP 1997 H=1997:00 +CPU= 16.090
  20:20:18 STEP 1998 H=1998:00 +CPU= 28.125
  20:20:35 STEP 1999 H=1999:00 +CPU= 16.382
  20:20:52 STEP 2000 H=2000:00 +CPU= 16.794
  20:21:17 STEP 2001 H=2001:00 +CPU= 25.307
  20:21:35 STEP 2002 H=2002:00 +CPU= 17.504
  20:21:53 STEP 2003 H=2003:00 +CPU= 18.552
  20:22:24 STEP 2004 H=2004:00 +CPU= 30.388
Moving to projects directory: /var/lib/boinc/slots/40/ICMGGhpi1+002004
Moving to projects directory: /var/lib/boinc/slots/40/ICMSHhpi1+002004
Moving to projects directory: /var/lib/boinc/slots/40/ICMUAhpi1+002004
  20:22:40 STEP 2005 H=2005:00 +CPU= 15.718
  20:22:57 STEP 2006 H=2006:00 +CPU= 17.155
  20:23:23 STEP 2007 H=2007:00 +CPU= 25.730
  20:23:40 STEP 2008 H=2008:00 +CPU= 16.945
  20:23:56 STEP 2009 H=2009:00 +CPU= 16.353
  20:24:25 STEP 2010 H=2010:00 +CPU= 28.354
  20:24:42 STEP 2011 H=2011:00 +CPU= 17.168
  20:25:00 STEP 2012 H=2012:00 +CPU= 17.660
  20:25:28 STEP 2013 H=2013:00 +CPU= 27.710
  20:25:46 STEP 2014 H=2014:00 +CPU= 17.809
  20:26:05 STEP 2015 H=2015:00 +CPU= 19.012
  20:26:41 STEP 2016 H=2016:00 +CPU= 32.925
Moving to projects directory: /var/lib/boinc/slots/40/ICMGGhpi1+002016
Moving to projects directory: /var/lib/boinc/slots/40/ICMSHhpi1+002016
Moving to projects directory: /var/lib/boinc/slots/40/ICMUAhpi1+002016
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMGGhpi1+001992
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMSHhpi1+001992
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMUAhpi1+001992
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMGGhpi1+002004
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMSHhpi1+002004
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMUAhpi1+002004
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_1683_2021050100_123_946_12164772_0_r592969078_7.zip
  20:26:59 STEP 2017 H=2017:00 +CPU= 17.828
Uploading the intermediate file: upload_file_7.zip
  20:27:16 STEP 2018 H=2018:00 +CPU= 17.495
Uploading trickle at timestep: 7257600
  20:27:44 STEP 2019 H=2019:00 +CPU= 27.340
  20:28:02 STEP 2020 H=2020:00 +CPU= 18.079
  20:28:17 STEP 2021 H=2021:00 +CPU= 15.034
  20:28:45 STEP 2022 H=2022:00 +CPU= 27.692
  20:29:03 STEP 2023 H=2023:00 +CPU= 17.166
  20:29:19 STEP 2024 H=2024:00 +CPU= 16.500
  20:29:47 STEP 2025 H=2025:00 +CPU= 27.872
  20:30:05 STEP 2026 H=2026:00 +CPU= 17.190
  20:30:22 STEP 2027 H=2027:00 +CPU= 16.785
  20:30:56 STEP 2028 H=2028:00 +CPU= 31.287
Moving to projects directory: /var/lib/boinc/slots/40/ICMGGhpi1+002028
Moving to projects directory: /var/lib/boinc/slots/40/ICMSHhpi1+002028
Moving to projects directory: /var/lib/boinc/slots/40/ICMUAhpi1+002028
  20:31:12 STEP 2029 H=2029:00 +CPU= 15.378
  20:31:29 STEP 2030 H=2030:00 +CPU= 16.748
  20:31:54 STEP 2031 H=2031:00 +CPU= 25.408
  20:32:12 STEP 2032 H=2032:00 +CPU= 17.333
  20:32:27 STEP 2033 H=2033:00 +CPU= 15.492
  20:32:56 STEP 2034 H=2034:00 +CPU= 28.172
  20:33:13 STEP 2035 H=2035:00 +CPU= 17.171
  20:33:30 STEP 2036 H=2036:00 +CPU= 17.573
  20:33:58 STEP 2037 H=2037:00 +CPU= 27.642
  20:34:17 STEP 2038 H=2038:00 +CPU= 18.093
  20:34:35 STEP 2039 H=2039:00 +CPU= 18.067
  20:35:16 STEP 2040 H=2040:00 +CPU= 32.613
Moving to projects directory: /var/lib/boinc/slots/40/ICMGGhpi1+002040
Moving to projects directory: /var/lib/boinc/slots/40/ICMSHhpi1+002040
Moving to projects directory: /var/lib/boinc/slots/40/ICMUAhpi1+002040
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMGGhpi1+002016
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMSHhpi1+002016
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMUAhpi1+002016
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMGGhpi1+002028
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMSHhpi1+002028
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMUAhpi1+002028
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_1683_2021050100_123_946_12164772_0_r592969078_8.zip
Uploading the intermediate file: upload_file_8.zip
  20:35:34 STEP 2041 H=2041:00 +CPU= 17.821
  20:35:52 STEP 2042 H=2042:00 +CPU= 18.018
Uploading trickle at timestep: 7344000
  20:36:21 STEP 2043 H=2043:00 +CPU= 28.177
  20:36:39 STEP 2044 H=2044:00 +CPU= 18.330
  20:36:57 STEP 2045 H=2045:00 +CPU= 17.308
  20:37:25 STEP 2046 H=2046:00 +CPU= 27.897
  20:37:42 STEP 2047 H=2047:00 +CPU= 17.189
  20:38:00 STEP 2048 H=2048:00 +CPU= 17.616
  20:38:28 STEP 2049 H=2049:00 +CPU= 27.839
  20:38:44 STEP 2050 H=2050:00 +CPU= 15.922
  20:39:01 STEP 2051 H=2051:00 +CPU= 15.948
  20:39:29 STEP 2052 H=2052:00 +CPU= 27.737
Moving to projects directory: /var/lib/boinc/slots/40/ICMGGhpi1+002052
Moving to projects directory: /var/lib/boinc/slots/40/ICMSHhpi1+002052
Moving to projects directory: /var/lib/boinc/slots/40/ICMUAhpi1+002052
  20:39:46 STEP 2053 H=2053:00 +CPU= 16.446
  20:40:03 STEP 2054 H=2054:00 +CPU= 16.824
  20:40:30 STEP 2055 H=2055:00 +CPU= 26.644
  20:40:48 STEP 2056 H=2056:00 +CPU= 18.378
  20:41:05 STEP 2057 H=2057:00 +CPU= 17.016
  20:41:34 STEP 2058 H=2058:00 +CPU= 28.506
  20:41:51 STEP 2059 H=2059:00 +CPU= 17.164
  20:42:09 STEP 2060 H=2060:00 +CPU= 17.633
  20:42:36 STEP 2061 H=2061:00 +CPU= 27.007
  20:42:54 STEP 2062 H=2062:00 +CPU= 17.585
  20:43:14 STEP 2063 H=2063:00 +CPU= 18.614
  20:44:00 STEP 2064 H=2064:00 +CPU= 32.034
Moving to projects directory: /var/lib/boinc/slots/40/ICMGGhpi1+002064
Moving to projects directory: /var/lib/boinc/slots/40/ICMSHhpi1+002064
Moving to projects directory: /var/lib/boinc/slots/40/ICMUAhpi1+002064
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMGGhpi1+002040
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMSHhpi1+002040
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMUAhpi1+002040
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMGGhpi1+002052
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMSHhpi1+002052
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMUAhpi1+002052
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_1683_2021050100_123_946_12164772_0_r592969078_9.zip
  20:44:17 STEP 2065 H=2065:00 +CPU= 17.578
Uploading the intermediate file: upload_file_9.zip
  20:44:36 STEP 2066 H=2066:00 +CPU= 18.153
Uploading trickle at timestep: 7430400
  20:45:03 STEP 2067 H=2067:00 +CPU= 27.141
  20:45:20 STEP 2068 H=2068:00 +CPU= 16.755
  20:45:37 STEP 2069 H=2069:00 +CPU= 16.826
  20:46:04 STEP 2070 H=2070:00 +CPU= 27.318
  20:46:21 STEP 2071 H=2071:00 +CPU= 16.506
  20:46:38 STEP 2072 H=2072:00 +CPU= 16.719
  20:47:05 STEP 2073 H=2073:00 +CPU= 27.441
  20:47:23 STEP 2074 H=2074:00 +CPU= 17.529
  20:47:41 STEP 2075 H=2075:00 +CPU= 17.347
  20:48:11 STEP 2076 H=2076:00 +CPU= 30.033
Moving to projects directory: /var/lib/boinc/slots/40/ICMGGhpi1+002076
Moving to projects directory: /var/lib/boinc/slots/40/ICMSHhpi1+002076
Moving to projects directory: /var/lib/boinc/slots/40/ICMUAhpi1+002076
  20:48:29 STEP 2077 H=2077:00 +CPU= 17.979
  20:48:48 STEP 2078 H=2078:00 +CPU= 18.907
  20:49:14 STEP 2079 H=2079:00 +CPU= 25.806
  20:49:28 STEP 2080 H=2080:00 +CPU= 14.412
  20:49:46 STEP 2081 H=2081:00 +CPU= 17.049
  20:50:14 STEP 2082 H=2082:00 +CPU= 28.001
  20:50:32 STEP 2083 H=2083:00 +CPU= 17.646
  20:50:49 STEP 2084 H=2084:00 +CPU= 17.180
  20:51:19 STEP 2085 H=2085:00 +CPU= 29.038
  20:51:36 STEP 2086 H=2086:00 +CPU= 17.556
  20:51:53 STEP 2087 H=2087:00 +CPU= 16.566
  20:52:37 STEP 2088 H=2088:00 +CPU= 33.469
Moving to projects directory: /var/lib/boinc/slots/40/ICMGGhpi1+002088
Moving to projects directory: /var/lib/boinc/slots/40/ICMSHhpi1+002088
Moving to projects directory: /var/lib/boinc/slots/40/ICMUAhpi1+002088
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMGGhpi1+002064
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMSHhpi1+002064
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMUAhpi1+002064
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMGGhpi1+002076
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMSHhpi1+002076
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMUAhpi1+002076
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_1683_2021050100_123_946_12164772_0_r592969078_10.zip
  20:52:55 STEP 2089 H=2089:00 +CPU= 18.201
Uploading the intermediate file: upload_file_10.zip
  20:53:14 STEP 2090 H=2090:00 +CPU= 17.930
Uploading trickle at timestep: 7516800
  20:53:41 STEP 2091 H=2091:00 +CPU= 27.736
  20:54:01 STEP 2092 H=2092:00 +CPU= 19.094
  20:54:18 STEP 2093 H=2093:00 +CPU= 16.999
  20:54:46 STEP 2094 H=2094:00 +CPU= 27.805
  20:55:02 STEP 2095 H=2095:00 +CPU= 16.552
  20:55:19 STEP 2096 H=2096:00 +CPU= 16.881
  20:55:46 STEP 2097 H=2097:00 +CPU= 26.948
  20:56:05 STEP 2098 H=2098:00 +CPU= 17.943
  20:56:22 STEP 2099 H=2099:00 +CPU= 17.382
  20:56:54 STEP 2100 H=2100:00 +CPU= 31.294
Moving to projects directory: /var/lib/boinc/slots/40/ICMGGhpi1+002100
Moving to projects directory: /var/lib/boinc/slots/40/ICMSHhpi1+002100
Moving to projects directory: /var/lib/boinc/slots/40/ICMUAhpi1+002100
  20:57:11 STEP 2101 H=2101:00 +CPU= 17.116
  20:57:29 STEP 2102 H=2102:00 +CPU= 18.229
  20:57:54 STEP 2103 H=2103:00 +CPU= 24.570
  20:58:12 STEP 2104 H=2104:00 +CPU= 17.901
  20:58:29 STEP 2105 H=2105:00 +CPU= 17.285
  20:58:58 STEP 2106 H=2106:00 +CPU= 28.168
  20:59:16 STEP 2107 H=2107:00 +CPU= 18.247
  20:59:35 STEP 2108 H=2108:00 +CPU= 18.153
  21:00:02 STEP 2109 H=2109:00 +CPU= 27.458
  21:00:20 STEP 2110 H=2110:00 +CPU= 17.708
  21:00:39 STEP 2111 H=2111:00 +CPU= 18.730
  21:01:13 STEP 2112 H=2112:00 +CPU= 30.011
Moving to projects directory: /var/lib/boinc/slots/40/ICMGGhpi1+002112
Moving to projects directory: /var/lib/boinc/slots/40/ICMSHhpi1+002112
Moving to projects directory: /var/lib/boinc/slots/40/ICMUAhpi1+002112
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMGGhpi1+002088
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMSHhpi1+002088
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMUAhpi1+002088
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMGGhpi1+002100
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMSHhpi1+002100
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMUAhpi1+002100
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_1683_2021050100_123_946_12164772_0_r592969078_11.zip
Uploading the intermediate file: upload_file_11.zip
  21:01:30 STEP 2113 H=2113:00 +CPU= 16.416
Uploading trickle at timestep: 7603200
  21:01:48 STEP 2114 H=2114:00 +CPU= 17.713
  21:02:16 STEP 2115 H=2115:00 +CPU= 27.802
  21:02:35 STEP 2116 H=2116:00 +CPU= 19.455
  21:02:54 STEP 2117 H=2117:00 +CPU= 17.801
  21:03:21 STEP 2118 H=2118:00 +CPU= 27.624
  21:03:39 STEP 2119 H=2119:00 +CPU= 17.802
  21:03:57 STEP 2120 H=2120:00 +CPU= 18.058
  21:04:24 STEP 2121 H=2121:00 +CPU= 26.614
  21:04:41 STEP 2122 H=2122:00 +CPU= 16.396
  21:04:58 STEP 2123 H=2123:00 +CPU= 17.331
  21:05:31 STEP 2124 H=2124:00 +CPU= 31.659
Moving to projects directory: /var/lib/boinc/slots/40/ICMGGhpi1+002124
Moving to projects directory: /var/lib/boinc/slots/40/ICMSHhpi1+002124
Moving to projects directory: /var/lib/boinc/slots/40/ICMUAhpi1+002124
  21:05:47 STEP 2125 H=2125:00 +CPU= 15.201
  21:06:05 STEP 2126 H=2126:00 +CPU= 18.411
  21:06:33 STEP 2127 H=2127:00 +CPU= 27.683
  21:06:49 STEP 2128 H=2128:00 +CPU= 15.389
  21:07:05 STEP 2129 H=2129:00 +CPU= 16.024
  21:07:34 STEP 2130 H=2130:00 +CPU= 29.051
  21:07:52 STEP 2131 H=2131:00 +CPU= 17.543
  21:08:10 STEP 2132 H=2132:00 +CPU= 17.592
  21:08:39 STEP 2133 H=2133:00 +CPU= 28.329
  21:08:56 STEP 2134 H=2134:00 +CPU= 17.214
  21:09:17 STEP 2135 H=2135:00 +CPU= 18.619
  21:10:12 STEP 2136 H=2136:00 +CPU= 32.743
Moving to projects directory: /var/lib/boinc/slots/40/ICMGGhpi1+002136
Moving to projects directory: /var/lib/boinc/slots/40/ICMSHhpi1+002136
Moving to projects directory: /var/lib/boinc/slots/40/ICMUAhpi1+002136
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMGGhpi1+002112
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMSHhpi1+002112
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMUAhpi1+002112
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMGGhpi1+002124
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMSHhpi1+002124
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMUAhpi1+002124
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_1683_2021050100_123_946_12164772_0_r592969078_12.zip
  21:10:29 STEP 2137 H=2137:00 +CPU= 17.569
  21:10:47 STEP 2138 H=2138:00 +CPU= 17.340
  21:11:15 STEP 2139 H=2139:00 +CPU= 28.016
  21:11:33 STEP 2140 H=2140:00 +CPU= 18.036
Uploading the intermediate file: upload_file_12.zip
  21:11:51 STEP 2141 H=2141:00 +CPU= 18.302
Uploading trickle at timestep: 7689600
  21:12:20 STEP 2142 H=2142:00 +CPU= 28.499
  21:12:38 STEP 2143 H=2143:00 +CPU= 17.922
  21:12:56 STEP 2144 H=2144:00 +CPU= 18.156
  21:13:24 STEP 2145 H=2145:00 +CPU= 27.386
  21:13:42 STEP 2146 H=2146:00 +CPU= 17.858
  21:14:00 STEP 2147 H=2147:00 +CPU= 17.951
  21:14:30 STEP 2148 H=2148:00 +CPU= 30.207
Moving to projects directory: /var/lib/boinc/slots/40/ICMGGhpi1+002148
Moving to projects directory: /var/lib/boinc/slots/40/ICMSHhpi1+002148
Moving to projects directory: /var/lib/boinc/slots/40/ICMUAhpi1+002148
  21:14:48 STEP 2149 H=2149:00 +CPU= 17.345
  21:15:06 STEP 2150 H=2150:00 +CPU= 18.415
  21:15:34 STEP 2151 H=2151:00 +CPU= 27.855
  21:15:52 STEP 2152 H=2152:00 +CPU= 17.234
  21:16:09 STEP 2153 H=2153:00 +CPU= 16.659
  21:16:37 STEP 2154 H=2154:00 +CPU= 28.482
  21:16:55 STEP 2155 H=2155:00 +CPU= 17.559
  21:17:14 STEP 2156 H=2156:00 +CPU= 18.630
  21:17:42 STEP 2157 H=2157:00 +CPU= 28.160
  21:18:01 STEP 2158 H=2158:00 +CPU= 18.273
  21:18:19 STEP 2159 H=2159:00 +CPU= 17.831
  21:18:56 STEP 2160 H=2160:00 +CPU= 32.529
Moving to projects directory: /var/lib/boinc/slots/40/ICMGGhpi1+002160
Moving to projects directory: /var/lib/boinc/slots/40/ICMSHhpi1+002160
Moving to projects directory: /var/lib/boinc/slots/40/ICMUAhpi1+002160
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMGGhpi1+002136
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMSHhpi1+002136
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMUAhpi1+002136
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMGGhpi1+002148
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMSHhpi1+002148
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMUAhpi1+002148
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_1683_2021050100_123_946_12164772_0_r592969078_13.zip
Uploading the intermediate file: upload_file_13.zip
  21:19:13 STEP 2161 H=2161:00 +CPU= 17.256
Uploading trickle at timestep: 7776000
  21:19:30 STEP 2162 H=2162:00 +CPU= 16.276
  21:19:57 STEP 2163 H=2163:00 +CPU= 27.065
  21:20:16 STEP 2164 H=2164:00 +CPU= 18.762
  21:20:34 STEP 2165 H=2165:00 +CPU= 17.658
  21:21:03 STEP 2166 H=2166:00 +CPU= 28.995
  21:21:21 STEP 2167 H=2167:00 +CPU= 17.808
  21:21:39 STEP 2168 H=2168:00 +CPU= 17.811
  21:22:06 STEP 2169 H=2169:00 +CPU= 27.324
  21:22:24 STEP 2170 H=2170:00 +CPU= 17.705
  21:22:42 STEP 2171 H=2171:00 +CPU= 17.911
  21:23:12 STEP 2172 H=2172:00 +CPU= 30.328
Moving to projects directory: /var/lib/boinc/slots/40/ICMGGhpi1+002172
Moving to projects directory: /var/lib/boinc/slots/40/ICMSHhpi1+002172
Moving to projects directory: /var/lib/boinc/slots/40/ICMUAhpi1+002172
  21:23:31 STEP 2173 H=2173:00 +CPU= 17.833
  21:23:48 STEP 2174 H=2174:00 +CPU= 17.873
  21:24:17 STEP 2175 H=2175:00 +CPU= 27.938
  21:24:36 STEP 2176 H=2176:00 +CPU= 18.942
  21:24:54 STEP 2177 H=2177:00 +CPU= 18.550
  21:25:22 STEP 2178 H=2178:00 +CPU= 27.873
  21:25:40 STEP 2179 H=2179:00 +CPU= 17.097
  21:25:57 STEP 2180 H=2180:00 +CPU= 17.311
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.01_x86_64-pc-linux-gnu
(argv1) start_date: 2021050100
(argv2) exptid: hpi1
(argv3) unique_member_id: 1683
(argv4) batchid: 946
(argv5) wuid: 12164772
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc/slots/40
Project directory is: /var/lib/boinc/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.01
Location of temp folder: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772
..mkdir for temp folder for results failed
Copying: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_app_1.01_x86_64-pc-linux-gnu.zip to: /var/lib/boinc/slots/40/oifs_43r3_ps_app_1.01_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc/slots/40/oifs_43r3_ps_app_1.01_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_5ba1e9f8b9fbf1bea9196537bb2387f1 to: /var/lib/boinc/slots/40/oifs_43r3_ps_1683_2021050100_123_946_12164772.zip
Unzipping the namelist zip file: /var/lib/boinc/slots/40/oifs_43r3_ps_1683_2021050100_123_946_12164772.zip
ic_ancil_file: ic_ancil_12164772
ifsdata_file: ifsdata_12164772
climate_data_file: clim_data_12164772
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
Copying IC ancils from: ../../projects/climateprediction.net/jf_86d8a9bb11002fce89519a65e26a69c6 to: /var/lib/boinc/slots/40/ic_ancil_12164772.zip
Unzipping the IC ancils zip file: /var/lib/boinc/slots/40/ic_ancil_12164772.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_922d18559d90fc947765e7eadc776432 to: /var/lib/boinc/slots/40/ifsdata/ifsdata_12164772.zip
Unzipping the ifsdata_zip file: /var/lib/boinc/slots/40/ifsdata/ifsdata_12164772.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc/slots/40/159l_2/clim_data_12164772.zip
Unzipping the climate data zip file: /var/lib/boinc/slots/40/159l_2/clim_data_12164772.zip
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_1683_2021050100_123_946_12164772_0_r592969078
The child process has been launched with process id: 275283
Executing the command: /var/lib/boinc/slots/40/./master.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:guoj-desktop:1:1:275283:275283] [20221203:212805:1670128085:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:guoj-desktop:1:1:275283:275283] [20221203:212805:1670128085:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:guoj-desktop:1:1:275283:275283] [20221203:212805:1670128085:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:guoj-desktop:1:1:275283:275283] [20221203:212805:1670128085:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:guoj-desktop:1:1:275283:275283] [20221203:212805:1670128085:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:guoj-desktop:1:1:275283:275283] [20221203:212805:1670128085:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:guoj-desktop:1:1:275283:275283] [20221203:212805:1670128085:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:guoj-desktop:1:1:275283:275283] [20221203:212805:1670128085:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:guoj-desktop:1:1:275283:275283] [20221203:212805:1670128085:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:guoj-desktop:1:1:275283:275283] [20221203:212805:1670128085:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:guoj-desktop:1:1:275283:275283] [20221203:212805:1670128085:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:guoj-desktop:1:1:275283:275283] [20221203:212805:1670128085:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:guoj-desktop:1:1:275283:275283] [20221203:212805:1670128085:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:guoj-desktop:1:1:275283:275283] [20221203:212805:1670128085:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:guoj-desktop:1:1:275283:275283] [20221203:212805:1670128085:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:guoj-desktop:1:1:275283:275283] [20221203:212805:1670128085:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:guoj-desktop:1:1:275283:275283] [20221203:212805:1670128085:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:guoj-desktop:1:1:275283:275283] [20221203:212805:1670128085:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:guoj-desktop:1:1:275283:275283] [20221203:212805:1670128085:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:guoj-desktop:1:1:275283:275283] [20221203:212805:1670128085:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:guoj-desktop:1:1:275283:275283] [20221203:212805:1670128085:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:guoj-desktop:1:1:275283:275283] [20221203:212805:1670128085:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:guoj-desktop:1:1:275283:275283] [20221203:212805:1670128085:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:guoj-desktop:1:1:275283:275283] [20221203:212805:1670128085:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:guoj-desktop:1:1:275283:275283] [20221203:212805:1670128085:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:guoj-desktop:1:1:275283:275283] [20221203:212805:1670128085:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:guoj-desktop:1:1:275283:275283] [20221203:212805:1670128085:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:guoj-desktop:1:1:275283:275283] [20221203:212805:1670128085:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:guoj-desktop:1:1:275283:275283] [20221203:212805:1670128085:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:guoj-desktop:1:1:275283:275283] [20221203:212805:1670128085:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  21:28:39 STEP 2160 H=2160:00 +CPU= 30.572
  21:28:56 STEP 2161 H=2161:00 +CPU= 16.711
  21:29:16 STEP 2162 H=2162:00 +CPU= 18.401
  21:29:45 STEP 2163 H=2163:00 +CPU= 27.896
  21:30:00 STEP 2164 H=2164:00 +CPU= 14.754
  21:30:16 STEP 2165 H=2165:00 +CPU= 15.343
  21:30:44 STEP 2166 H=2166:00 +CPU= 27.638
  21:31:02 STEP 2167 H=2167:00 +CPU= 17.720
  21:31:21 STEP 2168 H=2168:00 +CPU= 17.871
  21:31:48 STEP 2169 H=2169:00 +CPU= 26.788
  21:32:06 STEP 2170 H=2170:00 +CPU= 17.315
  21:32:23 STEP 2171 H=2171:00 +CPU= 16.822
  21:32:53 STEP 2172 H=2172:00 +CPU= 30.210
Moving to projects directory: /var/lib/boinc/slots/40/ICMGGhpi1+002172
Moving to projects directory: /var/lib/boinc/slots/40/ICMSHhpi1+002172
Moving to projects directory: /var/lib/boinc/slots/40/ICMUAhpi1+002172
  21:33:11 STEP 2173 H=2173:00 +CPU= 17.303
  21:33:29 STEP 2174 H=2174:00 +CPU= 17.516
  21:33:56 STEP 2175 H=2175:00 +CPU= 26.536
  21:34:14 STEP 2176 H=2176:00 +CPU= 17.808
  21:34:31 STEP 2177 H=2177:00 +CPU= 17.064
  21:35:00 STEP 2178 H=2178:00 +CPU= 27.946
  21:35:17 STEP 2179 H=2179:00 +CPU= 16.483
  21:35:37 STEP 2180 H=2180:00 +CPU= 18.206
  21:36:04 STEP 2181 H=2181:00 +CPU= 26.817
  21:36:23 STEP 2182 H=2182:00 +CPU= 18.067
  21:36:41 STEP 2183 H=2183:00 +CPU= 18.036
  21:37:15 STEP 2184 H=2184:00 +CPU= 32.376
Moving to projects directory: /var/lib/boinc/slots/40/ICMGGhpi1+002184
Moving to projects directory: /var/lib/boinc/slots/40/ICMSHhpi1+002184
Moving to projects directory: /var/lib/boinc/slots/40/ICMUAhpi1+002184
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMGGhpi1+002160
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMSHhpi1+002160
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMUAhpi1+002160
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMGGhpi1+002172
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMSHhpi1+002172
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMUAhpi1+002172
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_1683_2021050100_123_946_12164772_0_r592969078_1.zip
Uploading the intermediate file: upload_file_1.zip
  21:37:33 STEP 2185 H=2185:00 +CPU= 17.488
Uploading trickle at timestep: 7862400
  21:37:52 STEP 2186 H=2186:00 +CPU= 18.119
  21:38:17 STEP 2187 H=2187:00 +CPU= 23.518
  21:38:33 STEP 2188 H=2188:00 +CPU= 15.447
  21:38:47 STEP 2189 H=2189:00 +CPU= 13.887
  21:39:11 STEP 2190 H=2190:00 +CPU= 23.447
  21:39:26 STEP 2191 H=2191:00 +CPU= 15.121
  21:39:43 STEP 2192 H=2192:00 +CPU= 16.757
  21:40:11 STEP 2193 H=2193:00 +CPU= 27.507
  21:40:28 STEP 2194 H=2194:00 +CPU= 16.884
  21:40:45 STEP 2195 H=2195:00 +CPU= 16.192
  21:41:16 STEP 2196 H=2196:00 +CPU= 29.748
Moving to projects directory: /var/lib/boinc/slots/40/ICMGGhpi1+002196
Moving to projects directory: /var/lib/boinc/slots/40/ICMSHhpi1+002196
Moving to projects directory: /var/lib/boinc/slots/40/ICMUAhpi1+002196
  21:41:33 STEP 2197 H=2197:00 +CPU= 16.824
  21:41:50 STEP 2198 H=2198:00 +CPU= 16.936
  21:42:17 STEP 2199 H=2199:00 +CPU= 26.546
  21:42:36 STEP 2200 H=2200:00 +CPU= 18.017
  21:42:53 STEP 2201 H=2201:00 +CPU= 16.357
  21:43:21 STEP 2202 H=2202:00 +CPU= 27.675
  21:43:39 STEP 2203 H=2203:00 +CPU= 17.364
  21:43:56 STEP 2204 H=2204:00 +CPU= 16.743
  21:44:24 STEP 2205 H=2205:00 +CPU= 27.020
  21:44:41 STEP 2206 H=2206:00 +CPU= 17.468
  21:45:00 STEP 2207 H=2207:00 +CPU= 17.466
  21:46:10 STEP 2208 H=2208:00 +CPU= 30.929
Moving to projects directory: /var/lib/boinc/slots/40/ICMGGhpi1+002208
Moving to projects directory: /var/lib/boinc/slots/40/ICMSHhpi1+002208
Moving to projects directory: /var/lib/boinc/slots/40/ICMUAhpi1+002208
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMGGhpi1+002184
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMSHhpi1+002184
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMUAhpi1+002184
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMGGhpi1+002196
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMSHhpi1+002196
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMUAhpi1+002196
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_1683_2021050100_123_946_12164772_0_r592969078_2.zip
Uploading the intermediate file: upload_file_2.zip
  21:46:26 STEP 2209 H=2209:00 +CPU= 15.618
Uploading trickle at timestep: 7948800
  21:46:43 STEP 2210 H=2210:00 +CPU= 16.577
  21:47:10 STEP 2211 H=2211:00 +CPU= 26.614
  21:47:28 STEP 2212 H=2212:00 +CPU= 18.084
  21:47:45 STEP 2213 H=2213:00 +CPU= 16.604
  21:48:12 STEP 2214 H=2214:00 +CPU= 26.981
  21:48:29 STEP 2215 H=2215:00 +CPU= 17.085
  21:48:46 STEP 2216 H=2216:00 +CPU= 16.342
  21:49:13 STEP 2217 H=2217:00 +CPU= 27.219
  21:49:31 STEP 2218 H=2218:00 +CPU= 17.914
  21:49:48 STEP 2219 H=2219:00 +CPU= 16.956
  21:50:20 STEP 2220 H=2220:00 +CPU= 30.118
Moving to projects directory: /var/lib/boinc/slots/40/ICMGGhpi1+002220
Moving to projects directory: /var/lib/boinc/slots/40/ICMSHhpi1+002220
Moving to projects directory: /var/lib/boinc/slots/40/ICMUAhpi1+002220
  21:50:39 STEP 2221 H=2221:00 +CPU= 18.391
  21:50:57 STEP 2222 H=2222:00 +CPU= 17.710
  21:51:25 STEP 2223 H=2223:00 +CPU= 27.504
  21:51:44 STEP 2224 H=2224:00 +CPU= 18.144
  21:52:01 STEP 2225 H=2225:00 +CPU= 16.863
  21:52:28 STEP 2226 H=2226:00 +CPU= 26.361
  21:52:44 STEP 2227 H=2227:00 +CPU= 15.586
  21:53:01 STEP 2228 H=2228:00 +CPU= 17.587
  21:53:26 STEP 2229 H=2229:00 +CPU= 25.073
  21:53:43 STEP 2230 H=2230:00 +CPU= 16.061
  21:53:59 STEP 2231 H=2231:00 +CPU= 16.706
  21:54:35 STEP 2232 H=2232:00 +CPU= 32.483
Moving to projects directory: /var/lib/boinc/slots/40/ICMGGhpi1+002232
Moving to projects directory: /var/lib/boinc/slots/40/ICMSHhpi1+002232
Moving to projects directory: /var/lib/boinc/slots/40/ICMUAhpi1+002232
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMGGhpi1+002208
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMSHhpi1+002208
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMUAhpi1+002208
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMGGhpi1+002220
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMSHhpi1+002220
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMUAhpi1+002220
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_1683_2021050100_123_946_12164772_0_r592969078_3.zip
Uploading the intermediate file: upload_file_3.zip
  21:54:53 STEP 2233 H=2233:00 +CPU= 17.840
  21:55:10 STEP 2234 H=2234:00 +CPU= 17.041
Uploading trickle at timestep: 8035200
  21:55:37 STEP 2235 H=2235:00 +CPU= 26.743
  21:55:53 STEP 2236 H=2236:00 +CPU= 16.228
  21:56:07 STEP 2237 H=2237:00 +CPU= 13.365
  21:56:33 STEP 2238 H=2238:00 +CPU= 25.816
  21:56:50 STEP 2239 H=2239:00 +CPU= 17.446
  21:57:08 STEP 2240 H=2240:00 +CPU= 17.436
  21:57:35 STEP 2241 H=2241:00 +CPU= 27.063
  21:57:53 STEP 2242 H=2242:00 +CPU= 17.947
  21:58:10 STEP 2243 H=2243:00 +CPU= 16.943
  21:58:41 STEP 2244 H=2244:00 +CPU= 30.420
Moving to projects directory: /var/lib/boinc/slots/40/ICMGGhpi1+002244
Moving to projects directory: /var/lib/boinc/slots/40/ICMSHhpi1+002244
Moving to projects directory: /var/lib/boinc/slots/40/ICMUAhpi1+002244
  21:58:58 STEP 2245 H=2245:00 +CPU= 17.387
  21:59:17 STEP 2246 H=2246:00 +CPU= 18.083
  21:59:43 STEP 2247 H=2247:00 +CPU= 26.006
  22:00:02 STEP 2248 H=2248:00 +CPU= 18.465
  22:00:17 STEP 2249 H=2249:00 +CPU= 15.037
  22:00:46 STEP 2250 H=2250:00 +CPU= 28.422
  22:01:04 STEP 2251 H=2251:00 +CPU= 17.289
  22:01:21 STEP 2252 H=2252:00 +CPU= 17.090
  22:01:49 STEP 2253 H=2253:00 +CPU= 27.325
  22:02:07 STEP 2254 H=2254:00 +CPU= 17.605
  22:02:24 STEP 2255 H=2255:00 +CPU= 16.334
  22:03:00 STEP 2256 H=2256:00 +CPU= 31.754
Moving to projects directory: /var/lib/boinc/slots/40/ICMGGhpi1+002256
Moving to projects directory: /var/lib/boinc/slots/40/ICMSHhpi1+002256
Moving to projects directory: /var/lib/boinc/slots/40/ICMUAhpi1+002256
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMGGhpi1+002232
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMSHhpi1+002232
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMUAhpi1+002232
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMGGhpi1+002244
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMSHhpi1+002244
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772/ICMUAhpi1+002244
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_1683_2021050100_123_946_12164772_0_r592969078_4.zip
Uploading the intermediate file: upload_file_4.zip
  22:03:17 STEP 2257 H=2257:00 +CPU= 16.836
Uploading trickle at timestep: 8121600
  22:03:34 STEP 2258 H=2258:00 +CPU= 17.362
  22:04:01 STEP 2259 H=2259:00 +CPU= 26.463
  22:04:18 STEP 2260 H=2260:00 +CPU= 17.098
  22:04:34 STEP 2261 H=2261:00 +CPU= 16.056
  22:05:02 STEP 2262 H=2262:00 +CPU= 27.169
  22:05:19 STEP 2263 H=2263:00 +CPU= 17.281
  22:05:38 STEP 2264 H=2264:00 +CPU= 18.785
  22:06:06 STEP 2265 H=2265:00 +CPU= 27.213
  22:06:24 STEP 2266 H=2266:00 +CPU= 18.126
  22:06:41 STEP 2267 H=2267:00 +CPU= 16.942
  22:07:13 STEP 2268 H=2268:00 +CPU= 31.208
Moving to projects directory: /var/lib/boinc/slots/40/ICMGGhpi1+002268
Moving to projects directory: /var/lib/boinc/slots/40/ICMSHhpi1+002268
Moving to projects directory: /var/lib/boinc/slots/40/ICMUAhpi1+002268
  22:07:30 STEP 2269 H=2269:00 +CPU= 16.841
  22:07:47 STEP 2270 H=2270:00 +CPU= 17.468
  22:08:13 STEP 2271 H=2271:00 +CPU= 25.659
  22:08:32 STEP 2272 H=2272:00 +CPU= 18.680
  22:08:50 STEP 2273 H=2273:00 +CPU= 17.368
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.01_x86_64-pc-linux-gnu
(argv1) start_date: 2021050100
(argv2) exptid: hpi1
(argv3) unique_member_id: 1683
(argv4) batchid: 946
(argv5) wuid: 12164772
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc/slots/40
Project directory is: /var/lib/boinc/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.01
Location of temp folder: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12164772
..mkdir for temp folder for results failed
Copying: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_app_1.01_x86_64-pc-linux-gnu.zip to: /var/lib/boinc/slots/40/oifs_43r3_ps_app_1.01_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc/slots/40/oifs_43r3_ps_app_1.01_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_5ba1e9f8b9fbf1bea9196537bb2387f1 to: /var/lib/boinc/slots/40/oifs_43r3_ps_1683_2021050100_123_946_12164772.zip
Unzipping the namelist zip file: /var/lib/boinc/slots/40/oifs_43r3_ps_1683_2021050100_123_946_12164772.zip
ic_ancil_file: ic_ancil_12164772
ifsdata_file: ifsdata_12164772
climate_data_file: clim_data_12164772
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
Copying IC ancils from: ../../projects/climateprediction.net/jf_86d8a9bb11002fce89519a65e26a69c6 to: /var/lib/boinc/slots/40/ic_ancil_12164772.zip
Unzipping the IC ancils zip file: /var/lib/boinc/slots/40/ic_ancil_12164772.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_922d18559d90fc947765e7eadc776432 to: /var/lib/boinc/slots/40/ifsdata/ifsdata_12164772.zip

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 May 2023 15:32:53 1524992 22246771 oifs_43r3_ps_1683_2021050100_123_946_12164772_0 8,121,600 2,012 0.0002
09 May 2023 15:32:54 1524992 22246771 oifs_43r3_ps_1683_2021050100_123_946_12164772_0 8,035,200 1,516 0.0002
09 May 2023 15:32:53 1524992 22246771 oifs_43r3_ps_1683_2021050100_123_946_12164772_0 7,948,800 1,022 0.0001
09 May 2023 15:32:53 1524992 22246771 oifs_43r3_ps_1683_2021050100_123_946_12164772_0 7,862,400 540 0.0001
09 May 2023 15:32:46 1524992 22246771 oifs_43r3_ps_1683_2021050100_123_946_12164772_0 7,776,000 6,580 0.0008
09 May 2023 15:32:46 1524992 22246771 oifs_43r3_ps_1683_2021050100_123_946_12164772_0 7,689,600 6,060 0.0008
09 May 2023 15:32:46 1524992 22246771 oifs_43r3_ps_1683_2021050100_123_946_12164772_0 7,603,200 5,556 0.0007
09 May 2023 15:32:46 1524992 22246771 oifs_43r3_ps_1683_2021050100_123_946_12164772_0 7,516,800 5,048 0.0007
09 May 2023 15:32:46 1524992 22246771 oifs_43r3_ps_1683_2021050100_123_946_12164772_0 7,430,400 4,548 0.0006
09 May 2023 15:32:46 1524992 22246771 oifs_43r3_ps_1683_2021050100_123_946_12164772_0 7,344,000 4,044 0.0006
09 May 2023 15:32:46 1524992 22246771 oifs_43r3_ps_1683_2021050100_123_946_12164772_0 7,257,600 3,545 0.0005
09 May 2023 15:32:36 1524992 22246771 oifs_43r3_ps_1683_2021050100_123_946_12164772_0 7,171,200 3,035 0.0004
09 May 2023 15:32:36 1524992 22246771 oifs_43r3_ps_1683_2021050100_123_946_12164772_0 7,084,800 2,544 0.0004
09 May 2023 15:32:36 1524992 22246771 oifs_43r3_ps_1683_2021050100_123_946_12164772_0 6,998,400 2,039 0.0003
09 May 2023 15:32:36 1524992 22246771 oifs_43r3_ps_1683_2021050100_123_946_12164772_0 6,912,000 1,547 0.0002
09 May 2023 15:32:36 1524992 22246771 oifs_43r3_ps_1683_2021050100_123_946_12164772_0 6,825,600 1,055 0.0002
09 May 2023 15:32:37 1524992 22246771 oifs_43r3_ps_1683_2021050100_123_946_12164772_0 6,739,200 542 0.0001
09 May 2023 15:32:29 1524992 22246771 oifs_43r3_ps_1683_2021050100_123_946_12164772_0 6,652,800 6,634 0.0010
09 May 2023 15:32:29 1524992 22246771 oifs_43r3_ps_1683_2021050100_123_946_12164772_0 6,566,400 6,108 0.0009
09 May 2023 15:32:29 1524992 22246771 oifs_43r3_ps_1683_2021050100_123_946_12164772_0 6,480,000 5,595 0.0009


©2024 cpdn.org