climateprediction.net home page
Task 22306954

Task 22306954

Name oifs_43r3_ps_2566_2021050100_123_990_12206767_1
Workunit 12206767
Created 11 Feb 2023, 20:08:36 UTC
Sent 11 Feb 2023, 20:11:21 UTC
Report deadline 12 Apr 2023, 20:11:21 UTC
Received 12 Feb 2023, 15:16:06 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 9 (0x00000009) Unknown error code
Computer ID 1519861
Run time 15 hours 26 min 15 sec
CPU time 15 hours 26 min 15 sec
Validate state Invalid
Credit 0.00
Device peak FLOPS 4.57 GFLOPS
Application version OpenIFS 43r3 Perturbed Surface v1.09
x86_64-pc-linux-gnu
Peak working set size 4,409.62 MB
Peak swap size 4,978.13 MB
Peak disk usage 1,485.57 MB
Stderr 
<core_client_version>7.14.2</core_client_version>
<![CDATA[
<message>
process exited with code 9 (0x9, -247)</message>
<stderr_txt>
 STEP 1918 H=1918:00 +CPU= 19.855
  11:30:39 STEP 1919 H=1919:00 +CPU= 20.117
  11:31:18 STEP 1920 H=1920:00 +CPU= 38.063
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhpi1+001920
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhpi1+001920
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhpi1+001920
  11:31:38 STEP 1921 H=1921:00 +CPU= 20.389
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206767/ICMGGhpi1+001908
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206767/ICMSHhpi1+001908
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206767/ICMUAhpi1+001908
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206767/ICMGGhpi1+001920
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206767/ICMSHhpi1+001920
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206767/ICMUAhpi1+001920
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_2566_2021050100_123_990_12206767_1_r1289224932_79.zip
Uploading the intermediate file: upload_file_79.zip
  11:32:04 STEP 1922 H=1922:00 +CPU= 25.789
  11:32:40 STEP 1923 H=1923:00 +CPU= 35.776
  11:33:09 STEP 1924 H=1924:00 +CPU= 29.291
  11:33:32 STEP 1925 H=1925:00 +CPU= 23.013
  11:34:05 STEP 1926 H=1926:00 +CPU= 32.485
  11:34:25 STEP 1927 H=1927:00 +CPU= 20.065
  11:34:45 STEP 1928 H=1928:00 +CPU= 20.667
  11:35:16 STEP 1929 H=1929:00 +CPU= 30.960
  11:35:36 STEP 1930 H=1930:00 +CPU= 20.028
  11:35:56 STEP 1931 H=1931:00 +CPU= 19.590
  11:36:31 STEP 1932 H=1932:00 +CPU= 35.504
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhpi1+001932
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhpi1+001932
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhpi1+001932
  11:36:51 STEP 1933 H=1933:00 +CPU= 20.052
  11:37:13 STEP 1934 H=1934:00 +CPU= 21.859
  11:37:45 STEP 1935 H=1935:00 +CPU= 31.906
  11:38:05 STEP 1936 H=1936:00 +CPU= 19.629
  11:38:24 STEP 1937 H=1937:00 +CPU= 18.862
  11:38:54 STEP 1938 H=1938:00 +CPU= 30.582
  11:39:14 STEP 1939 H=1939:00 +CPU= 20.162
  11:39:35 STEP 1940 H=1940:00 +CPU= 20.732
  11:40:07 STEP 1941 H=1941:00 +CPU= 31.452
  11:40:26 STEP 1942 H=1942:00 +CPU= 19.371
  11:40:45 STEP 1943 H=1943:00 +CPU= 19.417
  11:41:25 STEP 1944 H=1944:00 +CPU= 38.272
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhpi1+001944
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhpi1+001944
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhpi1+001944
  11:41:46 STEP 1945 H=1945:00 +CPU= 20.159
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206767/ICMGGhpi1+001932
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206767/ICMSHhpi1+001932
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206767/ICMUAhpi1+001932
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206767/ICMGGhpi1+001944
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206767/ICMSHhpi1+001944
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206767/ICMUAhpi1+001944
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_2566_2021050100_123_990_12206767_1_r1289224932_80.zip
Uploading the intermediate file: upload_file_80.zip
  11:42:07 STEP 1946 H=1946:00 +CPU= 21.200
  11:42:38 STEP 1947 H=1947:00 +CPU= 30.985
  11:42:58 STEP 1948 H=1948:00 +CPU= 20.128
  11:43:18 STEP 1949 H=1949:00 +CPU= 20.178
  11:43:50 STEP 1950 H=1950:00 +CPU= 31.701
  11:44:10 STEP 1951 H=1951:00 +CPU= 19.846
  11:44:30 STEP 1952 H=1952:00 +CPU= 20.818
  11:45:02 STEP 1953 H=1953:00 +CPU= 32.001
  11:45:23 STEP 1954 H=1954:00 +CPU= 20.778
  11:45:43 STEP 1955 H=1955:00 +CPU= 20.157
  11:46:19 STEP 1956 H=1956:00 +CPU= 35.571
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhpi1+001956
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhpi1+001956
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhpi1+001956
  11:46:39 STEP 1957 H=1957:00 +CPU= 20.235
  11:47:00 STEP 1958 H=1958:00 +CPU= 20.769
  11:47:32 STEP 1959 H=1959:00 +CPU= 32.022
  11:47:52 STEP 1960 H=1960:00 +CPU= 19.969
  11:48:11 STEP 1961 H=1961:00 +CPU= 18.961
  11:48:42 STEP 1962 H=1962:00 +CPU= 30.723
  11:49:02 STEP 1963 H=1963:00 +CPU= 20.093
  11:49:22 STEP 1964 H=1964:00 +CPU= 20.672
  11:49:54 STEP 1965 H=1965:00 +CPU= 31.292
  11:50:13 STEP 1966 H=1966:00 +CPU= 19.542
  11:50:33 STEP 1967 H=1967:00 +CPU= 19.394
  11:51:09 STEP 1968 H=1968:00 +CPU= 36.341
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhpi1+001968
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhpi1+001968
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhpi1+001968
  11:51:29 STEP 1969 H=1969:00 +CPU= 19.548
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206767/ICMGGhpi1+001956
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206767/ICMSHhpi1+001956
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206767/ICMUAhpi1+001956
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206767/ICMGGhpi1+001968
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206767/ICMSHhpi1+001968
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206767/ICMUAhpi1+001968
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_2566_2021050100_123_990_12206767_1_r1289224932_81.zip
Uploading the intermediate file: upload_file_81.zip
  11:51:49 STEP 1970 H=1970:00 +CPU= 20.047
  11:52:20 STEP 1971 H=1971:00 +CPU= 30.780
  11:52:39 STEP 1972 H=1972:00 +CPU= 19.367
  11:52:58 STEP 1973 H=1973:00 +CPU= 18.828
  11:53:28 STEP 1974 H=1974:00 +CPU= 30.508
  11:53:47 STEP 1975 H=1975:00 +CPU= 18.845
  11:54:07 STEP 1976 H=1976:00 +CPU= 19.942
  11:54:38 STEP 1977 H=1977:00 +CPU= 30.854
  11:54:58 STEP 1978 H=1978:00 +CPU= 20.028
  11:55:18 STEP 1979 H=1979:00 +CPU= 19.450
  11:55:52 STEP 1980 H=1980:00 +CPU= 34.345
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhpi1+001980
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhpi1+001980
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhpi1+001980
  11:56:12 STEP 1981 H=1981:00 +CPU= 20.363
  11:56:33 STEP 1982 H=1982:00 +CPU= 20.926
  11:57:05 STEP 1983 H=1983:00 +CPU= 32.044
  11:57:26 STEP 1984 H=1984:00 +CPU= 20.670
  11:57:46 STEP 1985 H=1985:00 +CPU= 20.297
  11:58:18 STEP 1986 H=1986:00 +CPU= 32.046
  11:58:37 STEP 1987 H=1987:00 +CPU= 19.014
  11:58:57 STEP 1988 H=1988:00 +CPU= 19.410
  11:59:27 STEP 1989 H=1989:00 +CPU= 29.950
  11:59:47 STEP 1990 H=1990:00 +CPU= 20.663
  12:00:08 STEP 1991 H=1991:00 +CPU= 20.721
Quit request received from BOINC client, ending the child process
>>> Printing last 70 lines from file: NODE.001_01

         AVE   0.426167322903042E-05 0.000000000000000E+00 0.194080236065611E-02
 GPNORM CIW                  AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.119725460525452E-05 0.000000000000000E+00 0.645942508331993E-03
 GPNORM CC                   AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.584564870543792E-01 0.000000000000000E+00 0.100000000000000E+01
 GPNORM RFL                  AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.282067764925015E-05 0.000000000000000E+00 0.123803974076810E-02
 GPNORM SFL                  AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.176879921573951E-05 0.000000000000000E+00 0.719851959760494E-03
 GPNORM CRFL                 AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.574041167303852E-05 0.000000000000000E+00 0.281418487613872E-02
 GPNORM CSFL                 AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.177004649257888E-05 0.000000000000000E+00 0.270441173889636E-02
 GPNORM PFRC                 AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.129831981797328E+00 0.000000000000000E+00 0.100000000000000E+01
 NSTEP =  1992 STEPO   F00000000
 CREATE RESTART FILE ./srf00830000.0001                                                                                                                                                                                                                                              
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ./srf00830000.0001 MODE=w
 IO-STREAM CLOSED - ./srf00830000.0001
 MPI-TASK:   1 - ************ BYTES IN     35 RECORDS TRANSFERRED IN    1.8548 SECONDS******** Mbytes/s, TOTAL TIME=    2.4291(76.4%)
 CREATE RESTART CONTROL FILE ./rcf                                                                                                                   
 WRRESF: DELETED PREVIOUS RESTART FILE OK: ./srf00820000.0001
  MAXGPFV : MAX. VALUE =    1.11969374689931     
  MAXGPFV : MAX. VALUE =   0.705746265519531     
 NSTEP =  1992 SCAN2M_HPOS  P
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ICMGGhpi1+001992 MODE=w
 IO-STREAM CLOSED - ICMGGhpi1+001992
 MPI-TASK:   1 -    725090539 BYTES IN     11 RECORDS TRANSFERRED IN    0.0007 SECONDS******** Mbytes/s, TOTAL TIME=    0.0138( 5.1%)
 NSTEP =  1992 STEPO   0AA000000
  SPECTRAL NORMS: Pattern  1
 STEPO, YSPSDT_AR1     (  1,...), SP NORMS 2D: 5.206765592838E-01 1.985950154676E-01 5.739455683553E-02
 STEPO, YSPSDT_AR1     (SUM,...), SP NORMS 2D: 5.628555318770E-01
 STEPO, YGPSDT         (  1,...), Grid Point Norms
  FIELD LEV        AVERAGE               MINIMUM               MAXIMUM       
    101  60 -2.34199275517680E-02 -2.24432261241609E+00  2.01966425542127E+00
 STEPO, YGPSDT[clipped](  1,...), Grid Point Norms
  FIELD LEV        AVERAGE               MINIMUM               MAXIMUM       
    101  60 -1.85335572311133E-02 -1.00000000000000E+00  1.00000000000000E+00
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ./ICMSHhpi1+001992 MODE=a
 IO-STREAM CLOSED - ./ICMSHhpi1+001992
 MPI-TASK:   1 -   7892750722 BYTES IN    114 RECORDS TRANSFERRED IN    0.0048 SECONDS******** Mbytes/s, TOTAL TIME=    0.1222( 3.9%)
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ./ICMUAhpi1+001992 MODE=a
 IO-STREAM CLOSED - ./ICMUAhpi1+001992
 MPI-TASK:   1 -   1087635813 BYTES IN     19 RECORDS TRANSFERRED IN    0.0011 SECONDS988759.8 Mbytes/s, TOTAL TIME=    0.0202( 5.4%)
 STEPO_FPOS WAS : B00000F00E
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ./ICMSHhpi1+001992 MODE=a
 IO-STREAM CLOSED - ./ICMSHhpi1+001992
 MPI-TASK:   1 -            0 BYTES IN      1 RECORDS TRANSFERRED IN    0.0001 SECONDS     0.0 Mbytes/s, TOTAL TIME=    0.0018( 5.6%)
 STEPO_FPOS WAS : V00000F00Y
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ./ICMUAhpi1+001992 MODE=a
 IO-STREAM CLOSED - ./ICMUAhpi1+001992
 MPI-TASK:   1 -    354012489 BYTES IN      2 RECORDS TRANSFERRED IN    0.0002 SECONDS******** Mbytes/s, TOTAL TIME=    0.0033( 6.1%)
 STEPO_FPOS WAS : T00000000M
 NSTEP =  1992 STEPO   0AAA00AAA
  SPECTRAL NORMS: Pattern  1
 STEPO, YSPSDT_AR1     (  1,...), SP NORMS 2D: 5.206765592838E-01 1.985950154676E-01 5.739455683553E-02
 STEPO, YSPSDT_AR1     (SUM,...), SP NORMS 2D: 5.628555318770E-01
 STEPO, YGPSDT         (  1,...), Grid Point Norms
  FIELD LEV        AVERAGE               MINIMUM               MAXIMUM       
    101  60 -2.34199275517680E-02 -2.24432261241609E+00  2.01966425542127E+00
 STEPO, YGPSDT[clipped](  1,...), Grid Point Norms
  FIELD LEV        AVERAGE               MINIMUM               MAXIMUM       
    101  60 -1.85335572311133E-02 -1.00000000000000E+00  1.00000000000000E+00
------------------------------------------------
CNT0 not found; string returned was: 'STEPO'
>>> Printing last 8 lines from file: ifs.stat

 11:59:47 0AAA00AAA STEPO     1991    20.663   20.663   20.664     67:52     67:59 0.12707801855771E-04       2GB       0MB
 12:00:08 F00000000 STEPO     1992    20.934   20.934   21.136     68:13     68:20 0.12305763175660E-04       2GB       0MB
 12:00:11 0AA000000 STEPO     1992     2.697    2.697    2.697     68:15     68:23 0.12305763175660E-04       2GB       0MB
 12:00:12 FULLPOS-B DYNFPOS   1992     0.941    0.941    0.943     68:16     68:24 0.12305763175660E-04       2GB       0MB
 12:00:13 FULLPOS-V DYNFPOS   1992     1.221    1.221    1.222     68:18     68:25 0.12305763175660E-04       2GB       0MB
 12:00:14 FULLPOS-T DYNFPOS   1992     0.339    0.339    0.340     68:18     68:26 0.12305763175660E-04       2GB       0MB
 12:00:14 0AAA00AAA STEPO     1992     0.351    0.351    0.351     68:18     68:26 0.12305763175660E-04       2GB       0MB
------------------------------------------------
>>> Printing last 11 lines from file: rcf

 CSTEP   = "    1992",
 CTIME   = "00830000      ",
 NSTEPLPP        =         201,         202,          49,      228226,      228227, 5*0,
 IPRGPNSRES      =           1,
 IPRGPEWRES      =           1,
 IPRTRWRES       =           1,
 IPRTRVRES       =           1,
 GMASS0  =   98331.1192541930     ,
 GMASSI  =   98330.8726948463     
 /
------------------------------------------------
>>> Printing last 17 lines from file: waminfo

PARAM=10U/10V,                                                          
REPRES=GG,                                                              
ANALYSIS PERIOD =       0                                               
FORECAST PERIOD =   10022400                                            
WIND TIME STEP IN SECONDS =    3600                                     
ANALYSIS FROM 20210501000000 TO 20210501000000                          
ASSIMILATION NO                                                         
NEW FORECAST NO                                                         
STATUS STORM1= UNFINISHED                                               
STATUS STORM2= UNFINISHED                                               
STATUS STORM3= UNFINISHED                                               
DATE FOR OUTPUT OF BOTH RESTART FILES = 00000000000000                  
LAST DATE FOR SPECTRA FILE OUTPUT = 20210901000000                      
BEGIN DATE FOR USING SURFACE CURRENT = 20210501000000                   
PARALLEL RESTART = YES                                                  
NUMBER OF MPI TASKS USED =          1                                   
------------------------------------------------
>>> Printing last 8 lines from file: /var/lib/boinc-client/slots/8/progress_file_12206767.xml

<running_values>
  <last_cpu_time>48281.280000</last_cpu_time>
  <upload_file_number>82</upload_file_number>
  <last_iter>1992</last_iter>
  <last_upload>7088400</last_upload>
  <model_completed>0</model_completed>
</running_values>
------------------------------------------------
..Failed, model did not complete successfully
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.09_x86_64-pc-linux-gnu
(argv1) start_date: 2021050100
(argv2) exptid: hpi1
(argv3) unique_member_id: 2566
(argv4) batchid: 990
(argv5) wuid: 12206767
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/8
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.09
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206767
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.09_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/8/oifs_43r3_ps_app_1.09_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/8/oifs_43r3_ps_app_1.09_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_6674f8b38834b48117e4123f90b2a295 to: /var/lib/boinc-client/slots/8/oifs_43r3_ps_2566_2021050100_123_990_12206767.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/8/oifs_43r3_ps_2566_2021050100_123_990_12206767.zip
ic_ancil_file: ic_ancil_12206767
ifsdata_file: ifsdata_12206767
climate_data_file: clim_data_12206767
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_6536e9571c94503e1d63b5bf9813f917 to: /var/lib/boinc-client/slots/8/ic_ancil_12206767.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/8/ic_ancil_12206767.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_bb811feb435cbb58bdc77f16003530ae to: /var/lib/boinc-client/slots/8/ifsdata/ifsdata_12206767.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/8/ifsdata/ifsdata_12206767.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/8/159l_2/clim_data_12206767.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/8/159l_2/clim_data_12206767.zip
Checking for progress XML file: /var/lib/boinc-client/slots/8/progress_file_12206767.xml
Opened progress file ok : /var/lib/boinc-client/slots/8/progress_file_12206767.xml
-- Model is restarting --
Adjusting last_iter, 1992, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/8/progress_file_12206767.xml
last_cpu_time: 48281
upload_file_number: 82
last_iter: 1993
last_upload: 7088400
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_2566_2021050100_123_990_12206767_1_r1289224932
The child process has been launched with process id: 32333
Executing the command: /var/lib/boinc-client/slots/8/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:prj-boinc:1:1:32333:32333] [20230212:120143:1676203303:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fa0
[EC_DRHOOK:prj-boinc:1:1:32333:32333] [20230212:120143:1676203303:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:prj-boinc:1:1:32333:32333] [20230212:120143:1676203303:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:prj-boinc:1:1:32333:32333] [20230212:120143:1676203303:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:prj-boinc:1:1:32333:32333] [20230212:120143:1676203303:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:prj-boinc:1:1:32333:32333] [20230212:120143:1676203303:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:prj-boinc:1:1:32333:32333] [20230212:120143:1676203303:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:prj-boinc:1:1:32333:32333] [20230212:120143:1676203303:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:prj-boinc:1:1:32333:32333] [20230212:120143:1676203303:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:prj-boinc:1:1:32333:32333] [20230212:120143:1676203303:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:prj-boinc:1:1:32333:32333] [20230212:120143:1676203303:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:prj-boinc:1:1:32333:32333] [20230212:120143:1676203303:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:prj-boinc:1:1:32333:32333] [20230212:120143:1676203303:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:prj-boinc:1:1:32333:32333] [20230212:120143:1676203303:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:prj-boinc:1:1:32333:32333] [20230212:120143:1676203303:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:prj-boinc:1:1:32333:32333] [20230212:120143:1676203303:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x815590 (old at 0x1ce8cf0)
[EC_DRHOOK:prj-boinc:1:1:32333:32333] [20230212:120143:1676203303:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x815590 (old at (nil))
[EC_DRHOOK:prj-boinc:1:1:32333:32333] [20230212:120143:1676203303:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x815590 (old at 0x1ce8cf0)
[EC_DRHOOK:prj-boinc:1:1:32333:32333] [20230212:120143:1676203303:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x815590 (old at (nil))
[EC_DRHOOK:prj-boinc:1:1:32333:32333] [20230212:120143:1676203303:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:prj-boinc:1:1:32333:32333] [20230212:120143:1676203303:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x815590 (old at 0x1ce8cf0)
[EC_DRHOOK:prj-boinc:1:1:32333:32333] [20230212:120143:1676203303:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x815590 (old at 0x1ce8cf0)
[EC_DRHOOK:prj-boinc:1:1:32333:32333] [20230212:120143:1676203303:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x815590 (old at (nil))
[EC_DRHOOK:prj-boinc:1:1:32333:32333] [20230212:120143:1676203303:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x815590 (old at 0x1ce8cf0)
[EC_DRHOOK:prj-boinc:1:1:32333:32333] [20230212:120143:1676203303:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x815590 (old at 0x1ce8cf0)
[EC_DRHOOK:prj-boinc:1:1:32333:32333] [20230212:120143:1676203303:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x815590 (old at 0x1ce8cf0)
[EC_DRHOOK:prj-boinc:1:1:32333:32333] [20230212:120143:1676203303:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x815590 (old at (nil))
[EC_DRHOOK:prj-boinc:1:1:32333:32333] [20230212:120143:1676203303:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x815590 (old at (nil))
[EC_DRHOOK:prj-boinc:1:1:32333:32333] [20230212:120143:1676203303:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x815590 (old at (nil))
[EC_DRHOOK:prj-boinc:1:1:32333:32333] [20230212:120143:1676203303:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206767/ICMGGhpi1+001980
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206767/ICMSHhpi1+001980
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206767/ICMUAhpi1+001980
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_2566_2021050100_123_990_12206767_1_r1289224932_82.zip
Uploading the intermediate file: upload_file_82.zip
  12:02:35 STEP 1992 H=1992:00 +CPU= 45.568
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhpi1+001992
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhpi1+001992
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhpi1+001992
  12:02:56 STEP 1993 H=1993:00 +CPU= 20.316
  12:03:16 STEP 1994 H=1994:00 +CPU= 20.014
  12:03:46 STEP 1995 H=1995:00 +CPU= 30.432
  12:04:06 STEP 1996 H=1996:00 +CPU= 19.792
  12:04:25 STEP 1997 H=1997:00 +CPU= 18.829
  12:04:55 STEP 1998 H=1998:00 +CPU= 30.363
  12:05:14 STEP 1999 H=1999:00 +CPU= 18.884
  12:05:34 STEP 2000 H=2000:00 +CPU= 19.412
  12:06:03 STEP 2001 H=2001:00 +CPU= 29.767
  12:06:23 STEP 2002 H=2002:00 +CPU= 19.515
  12:06:43 STEP 2003 H=2003:00 +CPU= 20.113
  12:07:19 STEP 2004 H=2004:00 +CPU= 35.376
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhpi1+002004
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhpi1+002004
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhpi1+002004
  12:07:39 STEP 2005 H=2005:00 +CPU= 20.535
  12:08:00 STEP 2006 H=2006:00 +CPU= 20.656
  12:08:31 STEP 2007 H=2007:00 +CPU= 31.411
  12:08:51 STEP 2008 H=2008:00 +CPU= 20.050
  12:09:11 STEP 2009 H=2009:00 +CPU= 19.485
  12:09:43 STEP 2010 H=2010:00 +CPU= 32.024
  12:10:03 STEP 2011 H=2011:00 +CPU= 20.064
  12:10:24 STEP 2012 H=2012:00 +CPU= 20.643
  12:10:55 STEP 2013 H=2013:00 +CPU= 31.273
  12:11:15 STEP 2014 H=2014:00 +CPU= 20.044
  12:11:35 STEP 2015 H=2015:00 +CPU= 20.033
  12:12:14 STEP 2016 H=2016:00 +CPU= 38.388
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhpi1+002016
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhpi1+002016
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhpi1+002016
  12:12:34 STEP 2017 H=2017:00 +CPU= 20.140
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206767/ICMGGhpi1+002004
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206767/ICMSHhpi1+002004
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206767/ICMUAhpi1+002004
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206767/ICMGGhpi1+002016
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206767/ICMSHhpi1+002016
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206767/ICMUAhpi1+002016
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_2566_2021050100_123_990_12206767_1_r1289224932_83.zip
Uploading the intermediate file: upload_file_83.zip
  12:12:55 STEP 2018 H=2018:00 +CPU= 20.839
  12:13:27 STEP 2019 H=2019:00 +CPU= 31.757
  12:13:48 STEP 2020 H=2020:00 +CPU= 20.721
  12:14:08 STEP 2021 H=2021:00 +CPU= 20.186
  12:14:40 STEP 2022 H=2022:00 +CPU= 32.431
  12:15:00 STEP 2023 H=2023:00 +CPU= 20.145
  12:15:21 STEP 2024 H=2024:00 +CPU= 20.604
  12:15:53 STEP 2025 H=2025:00 +CPU= 31.699
  12:16:13 STEP 2026 H=2026:00 +CPU= 20.711
  12:16:33 STEP 2027 H=2027:00 +CPU= 20.045
  12:17:10 STEP 2028 H=2028:00 +CPU= 36.130
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhpi1+002028
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhpi1+002028
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhpi1+002028
  12:17:30 STEP 2029 H=2029:00 +CPU= 20.404
  12:17:51 STEP 2030 H=2030:00 +CPU= 21.040
  12:18:22 STEP 2031 H=2031:00 +CPU= 31.212
  12:18:42 STEP 2032 H=2032:00 +CPU= 19.636
  12:19:01 STEP 2033 H=2033:00 +CPU= 18.954
  12:19:31 STEP 2034 H=2034:00 +CPU= 30.543
  12:19:52 STEP 2035 H=2035:00 +CPU= 20.161
  12:20:12 STEP 2036 H=2036:00 +CPU= 20.739
  12:20:44 STEP 2037 H=2037:00 +CPU= 31.920
  12:21:04 STEP 2038 H=2038:00 +CPU= 20.224
  12:21:25 STEP 2039 H=2039:00 +CPU= 20.223
  12:22:02 STEP 2040 H=2040:00 +CPU= 37.619
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhpi1+002040
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhpi1+002040
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhpi1+002040
  12:22:23 STEP 2041 H=2041:00 +CPU= 20.299
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206767/ICMGGhpi1+002028
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206767/ICMSHhpi1+002028
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206767/ICMUAhpi1+002028
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206767/ICMGGhpi1+002040
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206767/ICMSHhpi1+002040
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206767/ICMUAhpi1+002040
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_2566_2021050100_123_990_12206767_1_r1289224932_84.zip
Uploading the intermediate file: upload_file_84.zip
  12:22:44 STEP 2042 H=2042:00 +CPU= 20.816
  12:23:15 STEP 2043 H=2043:00 +CPU= 31.749
  12:23:36 STEP 2044 H=2044:00 +CPU= 20.160
  12:23:55 STEP 2045 H=2045:00 +CPU= 19.576
  12:24:26 STEP 2046 H=2046:00 +CPU= 30.817
  12:24:45 STEP 2047 H=2047:00 +CPU= 19.073
  12:25:04 STEP 2048 H=2048:00 +CPU= 19.458
  12:25:34 STEP 2049 H=2049:00 +CPU= 29.992
  12:25:56 STEP 2050 H=2050:00 +CPU= 21.032
  12:26:16 STEP 2051 H=2051:00 +CPU= 20.604
  12:26:52 STEP 2052 H=2052:00 +CPU= 36.086
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhpi1+002052
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhpi1+002052
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhpi1+002052
  12:27:12 STEP 2053 H=2053:00 +CPU= 19.849
  12:27:32 STEP 2054 H=2054:00 +CPU= 20.382
  12:28:04 STEP 2055 H=2055:00 +CPU= 31.522
  12:28:25 STEP 2056 H=2056:00 +CPU= 20.537
  12:28:44 STEP 2057 H=2057:00 +CPU= 19.509
  12:29:16 STEP 2058 H=2058:00 +CPU= 31.910
  12:29:35 STEP 2059 H=2059:00 +CPU= 19.496
  12:29:56 STEP 2060 H=2060:00 +CPU= 20.087
  12:30:27 STEP 2061 H=2061:00 +CPU= 31.720
  12:30:48 STEP 2062 H=2062:00 +CPU= 20.779
  12:31:09 STEP 2063 H=2063:00 +CPU= 21.222
  12:31:48 STEP 2064 H=2064:00 +CPU= 38.724
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhpi1+002064
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhpi1+002064
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhpi1+002064
  12:32:08 STEP 2065 H=2065:00 +CPU= 20.273
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206767/ICMGGhpi1+002052
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206767/ICMSHhpi1+002052
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206767/ICMUAhpi1+002052
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206767/ICMGGhpi1+002064
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206767/ICMSHhpi1+002064
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206767/ICMUAhpi1+002064
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_2566_2021050100_123_990_12206767_1_r1289224932_85.zip
Uploading the intermediate file: upload_file_85.zip
  12:32:29 STEP 2066 H=2066:00 +CPU= 20.843
  12:33:01 STEP 2067 H=2067:00 +CPU= 31.940
  12:33:22 STEP 2068 H=2068:00 +CPU= 20.641
  12:33:41 STEP 2069 H=2069:00 +CPU= 19.515
  12:34:13 STEP 2070 H=2070:00 +CPU= 31.617
  12:34:33 STEP 2071 H=2071:00 +CPU= 19.518
  12:34:53 STEP 2072 H=2072:00 +CPU= 20.251
  12:35:24 STEP 2073 H=2073:00 +CPU= 30.881
  12:35:44 STEP 2074 H=2074:00 +CPU= 20.193
  12:36:03 STEP 2075 H=2075:00 +CPU= 19.555
  12:36:38 STEP 2076 H=2076:00 +CPU= 34.738
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhpi1+002076
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhpi1+002076
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhpi1+002076
  12:36:59 STEP 2077 H=2077:00 +CPU= 20.349
  12:37:19 STEP 2078 H=2078:00 +CPU= 20.847
  12:37:51 STEP 2079 H=2079:00 +CPU= 31.874
  12:38:12 STEP 2080 H=2080:00 +CPU= 20.707
  12:38:32 STEP 2081 H=2081:00 +CPU= 20.152
  12:39:05 STEP 2082 H=2082:00 +CPU= 32.445
  12:39:25 STEP 2083 H=2083:00 +CPU= 20.128
  12:39:45 STEP 2084 H=2084:00 +CPU= 20.639
  12:40:17 STEP 2085 H=2085:00 +CPU= 31.764
  12:40:38 STEP 2086 H=2086:00 +CPU= 20.665
  12:40:58 STEP 2087 H=2087:00 +CPU= 20.692
  12:41:37 STEP 2088 H=2088:00 +CPU= 38.802
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhpi1+002088
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhpi1+002088
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhpi1+002088
  12:41:58 STEP 2089 H=2089:00 +CPU= 20.556
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206767/ICMGGhpi1+002076
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206767/ICMSHhpi1+002076
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206767/ICMUAhpi1+002076
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206767/ICMGGhpi1+002088
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206767/ICMSHhpi1+002088
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206767/ICMUAhpi1+002088
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_2566_2021050100_123_990_12206767_1_r1289224932_86.zip
Uploading the intermediate file: upload_file_86.zip
  12:42:19 STEP 2090 H=2090:00 +CPU= 20.916
  12:42:51 STEP 2091 H=2091:00 +CPU= 31.646
  12:43:11 STEP 2092 H=2092:00 +CPU= 20.446
  12:43:31 STEP 2093 H=2093:00 +CPU= 19.878
  12:44:04 STEP 2094 H=2094:00 +CPU= 32.757
  12:44:24 STEP 2095 H=2095:00 +CPU= 20.370
  12:44:45 STEP 2096 H=2096:00 +CPU= 20.916
  12:45:17 STEP 2097 H=2097:00 +CPU= 31.987
  12:45:38 STEP 2098 H=2098:00 +CPU= 21.530
  12:45:59 STEP 2099 H=2099:00 +CPU= 20.298
  12:46:35 STEP 2100 H=2100:00 +CPU= 35.785
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhpi1+002100
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhpi1+002100
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhpi1+002100
  12:46:55 STEP 2101 H=2101:00 +CPU= 20.348
  12:47:16 STEP 2102 H=2102:00 +CPU= 21.014
  12:47:47 STEP 2103 H=2103:00 +CPU= 31.312
  12:48:07 STEP 2104 H=2104:00 +CPU= 19.620
  12:48:26 STEP 2105 H=2105:00 +CPU= 19.021
  12:48:57 STEP 2106 H=2106:00 +CPU= 30.617
  12:49:16 STEP 2107 H=2107:00 +CPU= 19.030
  12:49:35 STEP 2108 H=2108:00 +CPU= 19.625
  12:50:05 STEP 2109 H=2109:00 +CPU= 30.044
  12:50:26 STEP 2110 H=2110:00 +CPU= 20.503
  12:50:47 STEP 2111 H=2111:00 +CPU= 20.789
  12:51:26 STEP 2112 H=2112:00 +CPU= 39.341
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhpi1+002112
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhpi1+002112
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhpi1+002112
  12:51:47 STEP 2113 H=2113:00 +CPU= 20.607
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206767/ICMGGhpi1+002100
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206767/ICMSHhpi1+002100
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206767/ICMUAhpi1+002100
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206767/ICMGGhpi1+002112
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206767/ICMSHhpi1+002112
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206767/ICMUAhpi1+002112
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_2566_2021050100_123_990_12206767_1_r1289224932_87.zip
Uploading the intermediate file: upload_file_87.zip
  12:52:08 STEP 2114 H=2114:00 +CPU= 21.179
  12:52:41 STEP 2115 H=2115:00 +CPU= 32.530
  12:53:02 STEP 2116 H=2116:00 +CPU= 21.261
  12:53:22 STEP 2117 H=2117:00 +CPU= 20.587
  12:53:56 STEP 2118 H=2118:00 +CPU= 33.434
  12:54:16 STEP 2119 H=2119:00 +CPU= 20.160
  12:54:36 STEP 2120 H=2120:00 +CPU= 20.533
  12:55:08 STEP 2121 H=2121:00 +CPU= 31.496
  12:55:29 STEP 2122 H=2122:00 +CPU= 20.589
  12:55:49 STEP 2123 H=2123:00 +CPU= 19.921
  12:56:24 STEP 2124 H=2124:00 +CPU= 35.176
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhpi1+002124
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhpi1+002124
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhpi1+002124
  12:56:44 STEP 2125 H=2125:00 +CPU= 19.960
  12:57:04 STEP 2126 H=2126:00 +CPU= 20.549
  12:57:37 STEP 2127 H=2127:00 +CPU= 32.274
  12:57:58 STEP 2128 H=2128:00 +CPU= 21.325
  12:58:19 STEP 2129 H=2129:00 +CPU= 20.961
  12:58:51 STEP 2130 H=2130:00 +CPU= 32.225
  12:59:11 STEP 2131 H=2131:00 +CPU= 19.965
  12:59:32 STEP 2132 H=2132:00 +CPU= 20.444
  13:00:03 STEP 2133 H=2133:00 +CPU= 31.475
  13:00:24 STEP 2134 H=2134:00 +CPU= 20.668
  13:00:44 STEP 2135 H=2135:00 +CPU= 20.533
  13:01:22 STEP 2136 H=2136:00 +CPU= 37.805
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhpi1+002136
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhpi1+002136
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhpi1+002136
  13:01:42 STEP 2137 H=2137:00 +CPU= 19.944
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206767/ICMGGhpi1+002124
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206767/ICMSHhpi1+002124
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206767/ICMUAhpi1+002124
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206767/ICMGGhpi1+002136
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206767/ICMSHhpi1+002136
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206767/ICMUAhpi1+002136
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_2566_2021050100_123_990_12206767_1_r1289224932_88.zip
Uploading the intermediate file: upload_file_88.zip
  13:02:03 STEP 2138 H=2138:00 +CPU= 21.040
  13:02:36 STEP 2139 H=2139:00 +CPU= 32.621
  13:02:57 STEP 2140 H=2140:00 +CPU= 21.430
  13:03:18 STEP 2141 H=2141:00 +CPU= 20.549
  13:03:52 STEP 2142 H=2142:00 +CPU= 33.585
  13:04:12 STEP 2143 H=2143:00 +CPU= 20.554
  13:04:33 STEP 2144 H=2144:00 +CPU= 20.981
  13:05:06 STEP 2145 H=2145:00 +CPU= 32.216
  13:05:26 STEP 2146 H=2146:00 +CPU= 20.406
  13:05:46 STEP 2147 H=2147:00 +CPU= 20.342
  13:06:22 STEP 2148 H=2148:00 +CPU= 35.894
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhpi1+002148
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhpi1+002148
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhpi1+002148
  13:06:43 STEP 2149 H=2149:00 +CPU= 20.497
  13:07:04 STEP 2150 H=2150:00 +CPU= 20.884
  13:07:36 STEP 2151 H=2151:00 +CPU= 31.975
  13:07:56 STEP 2152 H=2152:00 +CPU= 20.337
  13:08:15 STEP 2153 H=2153:00 +CPU= 19.503
  13:08:47 STEP 2154 H=2154:00 +CPU= 31.849
  13:09:07 STEP 2155 H=2155:00 +CPU= 20.130
  13:09:28 STEP 2156 H=2156:00 +CPU= 20.690
  13:10:00 STEP 2157 H=2157:00 +CPU= 31.910
  13:10:20 STEP 2158 H=2158:00 +CPU= 20.417
  13:10:41 STEP 2159 H=2159:00 +CPU= 20.546
  13:11:21 STEP 2160 H=2160:00 +CPU= 38.386
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhpi1+002160
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhpi1+002160
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhpi1+002160
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.09_x86_64-pc-linux-gnu
(argv1) start_date: 2021050100
(argv2) exptid: hpi1
(argv3) unique_member_id: 2566
(argv4) batchid: 990
(argv5) wuid: 12206767
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/8
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.09
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206767
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.09_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/8/oifs_43r3_ps_app_1.09_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/8/oifs_43r3_ps_app_1.09_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_6674f8b38834b48117e4123f90b2a295 to: /var/lib/boinc-client/slots/8/oifs_43r3_ps_2566_2021050100_123_990_12206767.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/8/oifs_43r3_ps_2566_2021050100_123_990_12206767.zip
ic_ancil_file: ic_ancil_12206767
ifsdata_file: ifsdata_12206767
climate_data_file: clim_data_12206767
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_6536e9571c94503e1d63b5bf9813f917 to: /var/lib/boinc-client/slots/8/ic_ancil_12206767.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/8/ic_ancil_12206767.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_bb811feb435cbb58bdc77f16003530ae to: /var/lib/boinc-client/slots/8/ifsdata/ifsdata_12206767.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/8/ifsdata/ifsdata_12206767.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/8/159l_2/clim_data_12206767.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/8/159l_2/clim_data_12206767.zip
Checking for progress XML file: /var/lib/boinc-client/slots/8/progress_file_12206767.xml
Opened progress file ok : /var/lib/boinc-client/slots/8/progress_file_12206767.xml
-- Model is restarting --
Adjusting last_iter, 2161, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/8/progress_file_12206767.xml
last_cpu_time: 52458
upload_file_number: 89
last_iter: 2161
last_upload: 7693200
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_2566_2021050100_123_990_12206767_1_r1289224932
The child process has been launched with process id: 32446
Executing the command: /var/lib/boinc-client/slots/8/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:prj-boinc:1:1:32446:32446] [20230212:132308:1676208188:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fa0
[EC_DRHOOK:prj-boinc:1:1:32446:32446] [20230212:132308:1676208188:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:prj-boinc:1:1:32446:32446] [20230212:132308:1676208188:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:prj-boinc:1:1:32446:32446] [20230212:132308:1676208188:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:prj-boinc:1:1:32446:32446] [20230212:132308:1676208188:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:prj-boinc:1:1:32446:32446] [20230212:132308:1676208188:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:prj-boinc:1:1:32446:32446] [20230212:132308:1676208188:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:prj-boinc:1:1:32446:32446] [20230212:132308:1676208188:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:prj-boinc:1:1:32446:32446] [20230212:132308:1676208188:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:prj-boinc:1:1:32446:32446] [20230212:132308:1676208188:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:prj-boinc:1:1:32446:32446] [20230212:132308:1676208188:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:prj-boinc:1:1:32446:32446] [20230212:132308:1676208188:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:prj-boinc:1:1:32446:32446] [20230212:132308:1676208188:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:prj-boinc:1:1:32446:32446] [20230212:132308:1676208188:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:prj-boinc:1:1:32446:32446] [20230212:132308:1676208188:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:prj-boinc:1:1:32446:32446] [20230212:132308:1676208188:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x815590 (old at 0x1ce8cf0)
[EC_DRHOOK:prj-boinc:1:1:32446:32446] [20230212:132308:1676208188:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x815590 (old at (nil))
[EC_DRHOOK:prj-boinc:1:1:32446:32446] [20230212:132308:1676208188:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x815590 (old at 0x1ce8cf0)
[EC_DRHOOK:prj-boinc:1:1:32446:32446] [20230212:132308:1676208188:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x815590 (old at (nil))
[EC_DRHOOK:prj-boinc:1:1:32446:32446] [20230212:132308:1676208188:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:prj-boinc:1:1:32446:32446] [20230212:132308:1676208188:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x815590 (old at 0x1ce8cf0)
[EC_DRHOOK:prj-boinc:1:1:32446:32446] [20230212:132308:1676208188:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x815590 (old at 0x1ce8cf0)
[EC_DRHOOK:prj-boinc:1:1:32446:32446] [20230212:132308:1676208188:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x815590 (old at (nil))
[EC_DRHOOK:prj-boinc:1:1:32446:32446] [20230212:132308:1676208188:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x815590 (old at 0x1ce8cf0)
[EC_DRHOOK:prj-boinc:1:1:32446:32446] [20230212:132308:1676208188:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x815590 (old at 0x1ce8cf0)
[EC_DRHOOK:prj-boinc:1:1:32446:32446] [20230212:132308:1676208188:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x815590 (old at 0x1ce8cf0)
[EC_DRHOOK:prj-boinc:1:1:32446:32446] [20230212:132308:1676208188:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x815590 (old at (nil))
[EC_DRHOOK:prj-boinc:1:1:32446:32446] [20230212:132308:1676208188:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x815590 (old at (nil))
[EC_DRHOOK:prj-boinc:1:1:32446:32446] [20230212:132308:1676208188:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x815590 (old at (nil))
[EC_DRHOOK:prj-boinc:1:1:32446:32446] [20230212:132308:1676208188:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206767/ICMGGhpi1+002148
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206767/ICMSHhpi1+002148
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206767/ICMUAhpi1+002148
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206767/ICMGGhpi1+002160
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206767/ICMSHhpi1+002160
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206767/ICMUAhpi1+002160
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_2566_2021050100_123_990_12206767_1_r1289224932_89.zip
Uploading the intermediate file: upload_file_89.zip
  13:24:02 STEP 2160 H=2160:00 +CPU= 46.934
  13:24:29 STEP 2161 H=2161:00 +CPU= 27.659
  13:24:50 STEP 2162 H=2162:00 +CPU= 21.243
  13:25:23 STEP 2163 H=2163:00 +CPU= 32.741
  13:25:44 STEP 2164 H=2164:00 +CPU= 21.114
  13:26:05 STEP 2165 H=2165:00 +CPU= 20.411
  13:26:38 STEP 2166 H=2166:00 +CPU= 32.950
  13:26:58 STEP 2167 H=2167:00 +CPU= 20.435
  13:27:19 STEP 2168 H=2168:00 +CPU= 21.010
  13:27:51 STEP 2169 H=2169:00 +CPU= 31.443
  13:28:11 STEP 2170 H=2170:00 +CPU= 20.134
  13:28:30 STEP 2171 H=2171:00 +CPU= 19.557
  13:29:05 STEP 2172 H=2172:00 +CPU= 34.397
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhpi1+002172
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhpi1+002172
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhpi1+002172
  13:29:25 STEP 2173 H=2173:00 +CPU= 19.555
  13:29:45 STEP 2174 H=2174:00 +CPU= 20.141
  13:30:16 STEP 2175 H=2175:00 +CPU= 31.295
  13:30:36 STEP 2176 H=2176:00 +CPU= 20.216
  13:30:56 STEP 2177 H=2177:00 +CPU= 20.206
  13:31:29 STEP 2178 H=2178:00 +CPU= 32.692
  13:31:49 STEP 2179 H=2179:00 +CPU= 20.236
  13:32:10 STEP 2180 H=2180:00 +CPU= 20.853
  13:32:42 STEP 2181 H=2181:00 +CPU= 31.518
  13:33:02 STEP 2182 H=2182:00 +CPU= 20.158
  13:33:22 STEP 2183 H=2183:00 +CPU= 20.182
  13:34:00 STEP 2184 H=2184:00 +CPU= 37.996
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhpi1+002184
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhpi1+002184
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhpi1+002184
  13:34:20 STEP 2185 H=2185:00 +CPU= 20.076
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206767/ICMGGhpi1+002172
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206767/ICMSHhpi1+002172
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206767/ICMUAhpi1+002172
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206767/ICMGGhpi1+002184
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206767/ICMSHhpi1+002184
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206767/ICMUAhpi1+002184
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_2566_2021050100_123_990_12206767_1_r1289224932_90.zip
Uploading the intermediate file: upload_file_90.zip
  13:34:42 STEP 2186 H=2186:00 +CPU= 21.326
  13:35:13 STEP 2187 H=2187:00 +CPU= 31.268
  13:35:33 STEP 2188 H=2188:00 +CPU= 20.070
  13:35:53 STEP 2189 H=2189:00 +CPU= 19.732
  13:36:25 STEP 2190 H=2190:00 +CPU= 31.817
  13:36:45 STEP 2191 H=2191:00 +CPU= 20.036
  13:37:06 STEP 2192 H=2192:00 +CPU= 21.007
  13:37:38 STEP 2193 H=2193:00 +CPU= 32.229
  13:37:59 STEP 2194 H=2194:00 +CPU= 20.710
  13:38:19 STEP 2195 H=2195:00 +CPU= 19.827
  13:38:53 STEP 2196 H=2196:00 +CPU= 34.777
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhpi1+002196
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhpi1+002196
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhpi1+002196
  13:39:14 STEP 2197 H=2197:00 +CPU= 20.167
  13:39:34 STEP 2198 H=2198:00 +CPU= 20.710
  13:40:06 STEP 2199 H=2199:00 +CPU= 31.742
  13:40:27 STEP 2200 H=2200:00 +CPU= 20.677
  13:40:47 STEP 2201 H=2201:00 +CPU= 20.132
  13:41:19 STEP 2202 H=2202:00 +CPU= 32.459
  13:41:39 STEP 2203 H=2203:00 +CPU= 19.651
  13:41:59 STEP 2204 H=2204:00 +CPU= 20.033
  13:42:30 STEP 2205 H=2205:00 +CPU= 31.278
  13:42:51 STEP 2206 H=2206:00 +CPU= 20.179
  13:43:11 STEP 2207 H=2207:00 +CPU= 20.035
  13:43:50 STEP 2208 H=2208:00 +CPU= 38.764
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhpi1+002208
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhpi1+002208
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhpi1+002208
  13:44:11 STEP 2209 H=2209:00 +CPU= 21.210
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206767/ICMGGhpi1+002196
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206767/ICMSHhpi1+002196
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206767/ICMUAhpi1+002196
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206767/ICMGGhpi1+002208
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206767/ICMSHhpi1+002208
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206767/ICMUAhpi1+002208
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_2566_2021050100_123_990_12206767_1_r1289224932_91.zip
Uploading the intermediate file: upload_file_91.zip
  13:44:33 STEP 2210 H=2210:00 +CPU= 22.192
  13:45:06 STEP 2211 H=2211:00 +CPU= 33.305
  13:45:28 STEP 2212 H=2212:00 +CPU= 21.696
  13:45:49 STEP 2213 H=2213:00 +CPU= 20.893
  13:46:22 STEP 2214 H=2214:00 +CPU= 33.387
  13:46:44 STEP 2215 H=2215:00 +CPU= 21.114
  13:47:05 STEP 2216 H=2216:00 +CPU= 21.726
  13:47:38 STEP 2217 H=2217:00 +CPU= 32.320
  13:47:59 STEP 2218 H=2218:00 +CPU= 21.233
  13:48:20 STEP 2219 H=2219:00 +CPU= 20.868
  13:48:55 STEP 2220 H=2220:00 +CPU= 35.675
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhpi1+002220
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhpi1+002220
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhpi1+002220
  13:49:16 STEP 2221 H=2221:00 +CPU= 20.695
  13:49:37 STEP 2222 H=2222:00 +CPU= 21.084
  13:50:10 STEP 2223 H=2223:00 +CPU= 32.749
  13:50:31 STEP 2224 H=2224:00 +CPU= 21.471
  13:50:52 STEP 2225 H=2225:00 +CPU= 20.707
  13:51:26 STEP 2226 H=2226:00 +CPU= 33.440
  13:51:46 STEP 2227 H=2227:00 +CPU= 20.659
  13:52:07 STEP 2228 H=2228:00 +CPU= 21.176
  13:52:39 STEP 2229 H=2229:00 +CPU= 31.698
  13:53:00 STEP 2230 H=2230:00 +CPU= 20.568
  13:53:20 STEP 2231 H=2231:00 +CPU= 20.669
  13:53:59 STEP 2232 H=2232:00 +CPU= 38.549
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhpi1+002232
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhpi1+002232
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhpi1+002232
  13:54:20 STEP 2233 H=2233:00 +CPU= 20.860
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206767/ICMGGhpi1+002220
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206767/ICMSHhpi1+002220
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206767/ICMUAhpi1+002220
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206767/ICMGGhpi1+002232
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206767/ICMSHhpi1+002232
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206767/ICMUAhpi1+002232
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_2566_2021050100_123_990_12206767_1_r1289224932_92.zip
Uploading the intermediate file: upload_file_92.zip
  13:54:41 STEP 2234 H=2234:00 +CPU= 21.302
  13:55:14 STEP 2235 H=2235:00 +CPU= 32.783
  13:55:36 STEP 2236 H=2236:00 +CPU= 21.268
  13:55:57 STEP 2237 H=2237:00 +CPU= 20.990
  13:56:30 STEP 2238 H=2238:00 +CPU= 33.389
  13:56:50 STEP 2239 H=2239:00 +CPU= 20.105
  13:57:11 STEP 2240 H=2240:00 +CPU= 20.661
  13:57:42 STEP 2241 H=2241:00 +CPU= 31.358
  13:58:03 STEP 2242 H=2242:00 +CPU= 21.454
  13:58:24 STEP 2243 H=2243:00 +CPU= 20.698
  13:59:01 STEP 2244 H=2244:00 +CPU= 36.832
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhpi1+002244
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhpi1+002244
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhpi1+002244
  13:59:21 STEP 2245 H=2245:00 +CPU= 20.339
  13:59:42 STEP 2246 H=2246:00 +CPU= 20.995
  14:00:14 STEP 2247 H=2247:00 +CPU= 31.762
  14:00:35 STEP 2248 H=2248:00 +CPU= 20.896
  14:00:56 STEP 2249 H=2249:00 +CPU= 20.795
  14:01:29 STEP 2250 H=2250:00 +CPU= 33.562
  14:01:50 STEP 2251 H=2251:00 +CPU= 20.708
  14:02:11 STEP 2252 H=2252:00 +CPU= 21.130
  14:02:44 STEP 2253 H=2253:00 +CPU= 32.706
  14:03:05 STEP 2254 H=2254:00 +CPU= 20.620
  14:03:26 STEP 2255 H=2255:00 +CPU= 21.096
  14:04:04 STEP 2256 H=2256:00 +CPU= 38.245
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhpi1+002256
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhpi1+002256
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhpi1+002256
  14:04:25 STEP 2257 H=2257:00 +CPU= 20.565
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206767/ICMGGhpi1+002244
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206767/ICMSHhpi1+002244
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206767/ICMUAhpi1+002244
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206767/ICMGGhpi1+002256
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206767/ICMSHhpi1+002256
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206767/ICMUAhpi1+002256
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_2566_2021050100_123_990_12206767_1_r1289224932_93.zip
Uploading the intermediate file: upload_file_93.zip
  14:04:46 STEP 2258 H=2258:00 +CPU= 21.371
  14:05:19 STEP 2259 H=2259:00 +CPU= 33.160
  14:05:40 STEP 2260 H=2260:00 +CPU= 20.724
  14:06:00 STEP 2261 H=2261:00 +CPU= 20.158
  14:06:33 STEP 2262 H=2262:00 +CPU= 32.473
  14:06:52 STEP 2263 H=2263:00 +CPU= 19.507
  14:07:13 STEP 2264 H=2264:00 +CPU= 20.109
  14:07:44 STEP 2265 H=2265:00 +CPU= 31.360
  14:08:05 STEP 2266 H=2266:00 +CPU= 20.657
  14:08:25 STEP 2267 H=2267:00 +CPU= 20.845
  14:09:02 STEP 2268 H=2268:00 +CPU= 36.442
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhpi1+002268
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhpi1+002268
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhpi1+002268
  14:09:23 STEP 2269 H=2269:00 +CPU= 20.812
  14:09:44 STEP 2270 H=2270:00 +CPU= 21.339
  14:10:17 STEP 2271 H=2271:00 +CPU= 32.577
  14:10:38 STEP 2272 H=2272:00 +CPU= 21.359
  14:10:59 STEP 2273 H=2273:00 +CPU= 20.692
  14:11:31 STEP 2274 H=2274:00 +CPU= 32.726
  14:11:51 STEP 2275 H=2275:00 +CPU= 19.953
  14:12:12 STEP 2276 H=2276:00 +CPU= 20.738
  14:12:44 STEP 2277 H=2277:00 +CPU= 31.521
  14:13:04 STEP 2278 H=2278:00 +CPU= 20.587
  14:13:25 STEP 2279 H=2279:00 +CPU= 20.537
  14:14:03 STEP 2280 H=2280:00 +CPU= 38.331
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhpi1+002280
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhpi1+002280
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhpi1+002280
  14:14:23 STEP 2281 H=2281:00 +CPU= 19.920
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206767/ICMGGhpi1+002268
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206767/ICMSHhpi1+002268
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206767/ICMUAhpi1+002268
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206767/ICMGGhpi1+002280
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206767/ICMSHhpi1+002280
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12206767/ICMUAhpi1+002280
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_2566_2021050100_123_990_12206767_1_r1289224932_94.zip
Uploading the intermediate file: upload_file_94.zip
  14:14:44 STEP 2282 H=2282:00 +CPU= 20.586
double free or corruption (out)

</stderr_txt>
]]>
No trickles!


©2024 climateprediction.net