Name | oifs_43r3_ps_0526_2006050100_123_975_12192170_2 |
Workunit | 12192170 |
Created | 20 Jan 2023, 5:45:08 UTC |
Sent | 21 Jan 2023, 19:37:00 UTC |
Report deadline | 22 Mar 2023, 19:37:00 UTC |
Received | 27 Jan 2023, 12:10:25 UTC |
Server state | Over |
Outcome | Computation error |
Client state | Compute error |
Exit status | 1 (0x00000001) Unknown error code |
Computer ID | 1513123 |
Run time | 1 days 5 hours 46 min 4 sec |
CPU time | 1 days 5 hours 46 min 4 sec |
Validate state | Invalid |
Credit | 9,676.80 |
Device peak FLOPS | 1.00 GFLOPS |
Application version | OpenIFS 43r3 Perturbed Surface v1.05 x86_64-pc-linux-gnu |
Peak working set size | 4,780.69 MB |
Peak swap size | 4,978.07 MB |
Peak disk usage | 4,724.79 MB |
Stderr | <core_client_version>7.16.17</core_client_version> <![CDATA[ <message> process exited with code 1 (0x1, -255)</message> <stderr_txt> ng to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12192170/ICMUAhq0f+001884 Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0526_2006050100_123_975_12192170_2_r2040001171_78.zip 03:18:06 STEP 1897 H=1897:00 +CPU= 35.485 Uploading the intermediate file: upload_file_78.zip 03:18:59 STEP 1898 H=1898:00 +CPU= 46.808 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 03:20:25 STEP 1899 H=1899:00 +CPU= 74.221 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 03:21:41 STEP 1900 H=1900:00 +CPU= 50.987 Suspend request received from the BOINC client, suspending the child process Resuming the child process 03:22:30 STEP 1901 H=1901:00 +CPU= 45.193 03:23:54 STEP 1902 H=1902:00 +CPU= 76.758 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 03:24:50 STEP 1903 H=1903:00 +CPU= 45.875 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 03:25:56 STEP 1904 H=1904:00 +CPU= 47.409 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 03:27:20 STEP 1905 H=1905:00 +CPU= 73.275 Suspend request received from the BOINC client, suspending the child process Resuming the child process 03:28:11 STEP 1906 H=1906:00 +CPU= 44.176 Suspend request received from the BOINC client, suspending the child process Resuming the child process 03:29:01 STEP 1907 H=1907:00 +CPU= 42.677 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 03:30:39 STEP 1908 H=1908:00 +CPU= 80.584 Suspend request received from the BOINC client, suspending the child process Resuming the child process Moving to projects directory: /var/lib/boinc/slots/21/ICMGGhq0f+001908 Moving to projects directory: /var/lib/boinc/slots/21/ICMSHhq0f+001908 Moving to projects directory: /var/lib/boinc/slots/21/ICMUAhq0f+001908 Suspend request received from the BOINC client, suspending the child process Resuming the child process 03:31:40 STEP 1909 H=1909:00 +CPU= 44.841 03:32:32 STEP 1910 H=1910:00 +CPU= 46.258 Suspend request received from the BOINC client, suspending the child process Resuming the child process 03:33:40 STEP 1911 H=1911:00 +CPU= 63.482 Suspend request received from the BOINC client, suspending the child process Resuming the child process 03:34:24 STEP 1912 H=1912:00 +CPU= 40.496 Suspend request received from the BOINC client, suspending the child process Resuming the child process 03:35:08 STEP 1913 H=1913:00 +CPU= 40.961 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 03:36:36 STEP 1914 H=1914:00 +CPU= 73.997 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 03:37:55 STEP 1915 H=1915:00 +CPU= 45.876 03:38:58 STEP 1916 H=1916:00 +CPU= 54.341 03:40:59 STEP 1917 H=1917:00 +CPU=100.674 Suspend request received from the BOINC client, suspending the child process Resuming the child process 03:42:05 STEP 1918 H=1918:00 +CPU= 46.756 Suspend request received from the BOINC client, suspending the child process Resuming the child process 03:43:00 STEP 1919 H=1919:00 +CPU= 46.890 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 03:47:10 STEP 1920 H=1920:00 +CPU= 88.010 Moving to projects directory: /var/lib/boinc/slots/21/ICMGGhq0f+001920 Moving to projects directory: /var/lib/boinc/slots/21/ICMSHhq0f+001920 Moving to projects directory: /var/lib/boinc/slots/21/ICMUAhq0f+001920 Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12192170/ICMGGhq0f+001896 Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12192170/ICMSHhq0f+001896 Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12192170/ICMUAhq0f+001896 Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12192170/ICMGGhq0f+001908 Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12192170/ICMSHhq0f+001908 Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12192170/ICMUAhq0f+001908 Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0526_2006050100_123_975_12192170_2_r2040001171_79.zip Uploading the intermediate file: upload_file_79.zip 03:48:00 STEP 1921 H=1921:00 +CPU= 46.110 Uploading trickle at timestep: 6912000 Suspend request received from the BOINC client, suspending the child process Resuming the child process 03:48:54 STEP 1922 H=1922:00 +CPU= 47.724 Suspend request received from the BOINC client, suspending the child process Resuming the child process 03:50:24 STEP 1923 H=1923:00 +CPU= 74.347 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 03:51:41 STEP 1924 H=1924:00 +CPU= 50.187 Suspend request received from the BOINC client, suspending the child process Resuming the child process 03:52:31 STEP 1925 H=1925:00 +CPU= 44.368 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 03:53:55 STEP 1926 H=1926:00 +CPU= 72.000 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 03:54:50 STEP 1927 H=1927:00 +CPU= 43.596 03:55:43 STEP 1928 H=1928:00 +CPU= 46.007 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 03:57:42 STEP 1929 H=1929:00 +CPU= 73.212 Suspend request received from the BOINC client, suspending the child process Resuming the child process 03:58:41 STEP 1930 H=1930:00 +CPU= 48.340 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 03:59:42 STEP 1931 H=1931:00 +CPU= 45.324 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 04:01:24 STEP 1932 H=1932:00 +CPU= 83.633 Moving to projects directory: /var/lib/boinc/slots/21/ICMGGhq0f+001932 Moving to projects directory: /var/lib/boinc/slots/21/ICMSHhq0f+001932 Moving to projects directory: /var/lib/boinc/slots/21/ICMUAhq0f+001932 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 04:02:22 STEP 1933 H=1933:00 +CPU= 49.038 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 04:03:21 STEP 1934 H=1934:00 +CPU= 47.792 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 04:05:28 STEP 1935 H=1935:00 +CPU= 80.547 04:06:38 STEP 1936 H=1936:00 +CPU= 57.066 04:07:35 STEP 1937 H=1937:00 +CPU= 49.575 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 04:08:56 STEP 1938 H=1938:00 +CPU= 70.828 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 04:09:56 STEP 1939 H=1939:00 +CPU= 46.482 04:10:57 STEP 1940 H=1940:00 +CPU= 49.826 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 04:12:24 STEP 1941 H=1941:00 +CPU= 74.460 Suspend request received from the BOINC client, suspending the child process Resuming the child process 04:13:21 STEP 1942 H=1942:00 +CPU= 49.552 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 04:14:26 STEP 1943 H=1943:00 +CPU= 50.590 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 04:16:49 STEP 1944 H=1944:00 +CPU= 99.416 Suspend request received from the BOINC client, suspending the child process Resuming the child process Moving to projects directory: /var/lib/boinc/slots/21/ICMGGhq0f+001944 Moving to projects directory: /var/lib/boinc/slots/21/ICMSHhq0f+001944 Moving to projects directory: /var/lib/boinc/slots/21/ICMUAhq0f+001944 Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12192170/ICMGGhq0f+001920 Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12192170/ICMSHhq0f+001920 Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12192170/ICMUAhq0f+001920 Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12192170/ICMGGhq0f+001932 Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12192170/ICMSHhq0f+001932 Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12192170/ICMUAhq0f+001932 Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0526_2006050100_123_975_12192170_2_r2040001171_80.zip Uploading the intermediate file: upload_file_80.zip 04:17:47 STEP 1945 H=1945:00 +CPU= 51.642 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 04:18:51 STEP 1946 H=1946:00 +CPU= 48.318 Suspend request received from the BOINC client, suspending the child process Resuming the child process 04:20:13 STEP 1947 H=1947:00 +CPU= 70.127 04:21:28 STEP 1948 H=1948:00 +CPU= 58.035 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 04:22:38 STEP 1949 H=1949:00 +CPU= 54.483 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 04:24:40 STEP 1950 H=1950:00 +CPU= 82.207 Suspend request received from the BOINC client, suspending the child process Resuming the child process 04:25:51 STEP 1951 H=1951:00 +CPU= 49.275 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 04:26:55 STEP 1952 H=1952:00 +CPU= 53.990 Suspend request received from the BOINC client, suspending the child process Resuming the child process 04:28:20 STEP 1953 H=1953:00 +CPU= 74.740 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 04:29:23 STEP 1954 H=1954:00 +CPU= 47.870 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 04:30:24 STEP 1955 H=1955:00 +CPU= 47.733 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 04:32:07 STEP 1956 H=1956:00 +CPU= 82.721 Moving to projects directory: /var/lib/boinc/slots/21/ICMGGhq0f+001956 Moving to projects directory: /var/lib/boinc/slots/21/ICMSHhq0f+001956 Moving to projects directory: /var/lib/boinc/slots/21/ICMUAhq0f+001956 Suspend request received from the BOINC client, suspending the child process Resuming the child process 04:33:07 STEP 1957 H=1957:00 +CPU= 51.464 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 04:34:09 STEP 1958 H=1958:00 +CPU= 51.207 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 04:35:33 STEP 1959 H=1959:00 +CPU= 72.755 Suspend request received from the BOINC client, suspending the child process Resuming the child process 04:36:20 STEP 1960 H=1960:00 +CPU= 42.860 04:37:06 STEP 1961 H=1961:00 +CPU= 42.524 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 04:38:34 STEP 1962 H=1962:00 +CPU= 73.975 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 04:39:41 STEP 1963 H=1963:00 +CPU= 47.000 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 04:40:34 STEP 1964 H=1964:00 +CPU= 44.852 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 04:42:02 STEP 1965 H=1965:00 +CPU= 74.497 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 04:43:02 STEP 1966 H=1966:00 +CPU= 48.387 Suspend request received from the BOINC client, suspending the child process Resuming the child process 04:44:18 STEP 1967 H=1967:00 +CPU= 47.911 Suspend request received from the BOINC client, suspending the child process Resuming the child process 04:48:17 STEP 1968 H=1968:00 +CPU= 86.007 Moving to projects directory: /var/lib/boinc/slots/21/ICMGGhq0f+001968 Moving to projects directory: /var/lib/boinc/slots/21/ICMSHhq0f+001968 Moving to projects directory: /var/lib/boinc/slots/21/ICMUAhq0f+001968 Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12192170/ICMGGhq0f+001944 Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12192170/ICMSHhq0f+001944 Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12192170/ICMUAhq0f+001944 Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12192170/ICMGGhq0f+001956 Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12192170/ICMSHhq0f+001956 Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12192170/ICMUAhq0f+001956 Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0526_2006050100_123_975_12192170_2_r2040001171_81.zip Uploading the intermediate file: upload_file_81.zip 04:49:17 STEP 1969 H=1969:00 +CPU= 51.626 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 04:50:08 STEP 1970 H=1970:00 +CPU= 44.500 04:51:28 STEP 1971 H=1971:00 +CPU= 72.601 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 04:52:21 STEP 1972 H=1972:00 +CPU= 45.653 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 04:53:12 STEP 1973 H=1973:00 +CPU= 44.281 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 04:54:36 STEP 1974 H=1974:00 +CPU= 71.099 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 04:55:35 STEP 1975 H=1975:00 +CPU= 45.418 04:56:32 STEP 1976 H=1976:00 +CPU= 46.569 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 04:58:02 STEP 1977 H=1977:00 +CPU= 74.354 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 04:59:04 STEP 1978 H=1978:00 +CPU= 49.665 Suspend request received from the BOINC client, suspending the child process Resuming the child process 04:59:59 STEP 1979 H=1979:00 +CPU= 48.815 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 05:01:38 STEP 1980 H=1980:00 +CPU= 83.113 Moving to projects directory: /var/lib/boinc/slots/21/ICMGGhq0f+001980 Moving to projects directory: /var/lib/boinc/slots/21/ICMSHhq0f+001980 Moving to projects directory: /var/lib/boinc/slots/21/ICMUAhq0f+001980 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 05:02:31 STEP 1981 H=1981:00 +CPU= 44.995 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 05:03:27 STEP 1982 H=1982:00 +CPU= 45.966 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 05:04:57 STEP 1983 H=1983:00 +CPU= 73.046 Suspend request received from the BOINC client, suspending the child process Resuming the child process 05:05:44 STEP 1984 H=1984:00 +CPU= 40.531 05:06:40 STEP 1985 H=1985:00 +CPU= 46.809 Suspend request received from the BOINC client, suspending the child process Resuming the child process 05:08:01 STEP 1986 H=1986:00 +CPU= 73.770 Suspend request received from the BOINC client, suspending the child process Resuming the child process 05:08:47 STEP 1987 H=1987:00 +CPU= 41.708 Suspend request received from the BOINC client, suspending the child process Resuming the child process 05:09:36 STEP 1988 H=1988:00 +CPU= 42.780 Suspend request received from the BOINC client, suspending the child process Resuming the child process 05:10:57 STEP 1989 H=1989:00 +CPU= 69.067 05:11:52 STEP 1990 H=1990:00 +CPU= 46.200 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 05:12:51 STEP 1991 H=1991:00 +CPU= 46.081 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 05:14:56 STEP 1992 H=1992:00 +CPU= 76.214 Moving to projects directory: /var/lib/boinc/slots/21/ICMGGhq0f+001992 Moving to projects directory: /var/lib/boinc/slots/21/ICMSHhq0f+001992 Moving to projects directory: /var/lib/boinc/slots/21/ICMUAhq0f+001992 Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12192170/ICMGGhq0f+001968 Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12192170/ICMSHhq0f+001968 Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12192170/ICMUAhq0f+001968 Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12192170/ICMGGhq0f+001980 Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12192170/ICMSHhq0f+001980 Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12192170/ICMUAhq0f+001980 Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0526_2006050100_123_975_12192170_2_r2040001171_82.zip Uploading the intermediate file: upload_file_82.zip Suspend request received from the BOINC client, suspending the child process Resuming the child process 05:15:46 STEP 1993 H=1993:00 +CPU= 42.292 Suspend request received from the BOINC client, suspending the child process Resuming the child process 05:16:39 STEP 1994 H=1994:00 +CPU= 44.145 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 05:18:15 STEP 1995 H=1995:00 +CPU= 71.496 Suspend request received from the BOINC client, suspending the child process Resuming the child process 05:19:01 STEP 1996 H=1996:00 +CPU= 41.508 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 05:19:49 STEP 1997 H=1997:00 +CPU= 39.094 Suspend request received from the BOINC client, suspending the child process Resuming the child process 05:20:58 STEP 1998 H=1998:00 +CPU= 64.309 Suspend request received from the BOINC client, suspending the child process Resuming the child process 05:22:20 STEP 1999 H=1999:00 +CPU= 42.788 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 05:23:45 STEP 2000 H=2000:00 +CPU= 47.365 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 05:25:01 STEP 2001 H=2001:00 +CPU= 67.599 Suspend request received from the BOINC client, suspending the child process Resuming the child process 05:25:55 STEP 2002 H=2002:00 +CPU= 47.758 Suspend request received from the BOINC client, suspending the child process Resuming the child process 05:26:59 STEP 2003 H=2003:00 +CPU= 53.665 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 05:28:53 STEP 2004 H=2004:00 +CPU= 93.985 Suspend request received from the BOINC client, suspending the child process Resuming the child process Moving to projects directory: /var/lib/boinc/slots/21/ICMGGhq0f+002004 Moving to projects directory: /var/lib/boinc/slots/21/ICMSHhq0f+002004 Moving to projects directory: /var/lib/boinc/slots/21/ICMUAhq0f+002004 Suspend request received from the BOINC client, suspending the child process Resuming the child process 05:30:21 STEP 2005 H=2005:00 +CPU= 47.379 Suspend request received from the BOINC client, suspending the child process Resuming the child process 05:31:20 STEP 2006 H=2006:00 +CPU= 50.211 Suspend request received from the BOINC client, suspending the child process Resuming the child process 05:32:49 STEP 2007 H=2007:00 +CPU= 74.991 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 05:33:43 STEP 2008 H=2008:00 +CPU= 46.201 Suspend request received from the BOINC client, suspending the child process Resuming the child process 05:34:35 STEP 2009 H=2009:00 +CPU= 46.271 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 05:35:58 STEP 2010 H=2010:00 +CPU= 73.170 05:36:42 STEP 2011 H=2011:00 +CPU= 41.881 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 05:37:40 STEP 2012 H=2012:00 +CPU= 46.741 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 05:39:04 STEP 2013 H=2013:00 +CPU= 70.259 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 05:39:56 STEP 2014 H=2014:00 +CPU= 41.642 Suspend request received from the BOINC client, suspending the child process Resuming the child process 05:40:48 STEP 2015 H=2015:00 +CPU= 45.095 Suspend request received from the BOINC client, suspending the child process Resuming the child process 05:42:32 STEP 2016 H=2016:00 +CPU= 83.868 Moving to projects directory: /var/lib/boinc/slots/21/ICMGGhq0f+002016 Moving to projects directory: /var/lib/boinc/slots/21/ICMSHhq0f+002016 Moving to projects directory: /var/lib/boinc/slots/21/ICMUAhq0f+002016 Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12192170/ICMGGhq0f+001992 Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12192170/ICMSHhq0f+001992 Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12192170/ICMUAhq0f+001992 Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12192170/ICMGGhq0f+002004 Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12192170/ICMSHhq0f+002004 Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12192170/ICMUAhq0f+002004 Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0526_2006050100_123_975_12192170_2_r2040001171_83.zip Uploading the intermediate file: upload_file_83.zip 05:43:17 STEP 2017 H=2017:00 +CPU= 40.783 Suspend request received from the BOINC client, suspending the child process Resuming the child process 05:44:10 STEP 2018 H=2018:00 +CPU= 43.246 Suspend request received from the BOINC client, suspending the child process Resuming the child process 05:45:21 STEP 2019 H=2019:00 +CPU= 64.170 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 05:46:15 STEP 2020 H=2020:00 +CPU= 45.277 Suspend request received from the BOINC client, suspending the child process Resuming the child process 05:47:16 STEP 2021 H=2021:00 +CPU= 48.737 05:48:34 STEP 2022 H=2022:00 +CPU= 73.779 05:49:21 STEP 2023 H=2023:00 +CPU= 43.024 Suspend request received from the BOINC client, suspending the child process Resuming the child process 05:50:22 STEP 2024 H=2024:00 +CPU= 47.154 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 05:51:57 STEP 2025 H=2025:00 +CPU= 78.056 05:53:08 STEP 2026 H=2026:00 +CPU= 53.516 Suspend request received from the BOINC client, suspending the child process Resuming the child process 05:54:11 STEP 2027 H=2027:00 +CPU= 47.140 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 05:56:38 STEP 2028 H=2028:00 +CPU= 86.014 Suspend request received from the BOINC client, suspending the child process Resuming the child process Moving to projects directory: /var/lib/boinc/slots/21/ICMGGhq0f+002028 Moving to projects directory: /var/lib/boinc/slots/21/ICMSHhq0f+002028 Moving to projects directory: /var/lib/boinc/slots/21/ICMUAhq0f+002028 05:57:45 STEP 2029 H=2029:00 +CPU= 55.239 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 05:58:51 STEP 2030 H=2030:00 +CPU= 52.746 Suspend request received from the BOINC client, suspending the child process Resuming the child process 06:00:07 STEP 2031 H=2031:00 +CPU= 68.914 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 06:01:03 STEP 2032 H=2032:00 +CPU= 44.778 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 06:02:20 STEP 2033 H=2033:00 +CPU= 44.964 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 06:03:52 STEP 2034 H=2034:00 +CPU= 79.073 Suspend request received from the BOINC client, suspending the child process Resuming the child process 06:04:46 STEP 2035 H=2035:00 +CPU= 46.603 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 06:05:42 STEP 2036 H=2036:00 +CPU= 47.119 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 06:07:16 STEP 2037 H=2037:00 +CPU= 75.859 06:08:17 STEP 2038 H=2038:00 +CPU= 52.538 Suspend request received from the BOINC client, suspending the child process Resuming the child process 06:09:17 STEP 2039 H=2039:00 +CPU= 48.244 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 06:12:06 STEP 2040 H=2040:00 +CPU= 87.663 Moving to projects directory: /var/lib/boinc/slots/21/ICMGGhq0f+002040 Moving to projects directory: /var/lib/boinc/slots/21/ICMSHhq0f+002040 Moving to projects directory: /var/lib/boinc/slots/21/ICMUAhq0f+002040 Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12192170/ICMGGhq0f+002016 Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12192170/ICMSHhq0f+002016 Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12192170/ICMUAhq0f+002016 Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12192170/ICMGGhq0f+002028 Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12192170/ICMSHhq0f+002028 Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12192170/ICMUAhq0f+002028 Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0526_2006050100_123_975_12192170_2_r2040001171_84.zip Uploading the intermediate file: upload_file_84.zip Suspend request received from the BOINC client, suspending the child process Resuming the child process 06:13:15 STEP 2041 H=2041:00 +CPU= 50.964 06:14:14 STEP 2042 H=2042:00 +CPU= 49.398 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 06:15:39 STEP 2043 H=2043:00 +CPU= 70.032 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 06:16:39 STEP 2044 H=2044:00 +CPU= 48.898 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 06:17:32 STEP 2045 H=2045:00 +CPU= 42.808 Suspend request received from the BOINC client, suspending the child process Resuming the child process 06:19:09 STEP 2046 H=2046:00 +CPU= 77.396 Suspend request received from the BOINC client, suspending the child process Resuming the child process 06:20:25 STEP 2047 H=2047:00 +CPU= 55.336 06:21:43 STEP 2048 H=2048:00 +CPU= 59.060 06:23:45 STEP 2049 H=2049:00 +CPU=103.230 06:25:00 STEP 2050 H=2050:00 +CPU= 61.831 Suspend request received from the BOINC client, suspending the child process Resuming the child process 06:25:45 STEP 2051 H=2051:00 +CPU= 40.186 06:26:59 STEP 2052 H=2052:00 +CPU= 69.376 Moving to projects directory: /var/lib/boinc/slots/21/ICMGGhq0f+002052 Moving to projects directory: /var/lib/boinc/slots/21/ICMSHhq0f+002052 Moving to projects directory: /var/lib/boinc/slots/21/ICMUAhq0f+002052 06:27:49 STEP 2053 H=2053:00 +CPU= 39.605 06:28:34 STEP 2054 H=2054:00 +CPU= 43.534 Suspend request received from the BOINC client, suspending the child process Resuming the child process 06:30:33 STEP 2055 H=2055:00 +CPU= 68.072 06:31:30 STEP 2056 H=2056:00 +CPU= 46.271 Suspend request received from the BOINC client, suspending the child process Resuming the child process 06:32:30 STEP 2057 H=2057:00 +CPU= 44.744 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 06:35:14 STEP 2058 H=2058:00 +CPU= 74.912 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 06:36:35 STEP 2059 H=2059:00 +CPU= 40.986 Suspend request received from the BOINC client, suspending the child process Resuming the child process 06:37:45 STEP 2060 H=2060:00 +CPU= 46.725 Suspend request received from the BOINC client, suspending the child process Resuming the child process 06:39:15 STEP 2061 H=2061:00 +CPU= 70.420 Suspend request received from the BOINC client, suspending the child process Resuming the child process 06:40:31 STEP 2062 H=2062:00 +CPU= 47.412 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 06:44:37 STEP 2063 H=2063:00 +CPU= 45.101 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Quit request received from the BOINC client, ending the child process (argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu (argv1) start_date: 2006050100 (argv2) exptid: hq0f (argv3) unique_member_id: 0526 (argv4) batchid: 975 (argv5) wuid: 12192170 (argv6) fclen: 123 (argv7) app_name: oifs_43r3_ps (argv8) nthreads: 1 Working directory is: /var/lib/boinc/slots/21 Project directory is: /var/lib/boinc/projects/climateprediction.net/ app name: oifs_43r3_ps version: 1.05 Location of temp folder: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12192170 ..mkdir for temp folder for results failed Copying: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc/slots/21/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip Unzipping the app zip file: /var/lib/boinc/slots/21/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip Copying the namelist files from: ../../projects/climateprediction.net/jf_efe2565e36d42310871bcf9df9d70757 to: /var/lib/boinc/slots/21/oifs_43r3_ps_0526_2006050100_123_975_12192170.zip Unzipping the namelist zip file: /var/lib/boinc/slots/21/oifs_43r3_ps_0526_2006050100_123_975_12192170.zip ic_ancil_file: ic_ancil_12192170 ifsdata_file: ifsdata_12192170 climate_data_file: clim_data_12192170 horiz_resolution: 159 vert_resolution: 60 grid_type: l_2 upload_interval: 24 utstep: 3600 nfrres: restart dump frequency (steps) 24 Copying IC ancils from: ../../projects/climateprediction.net/jf_69e4e558c26f9822aac634dc435fc514 to: /var/lib/boinc/slots/21/ic_ancil_12192170.zip Unzipping the IC ancils zip file: /var/lib/boinc/slots/21/ic_ancil_12192170.zip ..mkdir for ifsdata folder failed Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_a4f6fc55f435e19e1517e87c5e7a2d3b to: /var/lib/boinc/slots/21/ifsdata/ifsdata_12192170.zip Unzipping the ifsdata_zip file: /var/lib/boinc/slots/21/ifsdata/ifsdata_12192170.zip ..mkdir for the climate data folder failed Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc/slots/21/159l_2/clim_data_12192170.zip Unzipping the climate data zip file: /var/lib/boinc/slots/21/159l_2/clim_data_12192170.zip Checking for progress XML file: /var/lib/boinc/slots/21/progress_file_12192170.xml Opened progress file ok : /var/lib/boinc/slots/21/progress_file_12192170.xml -- Model is restarting -- Adjusting last_iter, 2064, to previous model restart step. Creating progress file: /var/lib/boinc/slots/21/progress_file_12192170.xml last_cpu_time: 107079 upload_file_number: 85 last_iter: 2063 last_upload: 7344000 model_completed: 0 total_length_of_simulation: 10627200 result_base_name: oifs_43r3_ps_0526_2006050100_123_975_12192170_2_r2040001171 The child process has been launched with process id: 108 Executing the command: /var/lib/boinc/slots/21/oifs_43r3_model.exe [EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1 [EC_DRHOOK:Tower:1:1:108:108] [20230127:220244:1674817364:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0 [EC_DRHOOK:Tower:1:1:108:108] [20230127:220244:1674817364:0.002] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1 [EC_DRHOOK:Tower:1:1:108:108] [20230127:220244:1674817364:0.002] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0) [EC_DRHOOK:Tower:1:1:108:108] [20230127:220244:1674817364:0.002] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1 [EC_DRHOOK:Tower:1:1:108:108] [20230127:220244:1674817364:0.002] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000 [EC_DRHOOK:Tower:1:1:108:108] [20230127:220244:1674817364:0.002] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647) [EC_DRHOOK:Tower:1:1:108:108] [20230127:220244:1674817364:0.002] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16 [EC_DRHOOK:Tower:1:1:108:108] [20230127:220244:1674817364:0.002] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0 [EC_DRHOOK:Tower:1:1:108:108] [20230127:220244:1674817364:0.003] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500 [EC_DRHOOK:Tower:1:1:108:108] [20230127:220244:1674817364:0.003] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1 [EC_DRHOOK:Tower:1:1:108:108] [20230127:220244:1674817364:0.003] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1 [EC_DRHOOK:Tower:1:1:108:108] [20230127:220244:1674817364:0.003] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1 [EC_DRHOOK:Tower:1:1:108:108] [20230127:220244:1674817364:0.003] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1 [EC_DRHOOK:Tower:1:1:108:108] [20230127:220244:1674817364:0.003] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef> [EC_DRHOOK:Tower:1:1:108:108] [20230127:220244:1674817364:0.003] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef> [EC_DRHOOK:Tower:1:1:108:108] [20230127:220244:1674817364:0.003] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:Tower:1:1:108:108] [20230127:220244:1674817364:0.004] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil)) [EC_DRHOOK:Tower:1:1:108:108] [20230127:220244:1674817364:0.004] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:Tower:1:1:108:108] [20230127:220244:1674817364:0.004] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil)) [EC_DRHOOK:Tower:1:1:108:108] [20230127:220244:1674817364:0.004] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1 [EC_DRHOOK:Tower:1:1:108:108] [20230127:220244:1674817364:0.004] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:Tower:1:1:108:108] [20230127:220244:1674817364:0.004] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:Tower:1:1:108:108] [20230127:220244:1674817364:0.004] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil)) [EC_DRHOOK:Tower:1:1:108:108] [20230127:220244:1674817364:0.004] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:Tower:1:1:108:108] [20230127:220244:1674817364:0.004] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:Tower:1:1:108:108] [20230127:220244:1674817364:0.005] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:Tower:1:1:108:108] [20230127:220244:1674817364:0.005] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil)) [EC_DRHOOK:Tower:1:1:108:108] [20230127:220244:1674817364:0.005] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil)) [EC_DRHOOK:Tower:1:1:108:108] [20230127:220244:1674817364:0.005] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil)) [EC_DRHOOK:Tower:1:1:108:108] [20230127:220244:1674817364:0.005] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef> Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process ABORT! 1 !! *** WAVE MODEL HAS ABORTED [EC_DRHOOK:Tower:1:1:108:108] [20230127:220607:1674817567:203.110] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3843] 1721 MB (maxheap), 3512 MB (maxrss), 0 MB (maxstack) [EC_DRHOOK:Tower:1:1:108:108] [20230127:220607:1674817567:203.117] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : MASTER [EC_DRHOOK:Tower:1:1:108:108] [20230127:220607:1674817567:203.124] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT0 [EC_DRHOOK:Tower:1:1:108:108] [20230127:220607:1674817567:203.127] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT1 [EC_DRHOOK:Tower:1:1:108:108] [20230127:220607:1674817567:203.132] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT2 [EC_DRHOOK:Tower:1:1:108:108] [20230127:220607:1674817567:203.137] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT3 [EC_DRHOOK:Tower:1:1:108:108] [20230127:220607:1674817567:203.137] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT4 [EC_DRHOOK:Tower:1:1:108:108] [20230127:220607:1674817567:203.144] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : WVCOUPLE [EC_DRHOOK:Tower:1:1:108:108] [20230127:220607:1674817567:203.144] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : WAVEMDL [EC_DRHOOK:Tower:1:1:108:108] [20230127:220607:1674817567:203.151] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : INITMDL [EC_DRHOOK:Tower:1:1:108:108] [20230127:220607:1674817567:203.151] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : USERIN SDL_TRACEBACK: Calling INTEL_TRBK, THRD = 1 Process 1 thread 1 calling linux_trbk from intel_trbk() [LinuxTraceBack@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/utilities/linuxtrbk.c:240] Backtrace(s) for program '/var/lib/boinc/slots/21/oifs_43r3_model.exe' : sigcontextptr=0x7ffdbbae2920 [LinuxTraceBack@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/utilities/linuxtrbk.c:277] Backtrace (size = 19) with plain hex-dump [LinuxTraceBack@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/utilities/linuxtrbk.c:340] [0]: [0x839273] : 0x839273 [LinuxTraceBack@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/utilities/linuxtrbk.c:340] [1]: [0x839879] : 0x839879 [LinuxTraceBack@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/utilities/linuxtrbk.c:340] [2]: [0x8c2957] : 0x8c2957 [LinuxTraceBack@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/utilities/linuxtrbk.c:340] [3]: [0x81d89e] : 0x81d89e [LinuxTraceBack@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/utilities/linuxtrbk.c:340] [4]: [0x15f8aca] : 0x15f8aca [LinuxTraceBack@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/utilities/linuxtrbk.c:340] [5]: [0x15d51c8] : 0x15d51c8 [LinuxTraceBack@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/utilities/linuxtrbk.c:340] [6]: [0x166b07f] : 0x166b07f [LinuxTraceBack@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/utilities/linuxtrbk.c:340] [7]: [0x15d7ff0] : 0x15d7ff0 [LinuxTraceBack@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/utilities/linuxtrbk.c:340] [8]: [0x8711e0] : 0x8711e0 [LinuxTraceBack@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/utilities/linuxtrbk.c:340] [9]: [0x7e0165] : 0x7e0165 [LinuxTraceBack@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/utilities/linuxtrbk.c:340] [10]: [0x55e4ad] : 0x55e4ad [LinuxTraceBack@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/utilities/linuxtrbk.c:340] [11]: [0x5471c9] : 0x5471c9 [LinuxTraceBack@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/utilities/linuxtrbk.c:340] [12]: [0x546412] : 0x546412 [LinuxTraceBack@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/utilities/linuxtrbk.c:340] [13]: [0x545e28] : 0x545e28 [LinuxTraceBack@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/utilities/linuxtrbk.c:340] [14]: [0x40708d] : 0x40708d [LinuxTraceBack@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/utilities/linuxtrbk.c:340] [15]: [0x40220f] : 0x40220f [LinuxTraceBack@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/utilities/linuxtrbk.c:340] [16]: [0x4021a2] : 0x4021a2 [LinuxTraceBack@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/utilities/linuxtrbk.c:340] [17]: [0x1dca390] : 0x1dca390 [LinuxTraceBack@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/utilities/linuxtrbk.c:340] [18]: [0x40206e] : 0x40206e [LinuxTraceBack@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/utilities/linuxtrbk.c:370] End of backtrace(s) SDL_TRACEBACK: Done INTEL_TRBK, THRD = 1 [EC_DRHOOK:Tower:1:1:108:108] [20230127:220608:1674817568:203.286] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1538] Received signal#6 (SIGABRT) :: 1721MB (heap), 3512MB (maxrss), 0MB (maxstack), 0 (paging), nsigs = 1 [EC_DRHOOK:Tower:1:1:108:108] [20230127:220608:1674817568:203.290] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1542] Also activating Harakiri-alarm (SIGALRM=14) to expire after 500s elapsed to prevent hangs, nsigs = 1 [EC_DRHOOK:Tower:1:1:108:108] [20230127:220608:1674817568:203.290] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1544] Harakiri signal handler 'signal_harakiri' for signal#14 (SIGALRM) installed at 0x81f3c0 (old at (nil)) [EC_DRHOOK:Tower:1:1:108:108] [20230127:220608:1674817568:203.290] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1617] Signal#6 was caused by unrecognized si_code [memaddr=0x6c], nsigs = 1 [EC_DRHOOK:Tower:1:1:108:108] [20230127:220608:1674817568:203.290] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1686] Starting DrHook backtrace for signal#6, nsigs = 1 [EC_DRHOOK:Tower:1:1:108:108] [20230127:220609:1674817569:204.296] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1734] DrHook backtrace done for signal#6, nsigs = 1 [EC_DRHOOK:Tower:1:1:108:108] [20230127:220609:1674817569:204.300] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1785] Calling previous signal handler at 0x1ce8bb0 for signal#6, nsigs = 1 forrtl: error (76): Abort trap signal Image PC Routine Line Source oifs_43r3_model.e 0000000001CE916B Unknown Unknown Unknown oifs_43r3_model.e 000000000081FFF1 Unknown Unknown Unknown oifs_43r3_model.e 0000000001DC9090 Unknown Unknown Unknown oifs_43r3_model.e 0000000001DC903B Unknown Unknown Unknown oifs_43r3_model.e 00000000008C2880 sdl_mod_mp_sdl_sr 100 sdl_mod.F90 oifs_43r3_model.e 000000000081D8AF abor1_ 41 abor1.F90 oifs_43r3_model.e 00000000015F8ACA mpabort_ 50 mpabort.F oifs_43r3_model.e 00000000015D51C8 abort1_ 44 abort1.F oifs_43r3_model.e 000000000166B07F userin_ 553 userin.F oifs_43r3_model.e 00000000015D7FF0 initmdl_ 452 initmdl.F oifs_43r3_model.e 00000000008711E0 wavemdl_ 393 wavemdl.F oifs_43r3_model.e 00000000007E0165 wvcouple_ 260 wvcouple.F90 oifs_43r3_model.e 000000000055E4AD cnt4_ 1283 cnt4.F90 oifs_43r3_model.e 00000000005471C9 cnt3_ 267 cnt3.F90 oifs_43r3_model.e 0000000000546412 cnt2_ 88 cnt2.F90 oifs_43r3_model.e 0000000000545E28 cnt1_ 92 cnt1.F90 oifs_43r3_model.e 000000000040708D cnt0_ 146 cnt0.F90 oifs_43r3_model.e 000000000040220F MAIN__ 96 master.F90 oifs_43r3_model.e 00000000004021A2 Unknown Unknown Unknown oifs_43r3_model.e 0000000001DCA390 Unknown Unknown Unknown oifs_43r3_model.e 000000000040206E Unknown Unknown Unknown The child process terminated with status: 0 ..Failed, model did not complete successfully </stderr_txt> ]]> |
Latest Trickles Received | ||||||
---|---|---|---|---|---|---|
Time Sent (UTC) | Host ID | Result ID | Result Name | Timestep | CPU Time (sec) | Average (sec/TS) |
09 May 2023 16:41:01 | 1513123 | 22300257 | oifs_43r3_ps_0526_2006050100_123_975_12192170_2 | 6,912,000 | 98,959 | 0.0143 |
09 May 2023 16:39:44 | 1513123 | 22300257 | oifs_43r3_ps_0526_2006050100_123_975_12192170_2 | 5,702,400 | 79,802 | 0.0140 |
09 May 2023 16:39:31 | 1513123 | 22300257 | oifs_43r3_ps_0526_2006050100_123_975_12192170_2 | 4,838,400 | 68,072 | 0.0141 |
09 May 2023 16:38:07 | 1513123 | 22300257 | oifs_43r3_ps_0526_2006050100_123_975_12192170_2 | 3,456,000 | 48,501 | 0.0140 |
09 May 2023 16:37:44 | 1513123 | 22300257 | oifs_43r3_ps_0526_2006050100_123_975_12192170_2 | 2,592,000 | 35,152 | 0.0136 |
09 May 2023 16:36:09 | 1513123 | 22300257 | oifs_43r3_ps_0526_2006050100_123_975_12192170_2 | 1,728,000 | 22,135 | 0.0128 |
09 May 2023 16:35:51 | 1513123 | 22300257 | oifs_43r3_ps_0526_2006050100_123_975_12192170_2 | 864,000 | 10,351 | 0.0120 |
©2024 climateprediction.net