climateprediction.net home page
Task 22293755

Task 22293755

Name oifs_43r3_ps_0454_2015050100_123_984_12201098_1
Workunit 12201098
Created 26 Dec 2022, 22:14:50 UTC
Sent 26 Dec 2022, 22:23:42 UTC
Report deadline 25 Jan 2023, 22:23:42 UTC
Received 31 Dec 2022, 23:01:48 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 196 (0x000000C4) EXIT_DISK_LIMIT_EXCEEDED
Computer ID 1527792
Run time 21 hours 13 min 14 sec
CPU time 19 hours 3 min 6 sec
Validate state Invalid
Credit 12,458.88
Device peak FLOPS 6.83 GFLOPS
Application version OpenIFS 43r3 Perturbed Surface v1.05
x86_64-pc-linux-gnu
Peak working set size 4,419.48 MB
Peak swap size 4,978.08 MB
Peak disk usage 7,548.71 MB
Stderr
<core_client_version>7.18.1</core_client_version>
<![CDATA[
<message>
Disk usage limit exceeded</message>
<stderr_txt>
5.zip
Uploading the intermediate file: upload_file_85.zip
  01:03:22 STEP 2065 H=2065:00 +CPU= 21.812
  01:03:46 STEP 2066 H=2066:00 +CPU= 22.495
  01:04:23 STEP 2067 H=2067:00 +CPU= 33.575
  01:04:47 STEP 2068 H=2068:00 +CPU= 22.086
  01:05:10 STEP 2069 H=2069:00 +CPU= 21.608
  01:05:49 STEP 2070 H=2070:00 +CPU= 35.024
  01:06:13 STEP 2071 H=2071:00 +CPU= 22.106
  01:06:38 STEP 2072 H=2072:00 +CPU= 22.562
  01:07:15 STEP 2073 H=2073:00 +CPU= 33.768
  01:07:39 STEP 2074 H=2074:00 +CPU= 21.969
  01:08:04 STEP 2075 H=2075:00 +CPU= 22.547
  01:08:45 STEP 2076 H=2076:00 +CPU= 37.437
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+002076
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+002076
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+002076
  01:09:09 STEP 2077 H=2077:00 +CPU= 22.065
  01:09:35 STEP 2078 H=2078:00 +CPU= 23.086
  01:10:12 STEP 2079 H=2079:00 +CPU= 34.035
  01:10:36 STEP 2080 H=2080:00 +CPU= 22.530
  01:11:01 STEP 2081 H=2081:00 +CPU= 22.650
  01:11:40 STEP 2082 H=2082:00 +CPU= 35.076
  01:12:04 STEP 2083 H=2083:00 +CPU= 21.799
  01:12:29 STEP 2084 H=2084:00 +CPU= 22.893
  01:13:06 STEP 2085 H=2085:00 +CPU= 34.076
  01:13:32 STEP 2086 H=2086:00 +CPU= 23.110
  01:13:58 STEP 2087 H=2087:00 +CPU= 23.779
  01:14:43 STEP 2088 H=2088:00 +CPU= 40.024
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+002088
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+002088
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+002088
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMGGhq0f+002064
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMSHhq0f+002064
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMUAhq0f+002064
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMGGhq0f+002076
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMSHhq0f+002076
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMUAhq0f+002076
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0454_2015050100_123_984_12201098_1_r725644482_86.zip
Uploading the intermediate file: upload_file_86.zip
  01:15:07 STEP 2089 H=2089:00 +CPU= 22.262
  01:15:33 STEP 2090 H=2090:00 +CPU= 23.056
  01:16:08 STEP 2091 H=2091:00 +CPU= 32.496
  01:16:32 STEP 2092 H=2092:00 +CPU= 20.957
  01:16:53 STEP 2093 H=2093:00 +CPU= 19.845
  01:17:26 STEP 2094 H=2094:00 +CPU= 30.103
  01:17:47 STEP 2095 H=2095:00 +CPU= 19.490
  01:18:09 STEP 2096 H=2096:00 +CPU= 20.054
  01:18:41 STEP 2097 H=2097:00 +CPU= 29.839
  01:19:04 STEP 2098 H=2098:00 +CPU= 20.789
  01:19:27 STEP 2099 H=2099:00 +CPU= 20.952
  01:20:05 STEP 2100 H=2100:00 +CPU= 34.856
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+002100
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+002100
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+002100
  01:20:28 STEP 2101 H=2101:00 +CPU= 20.253
  01:20:50 STEP 2102 H=2102:00 +CPU= 20.546
  01:21:24 STEP 2103 H=2103:00 +CPU= 30.927
  01:21:46 STEP 2104 H=2104:00 +CPU= 20.583
  01:22:10 STEP 2105 H=2105:00 +CPU= 21.087
  01:22:45 STEP 2106 H=2106:00 +CPU= 31.762
  01:23:07 STEP 2107 H=2107:00 +CPU= 20.231
  01:23:30 STEP 2108 H=2108:00 +CPU= 20.738
  01:24:04 STEP 2109 H=2109:00 +CPU= 31.244
  01:24:27 STEP 2110 H=2110:00 +CPU= 20.767
  01:24:51 STEP 2111 H=2111:00 +CPU= 22.090
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:25:44 STEP 2112 H=2112:00 +CPU= 37.743
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+002112
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+002112
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+002112
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMGGhq0f+002088
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMSHhq0f+002088
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMUAhq0f+002088
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMGGhq0f+002100
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMSHhq0f+002100
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMUAhq0f+002100
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0454_2015050100_123_984_12201098_1_r725644482_87.zip
Uploading the intermediate file: upload_file_87.zip
  01:26:08 STEP 2113 H=2113:00 +CPU= 21.315
  01:26:31 STEP 2114 H=2114:00 +CPU= 20.969
  01:27:08 STEP 2115 H=2115:00 +CPU= 32.610
  01:27:33 STEP 2116 H=2116:00 +CPU= 22.355
  01:27:56 STEP 2117 H=2117:00 +CPU= 21.229
  01:28:33 STEP 2118 H=2118:00 +CPU= 33.393
  01:28:57 STEP 2119 H=2119:00 +CPU= 21.381
  01:29:20 STEP 2120 H=2120:00 +CPU= 20.490
  01:29:55 STEP 2121 H=2121:00 +CPU= 32.117
  01:30:19 STEP 2122 H=2122:00 +CPU= 21.392
  01:30:41 STEP 2123 H=2123:00 +CPU= 19.970
  01:31:18 STEP 2124 H=2124:00 +CPU= 34.769
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+002124
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+002124
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+002124
  01:31:41 STEP 2125 H=2125:00 +CPU= 20.406
  01:32:04 STEP 2126 H=2126:00 +CPU= 21.023
  01:32:39 STEP 2127 H=2127:00 +CPU= 31.318
  01:33:02 STEP 2128 H=2128:00 +CPU= 21.015
  01:33:24 STEP 2129 H=2129:00 +CPU= 19.923
  01:34:00 STEP 2130 H=2130:00 +CPU= 32.566
  01:34:24 STEP 2131 H=2131:00 +CPU= 21.141
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:34:58 STEP 2132 H=2132:00 +CPU= 21.528
  01:35:32 STEP 2133 H=2133:00 +CPU= 30.969
  01:35:55 STEP 2134 H=2134:00 +CPU= 20.950
  01:36:17 STEP 2135 H=2135:00 +CPU= 20.244
  01:36:56 STEP 2136 H=2136:00 +CPU= 35.979
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+002136
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+002136
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+002136
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMGGhq0f+002112
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMSHhq0f+002112
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMUAhq0f+002112
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMGGhq0f+002124
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMSHhq0f+002124
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMUAhq0f+002124
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0454_2015050100_123_984_12201098_1_r725644482_88.zip
Uploading the intermediate file: upload_file_88.zip
  01:37:18 STEP 2137 H=2137:00 +CPU= 20.374
  01:37:42 STEP 2138 H=2138:00 +CPU= 21.339
  01:38:17 STEP 2139 H=2139:00 +CPU= 32.001
  01:38:41 STEP 2140 H=2140:00 +CPU= 21.582
  01:39:03 STEP 2141 H=2141:00 +CPU= 20.087
  01:39:37 STEP 2142 H=2142:00 +CPU= 31.165
  01:40:01 STEP 2143 H=2143:00 +CPU= 21.728
  01:40:25 STEP 2144 H=2144:00 +CPU= 21.404
  01:41:01 STEP 2145 H=2145:00 +CPU= 32.605
  01:41:25 STEP 2146 H=2146:00 +CPU= 21.225
  01:41:47 STEP 2147 H=2147:00 +CPU= 20.345
  01:42:26 STEP 2148 H=2148:00 +CPU= 34.994
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+002148
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+002148
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+002148
  01:42:48 STEP 2149 H=2149:00 +CPU= 20.773
  01:43:11 STEP 2150 H=2150:00 +CPU= 20.704
  01:43:42 STEP 2151 H=2151:00 +CPU= 28.490
  01:44:05 STEP 2152 H=2152:00 +CPU= 20.750
  01:44:27 STEP 2153 H=2153:00 +CPU= 20.529
  01:45:03 STEP 2154 H=2154:00 +CPU= 32.869
  01:45:26 STEP 2155 H=2155:00 +CPU= 20.990
  01:45:49 STEP 2156 H=2156:00 +CPU= 21.098
  01:46:26 STEP 2157 H=2157:00 +CPU= 32.825
  01:46:50 STEP 2158 H=2158:00 +CPU= 21.775
  01:47:14 STEP 2159 H=2159:00 +CPU= 21.744
  01:47:55 STEP 2160 H=2160:00 +CPU= 37.349
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+002160
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+002160
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+002160
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMGGhq0f+002136
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMSHhq0f+002136
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMUAhq0f+002136
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMGGhq0f+002148
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMSHhq0f+002148
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMUAhq0f+002148
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0454_2015050100_123_984_12201098_1_r725644482_89.zip
Uploading the intermediate file: upload_file_89.zip
  01:48:18 STEP 2161 H=2161:00 +CPU= 20.513
  01:48:39 STEP 2162 H=2162:00 +CPU= 19.941
  01:49:15 STEP 2163 H=2163:00 +CPU= 32.098
  01:49:38 STEP 2164 H=2164:00 +CPU= 21.002
  01:49:59 STEP 2165 H=2165:00 +CPU= 19.937
  01:50:34 STEP 2166 H=2166:00 +CPU= 31.904
  01:50:57 STEP 2167 H=2167:00 +CPU= 21.056
  01:51:19 STEP 2168 H=2168:00 +CPU= 20.547
  01:51:54 STEP 2169 H=2169:00 +CPU= 32.081
  01:52:17 STEP 2170 H=2170:00 +CPU= 21.103
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:52:56 STEP 2171 H=2171:00 +CPU= 20.720
  01:53:35 STEP 2172 H=2172:00 +CPU= 35.419
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+002172
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+002172
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+002172
  01:53:59 STEP 2173 H=2173:00 +CPU= 21.279
  01:54:22 STEP 2174 H=2174:00 +CPU= 21.171
  01:54:57 STEP 2175 H=2175:00 +CPU= 31.302
  01:55:19 STEP 2176 H=2176:00 +CPU= 20.722
  01:55:42 STEP 2177 H=2177:00 +CPU= 20.473
  01:56:17 STEP 2178 H=2178:00 +CPU= 31.899
  01:56:40 STEP 2179 H=2179:00 +CPU= 21.182
  01:57:04 STEP 2180 H=2180:00 +CPU= 21.796
  01:57:41 STEP 2181 H=2181:00 +CPU= 33.129
  01:58:06 STEP 2182 H=2182:00 +CPU= 22.567
  01:58:29 STEP 2183 H=2183:00 +CPU= 20.777
  01:59:07 STEP 2184 H=2184:00 +CPU= 35.067
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+002184
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+002184
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+002184
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMGGhq0f+002160
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMSHhq0f+002160
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMUAhq0f+002160
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMGGhq0f+002172
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMSHhq0f+002172
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMUAhq0f+002172
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0454_2015050100_123_984_12201098_1_r725644482_90.zip
Uploading the intermediate file: upload_file_90.zip
  01:59:29 STEP 2185 H=2185:00 +CPU= 20.531
  01:59:53 STEP 2186 H=2186:00 +CPU= 21.639
  02:00:29 STEP 2187 H=2187:00 +CPU= 32.375
  02:00:53 STEP 2188 H=2188:00 +CPU= 21.634
  02:01:15 STEP 2189 H=2189:00 +CPU= 20.186
  02:01:49 STEP 2190 H=2190:00 +CPU= 31.270
  02:02:12 STEP 2191 H=2191:00 +CPU= 20.778
  02:02:36 STEP 2192 H=2192:00 +CPU= 21.517
  02:03:10 STEP 2193 H=2193:00 +CPU= 31.203
  02:03:33 STEP 2194 H=2194:00 +CPU= 20.837
  02:03:56 STEP 2195 H=2195:00 +CPU= 19.922
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:05:09 STEP 2196 H=2196:00 +CPU= 36.948
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+002196
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+002196
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+002196
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:06:11 STEP 2197 H=2197:00 +CPU= 23.082
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:06:58 STEP 2198 H=2198:00 +CPU= 21.566
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:08:06 STEP 2199 H=2199:00 +CPU= 32.238
  02:08:29 STEP 2200 H=2200:00 +CPU= 20.819
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:09:06 STEP 2201 H=2201:00 +CPU= 21.858
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:10:07 STEP 2202 H=2202:00 +CPU= 33.648
  02:10:31 STEP 2203 H=2203:00 +CPU= 21.321
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:11:06 STEP 2204 H=2204:00 +CPU= 21.996
  02:11:42 STEP 2205 H=2205:00 +CPU= 32.050
  02:12:06 STEP 2206 H=2206:00 +CPU= 22.124
  02:12:29 STEP 2207 H=2207:00 +CPU= 20.955
  02:13:10 STEP 2208 H=2208:00 +CPU= 37.202
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+002208
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+002208
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+002208
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMGGhq0f+002184
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMSHhq0f+002184
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMUAhq0f+002184
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMGGhq0f+002196
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMSHhq0f+002196
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMUAhq0f+002196
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0454_2015050100_123_984_12201098_1_r725644482_91.zip
Uploading the intermediate file: upload_file_91.zip
  02:13:34 STEP 2209 H=2209:00 +CPU= 21.330
  02:13:58 STEP 2210 H=2210:00 +CPU= 21.897
  02:14:32 STEP 2211 H=2211:00 +CPU= 31.587
  02:14:56 STEP 2212 H=2212:00 +CPU= 21.831
  02:15:20 STEP 2213 H=2213:00 +CPU= 20.719
  02:15:55 STEP 2214 H=2214:00 +CPU= 32.454
  02:16:18 STEP 2215 H=2215:00 +CPU= 21.055
  02:16:42 STEP 2216 H=2216:00 +CPU= 21.396
  02:17:17 STEP 2217 H=2217:00 +CPU= 31.400
  02:17:42 STEP 2218 H=2218:00 +CPU= 22.212
  02:18:04 STEP 2219 H=2219:00 +CPU= 20.686
  02:18:42 STEP 2220 H=2220:00 +CPU= 34.674
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+002220
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+002220
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+002220
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:19:17 STEP 2221 H=2221:00 +CPU= 20.903
  02:19:39 STEP 2222 H=2222:00 +CPU= 20.723
  02:20:15 STEP 2223 H=2223:00 +CPU= 31.787
  02:20:36 STEP 2224 H=2224:00 +CPU= 20.226
  02:20:57 STEP 2225 H=2225:00 +CPU= 19.998
  02:21:34 STEP 2226 H=2226:00 +CPU= 32.814
  02:21:58 STEP 2227 H=2227:00 +CPU= 21.375
  02:22:22 STEP 2228 H=2228:00 +CPU= 21.544
  02:22:58 STEP 2229 H=2229:00 +CPU= 32.245
  02:23:21 STEP 2230 H=2230:00 +CPU= 21.391
  02:23:45 STEP 2231 H=2231:00 +CPU= 21.328
  02:24:24 STEP 2232 H=2232:00 +CPU= 35.914
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+002232
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+002232
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+002232
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMGGhq0f+002208
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMSHhq0f+002208
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMUAhq0f+002208
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMGGhq0f+002220
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMSHhq0f+002220
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMUAhq0f+002220
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0454_2015050100_123_984_12201098_1_r725644482_92.zip
Uploading the intermediate file: upload_file_92.zip
Uploading trickle at timestep: 8035200
  02:24:47 STEP 2233 H=2233:00 +CPU= 20.478
  02:25:10 STEP 2234 H=2234:00 +CPU= 20.993
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:25:58 STEP 2235 H=2235:00 +CPU= 32.521
  02:26:23 STEP 2236 H=2236:00 +CPU= 22.222
  02:26:45 STEP 2237 H=2237:00 +CPU= 20.247
  02:27:20 STEP 2238 H=2238:00 +CPU= 31.630
  02:27:42 STEP 2239 H=2239:00 +CPU= 20.339
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:28:17 STEP 2240 H=2240:00 +CPU= 21.621
  02:28:53 STEP 2241 H=2241:00 +CPU= 32.468
  02:29:18 STEP 2242 H=2242:00 +CPU= 22.505
  02:29:41 STEP 2243 H=2243:00 +CPU= 20.753
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:30:31 STEP 2244 H=2244:00 +CPU= 35.305
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+002244
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+002244
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+002244
  02:30:54 STEP 2245 H=2245:00 +CPU= 20.423
  02:31:16 STEP 2246 H=2246:00 +CPU= 20.666
  02:31:50 STEP 2247 H=2247:00 +CPU= 31.112
  02:32:14 STEP 2248 H=2248:00 +CPU= 22.026
  02:32:37 STEP 2249 H=2249:00 +CPU= 20.512
  02:33:11 STEP 2250 H=2250:00 +CPU= 31.320
  02:33:34 STEP 2251 H=2251:00 +CPU= 21.125
  02:34:00 STEP 2252 H=2252:00 +CPU= 22.836
  02:34:35 STEP 2253 H=2253:00 +CPU= 32.011
  02:35:00 STEP 2254 H=2254:00 +CPU= 22.591
  02:35:24 STEP 2255 H=2255:00 +CPU= 21.410
  02:36:04 STEP 2256 H=2256:00 +CPU= 36.526
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+002256
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+002256
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+002256
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMGGhq0f+002232
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMSHhq0f+002232
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMUAhq0f+002232
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMGGhq0f+002244
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMSHhq0f+002244
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMUAhq0f+002244
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0454_2015050100_123_984_12201098_1_r725644482_93.zip
Uploading the intermediate file: upload_file_93.zip
  02:36:28 STEP 2257 H=2257:00 +CPU= 21.353
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:37:04 STEP 2258 H=2258:00 +CPU= 22.417
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:37:51 STEP 2259 H=2259:00 +CPU= 32.785
  02:38:16 STEP 2260 H=2260:00 +CPU= 21.944
  02:38:38 STEP 2261 H=2261:00 +CPU= 20.020
  02:39:14 STEP 2262 H=2262:00 +CPU= 32.396
  02:39:36 STEP 2263 H=2263:00 +CPU= 20.198
  02:39:58 STEP 2264 H=2264:00 +CPU= 20.154
  02:40:33 STEP 2265 H=2265:00 +CPU= 32.154
  02:40:58 STEP 2266 H=2266:00 +CPU= 22.104
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:41:33 STEP 2267 H=2267:00 +CPU= 21.704
  02:42:12 STEP 2268 H=2268:00 +CPU= 35.223
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+002268
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+002268
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+002268
  02:42:35 STEP 2269 H=2269:00 +CPU= 21.185
  02:43:00 STEP 2270 H=2270:00 +CPU= 22.019
  02:43:34 STEP 2271 H=2271:00 +CPU= 30.559
  02:43:58 STEP 2272 H=2272:00 +CPU= 21.815
  02:44:22 STEP 2273 H=2273:00 +CPU= 21.453
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:45:22 STEP 2274 H=2274:00 +CPU= 33.730
  02:45:46 STEP 2275 H=2275:00 +CPU= 21.663
  02:46:10 STEP 2276 H=2276:00 +CPU= 21.798
  02:46:44 STEP 2277 H=2277:00 +CPU= 30.984
  02:47:08 STEP 2278 H=2278:00 +CPU= 21.698
  02:47:33 STEP 2279 H=2279:00 +CPU= 22.100
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:48:25 STEP 2280 H=2280:00 +CPU= 37.354
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+002280
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+002280
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+002280
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMGGhq0f+002256
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMSHhq0f+002256
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMUAhq0f+002256
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMGGhq0f+002268
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMSHhq0f+002268
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMUAhq0f+002268
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0454_2015050100_123_984_12201098_1_r725644482_94.zip
Uploading the intermediate file: upload_file_94.zip
  02:48:49 STEP 2281 H=2281:00 +CPU= 21.577
  02:49:13 STEP 2282 H=2282:00 +CPU= 21.418
  02:49:47 STEP 2283 H=2283:00 +CPU= 31.229
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:50:23 STEP 2284 H=2284:00 +CPU= 21.992
  02:50:46 STEP 2285 H=2285:00 +CPU= 20.617
  02:51:23 STEP 2286 H=2286:00 +CPU= 33.009
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:51:58 STEP 2287 H=2287:00 +CPU= 21.689
  02:52:21 STEP 2288 H=2288:00 +CPU= 21.312
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:53:09 STEP 2289 H=2289:00 +CPU= 32.682
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:53:44 STEP 2290 H=2290:00 +CPU= 21.962
  02:54:07 STEP 2291 H=2291:00 +CPU= 20.851
  02:54:44 STEP 2292 H=2292:00 +CPU= 34.025
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+002292
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+002292
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+002292
  02:55:07 STEP 2293 H=2293:00 +CPU= 20.821
  02:55:28 STEP 2294 H=2294:00 +CPU= 19.640
  02:56:01 STEP 2295 H=2295:00 +CPU= 30.445
  02:56:26 STEP 2296 H=2296:00 +CPU= 22.389
  02:56:49 STEP 2297 H=2297:00 +CPU= 21.519
  02:57:23 STEP 2298 H=2298:00 +CPU= 30.024
  02:57:47 STEP 2299 H=2299:00 +CPU= 21.379
  02:58:11 STEP 2300 H=2300:00 +CPU= 21.921
  02:58:47 STEP 2301 H=2301:00 +CPU= 32.572
  02:59:11 STEP 2302 H=2302:00 +CPU= 21.747
  02:59:36 STEP 2303 H=2303:00 +CPU= 22.658
  03:00:18 STEP 2304 H=2304:00 +CPU= 37.898
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+002304
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+002304
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+002304
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMGGhq0f+002280
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMSHhq0f+002280
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMUAhq0f+002280
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMGGhq0f+002292
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMSHhq0f+002292
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMUAhq0f+002292
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0454_2015050100_123_984_12201098_1_r725644482_95.zip
Uploading the intermediate file: upload_file_95.zip
  03:00:41 STEP 2305 H=2305:00 +CPU= 21.099
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  03:01:18 STEP 2306 H=2306:00 +CPU= 23.018
  03:01:54 STEP 2307 H=2307:00 +CPU= 32.788
  03:02:19 STEP 2308 H=2308:00 +CPU= 22.239
  03:02:44 STEP 2309 H=2309:00 +CPU= 21.857
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  03:03:32 STEP 2310 H=2310:00 +CPU= 33.329
  03:03:55 STEP 2311 H=2311:00 +CPU= 21.452
  03:04:20 STEP 2312 H=2312:00 +CPU= 21.894
  03:04:54 STEP 2313 H=2313:00 +CPU= 30.721
  03:05:15 STEP 2314 H=2314:00 +CPU= 20.320
  03:05:35 STEP 2315 H=2315:00 +CPU= 19.317
  03:06:07 STEP 2316 H=2316:00 +CPU= 30.910
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+002316
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+002316
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+002316
  03:06:26 STEP 2317 H=2317:00 +CPU= 18.785
  03:06:46 STEP 2318 H=2318:00 +CPU= 18.960
  03:07:15 STEP 2319 H=2319:00 +CPU= 28.182
  03:07:34 STEP 2320 H=2320:00 +CPU= 18.251
  03:07:52 STEP 2321 H=2321:00 +CPU= 18.163
  03:08:21 STEP 2322 H=2322:00 +CPU= 28.460
  03:08:41 STEP 2323 H=2323:00 +CPU= 18.828
  03:09:01 STEP 2324 H=2324:00 +CPU= 19.052
  03:09:30 STEP 2325 H=2325:00 +CPU= 28.364
  03:09:50 STEP 2326 H=2326:00 +CPU= 18.899
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  03:10:24 STEP 2327 H=2327:00 +CPU= 18.971
  03:10:59 STEP 2328 H=2328:00 +CPU= 33.488
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+002328
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+002328
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+002328
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMGGhq0f+002304
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMSHhq0f+002304
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMUAhq0f+002304
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMGGhq0f+002316
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMSHhq0f+002316
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMUAhq0f+002316
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0454_2015050100_123_984_12201098_1_r725644482_96.zip
Uploading the intermediate file: upload_file_96.zip
  03:11:19 STEP 2329 H=2329:00 +CPU= 19.504
  03:11:39 STEP 2330 H=2330:00 +CPU= 19.338
  03:12:10 STEP 2331 H=2331:00 +CPU= 29.597
  03:12:32 STEP 2332 H=2332:00 +CPU= 20.251
  03:12:52 STEP 2333 H=2333:00 +CPU= 18.861
  03:13:24 STEP 2334 H=2334:00 +CPU= 29.750
  03:13:46 STEP 2335 H=2335:00 +CPU= 20.706
  03:14:07 STEP 2336 H=2336:00 +CPU= 20.412
  03:14:39 STEP 2337 H=2337:00 +CPU= 29.860
  03:15:02 STEP 2338 H=2338:00 +CPU= 21.119
  03:15:22 STEP 2339 H=2339:00 +CPU= 18.882
  03:15:54 STEP 2340 H=2340:00 +CPU= 30.993
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+002340
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+002340
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+002340
  03:16:12 STEP 2341 H=2341:00 +CPU= 17.729
  03:16:29 STEP 2342 H=2342:00 +CPU= 16.391
  03:16:54 STEP 2343 H=2343:00 +CPU= 24.471
  03:17:12 STEP 2344 H=2344:00 +CPU= 17.838
  03:17:29 STEP 2345 H=2345:00 +CPU= 16.620
  03:17:57 STEP 2346 H=2346:00 +CPU= 26.956
  03:18:15 STEP 2347 H=2347:00 +CPU= 17.365
  03:18:34 STEP 2348 H=2348:00 +CPU= 17.886
  03:19:01 STEP 2349 H=2349:00 +CPU= 26.475
  03:19:18 STEP 2350 H=2350:00 +CPU= 17.134
  03:19:36 STEP 2351 H=2351:00 +CPU= 17.265
  03:20:09 STEP 2352 H=2352:00 +CPU= 31.553
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+002352
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+002352
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+002352
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMGGhq0f+002328
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMSHhq0f+002328
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMUAhq0f+002328
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMGGhq0f+002340
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMSHhq0f+002340
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMUAhq0f+002340
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0454_2015050100_123_984_12201098_1_r725644482_97.zip
Uploading the intermediate file: upload_file_97.zip
  03:20:27 STEP 2353 H=2353:00 +CPU= 17.277
  03:20:44 STEP 2354 H=2354:00 +CPU= 17.104
  03:21:11 STEP 2355 H=2355:00 +CPU= 26.243
  03:21:29 STEP 2356 H=2356:00 +CPU= 16.866
  03:21:46 STEP 2357 H=2357:00 +CPU= 16.601
  03:22:14 STEP 2358 H=2358:00 +CPU= 28.242
  03:22:36 STEP 2359 H=2359:00 +CPU= 20.835
  03:22:59 STEP 2360 H=2360:00 +CPU= 21.531
  03:23:31 STEP 2361 H=2361:00 +CPU= 31.072
  03:23:53 STEP 2362 H=2362:00 +CPU= 21.402
  03:24:15 STEP 2363 H=2363:00 +CPU= 21.101
  03:24:51 STEP 2364 H=2364:00 +CPU= 34.777
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+002364
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+002364
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+002364
  03:25:13 STEP 2365 H=2365:00 +CPU= 20.881
  03:25:35 STEP 2366 H=2366:00 +CPU= 21.488
  03:26:08 STEP 2367 H=2367:00 +CPU= 31.569
  03:26:30 STEP 2368 H=2368:00 +CPU= 21.303
  03:26:53 STEP 2369 H=2369:00 +CPU= 21.405
  03:27:24 STEP 2370 H=2370:00 +CPU= 31.330
  03:27:46 STEP 2371 H=2371:00 +CPU= 21.130
  03:28:08 STEP 2372 H=2372:00 +CPU= 21.146
  03:28:40 STEP 2373 H=2373:00 +CPU= 30.994
  03:29:01 STEP 2374 H=2374:00 +CPU= 20.874
  03:29:22 STEP 2375 H=2375:00 +CPU= 20.257
  03:29:59 STEP 2376 H=2376:00 +CPU= 36.725
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+002376
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+002376
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+002376
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMGGhq0f+002352
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMSHhq0f+002352
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMUAhq0f+002352
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMGGhq0f+002364
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMSHhq0f+002364
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMUAhq0f+002364
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0454_2015050100_123_984_12201098_1_r725644482_98.zip
Uploading the intermediate file: upload_file_98.zip
  03:30:21 STEP 2377 H=2377:00 +CPU= 20.756
  03:30:43 STEP 2378 H=2378:00 +CPU= 21.244
  03:31:16 STEP 2379 H=2379:00 +CPU= 31.800
  03:31:38 STEP 2380 H=2380:00 +CPU= 21.590
  03:31:59 STEP 2381 H=2381:00 +CPU= 20.711
  03:32:32 STEP 2382 H=2382:00 +CPU= 31.844
  03:32:53 STEP 2383 H=2383:00 +CPU= 20.583
  03:33:15 STEP 2384 H=2384:00 +CPU= 21.187
  03:33:48 STEP 2385 H=2385:00 +CPU= 31.419
  03:34:09 STEP 2386 H=2386:00 +CPU= 21.154
  03:34:32 STEP 2387 H=2387:00 +CPU= 21.366
  03:35:08 STEP 2388 H=2388:00 +CPU= 35.217
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+002388
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+002388
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+002388
  03:35:30 STEP 2389 H=2389:00 +CPU= 21.444
  03:35:53 STEP 2390 H=2390:00 +CPU= 21.446
  03:36:26 STEP 2391 H=2391:00 +CPU= 31.777
  03:36:48 STEP 2392 H=2392:00 +CPU= 21.327
  03:37:10 STEP 2393 H=2393:00 +CPU= 21.096
  03:37:43 STEP 2394 H=2394:00 +CPU= 32.076
  03:38:04 STEP 2395 H=2395:00 +CPU= 19.790
  03:38:25 STEP 2396 H=2396:00 +CPU= 19.644
  03:38:54 STEP 2397 H=2397:00 +CPU= 28.699
  03:39:14 STEP 2398 H=2398:00 +CPU= 19.365
  03:39:34 STEP 2399 H=2399:00 +CPU= 18.998
  03:40:08 STEP 2400 H=2400:00 +CPU= 33.041
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+002400
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+002400
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+002400
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMGGhq0f+002376
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMSHhq0f+002376
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMUAhq0f+002376
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMGGhq0f+002388
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMSHhq0f+002388
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMUAhq0f+002388
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0454_2015050100_123_984_12201098_1_r725644482_99.zip
Uploading the intermediate file: upload_file_99.zip
  03:40:27 STEP 2401 H=2401:00 +CPU= 18.319
  03:40:46 STEP 2402 H=2402:00 +CPU= 18.698
  03:41:14 STEP 2403 H=2403:00 +CPU= 28.139
  03:41:34 STEP 2404 H=2404:00 +CPU= 19.299
  03:41:53 STEP 2405 H=2405:00 +CPU= 18.142
  03:42:22 STEP 2406 H=2406:00 +CPU= 28.081
  03:42:40 STEP 2407 H=2407:00 +CPU= 18.295
  03:42:59 STEP 2408 H=2408:00 +CPU= 18.451
  03:43:27 STEP 2409 H=2409:00 +CPU= 27.731
  03:43:48 STEP 2410 H=2410:00 +CPU= 19.512
  03:44:07 STEP 2411 H=2411:00 +CPU= 18.827
  03:44:39 STEP 2412 H=2412:00 +CPU= 31.078
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+002412
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+002412
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+002412
  03:44:58 STEP 2413 H=2413:00 +CPU= 18.873
  03:45:18 STEP 2414 H=2414:00 +CPU= 19.507
  03:45:47 STEP 2415 H=2415:00 +CPU= 28.101
  03:46:08 STEP 2416 H=2416:00 +CPU= 19.542
  03:46:26 STEP 2417 H=2417:00 +CPU= 18.189
  03:46:57 STEP 2418 H=2418:00 +CPU= 29.189
  03:47:16 STEP 2419 H=2419:00 +CPU= 18.846
  03:47:36 STEP 2420 H=2420:00 +CPU= 19.408
  03:48:05 STEP 2421 H=2421:00 +CPU= 28.442
  03:48:25 STEP 2422 H=2422:00 +CPU= 19.172
  03:48:45 STEP 2423 H=2423:00 +CPU= 19.216
  03:49:20 STEP 2424 H=2424:00 +CPU= 34.155
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+002424
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+002424
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+002424
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMGGhq0f+002400
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMSHhq0f+002400
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMUAhq0f+002400
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMGGhq0f+002412
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMSHhq0f+002412
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMUAhq0f+002412
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0454_2015050100_123_984_12201098_1_r725644482_100.zip
Uploading the intermediate file: upload_file_100.zip
  03:49:40 STEP 2425 H=2425:00 +CPU= 18.889
  03:50:01 STEP 2426 H=2426:00 +CPU= 19.799
  03:50:31 STEP 2427 H=2427:00 +CPU= 28.807
  03:50:50 STEP 2428 H=2428:00 +CPU= 19.079
  03:51:11 STEP 2429 H=2429:00 +CPU= 19.409
  03:51:43 STEP 2430 H=2430:00 +CPU= 31.182
  03:52:05 STEP 2431 H=2431:00 +CPU= 21.241
  03:52:27 STEP 2432 H=2432:00 +CPU= 21.522
  03:53:00 STEP 2433 H=2433:00 +CPU= 31.857
  03:53:22 STEP 2434 H=2434:00 +CPU= 22.072
  03:53:44 STEP 2435 H=2435:00 +CPU= 21.165
  03:54:22 STEP 2436 H=2436:00 +CPU= 35.906
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+002436
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+002436
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+002436
  03:54:44 STEP 2437 H=2437:00 +CPU= 21.091
  03:55:06 STEP 2438 H=2438:00 +CPU= 21.801
  03:55:39 STEP 2439 H=2439:00 +CPU= 31.893
  03:56:02 STEP 2440 H=2440:00 +CPU= 21.914
  03:56:24 STEP 2441 H=2441:00 +CPU= 20.924
  03:56:56 STEP 2442 H=2442:00 +CPU= 31.753
  03:57:18 STEP 2443 H=2443:00 +CPU= 21.111
  03:57:41 STEP 2444 H=2444:00 +CPU= 21.916
  03:58:14 STEP 2445 H=2445:00 +CPU= 31.739
  03:58:36 STEP 2446 H=2446:00 +CPU= 21.728
  03:58:59 STEP 2447 H=2447:00 +CPU= 21.913
  03:59:37 STEP 2448 H=2448:00 +CPU= 36.897
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+002448
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+002448
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+002448
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMGGhq0f+002424
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMSHhq0f+002424
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMUAhq0f+002424
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMGGhq0f+002436
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMSHhq0f+002436
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMUAhq0f+002436
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0454_2015050100_123_984_12201098_1_r725644482_101.zip
Uploading the intermediate file: upload_file_101.zip
  03:59:59 STEP 2449 H=2449:00 +CPU= 20.979
  04:00:22 STEP 2450 H=2450:00 +CPU= 22.061
  04:00:56 STEP 2451 H=2451:00 +CPU= 32.638
  04:01:18 STEP 2452 H=2452:00 +CPU= 21.585
  04:01:39 STEP 2453 H=2453:00 +CPU= 20.961
  04:02:12 STEP 2454 H=2454:00 +CPU= 32.207
  04:02:33 STEP 2455 H=2455:00 +CPU= 20.993
  04:02:56 STEP 2456 H=2456:00 +CPU= 21.795
  04:03:28 STEP 2457 H=2457:00 +CPU= 31.706
  04:03:51 STEP 2458 H=2458:00 +CPU= 21.650
  04:04:12 STEP 2459 H=2459:00 +CPU= 20.385
  04:04:45 STEP 2460 H=2460:00 +CPU= 33.320
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+002460
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+002460
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+002460
  04:05:04 STEP 2461 H=2461:00 +CPU= 19.172
  04:05:25 STEP 2462 H=2462:00 +CPU= 20.585
  04:05:55 STEP 2463 H=2463:00 +CPU= 30.036
  04:06:16 STEP 2464 H=2464:00 +CPU= 20.313
  04:06:35 STEP 2465 H=2465:00 +CPU= 19.545
  04:07:08 STEP 2466 H=2466:00 +CPU= 32.418
  04:07:34 STEP 2467 H=2467:00 +CPU= 19.839
  04:08:10 STEP 2468 H=2468:00 +CPU= 23.947
  04:09:13 STEP 2469 H=2469:00 +CPU= 37.448
  04:09:56 STEP 2470 H=2470:00 +CPU= 24.944
  04:10:25 STEP 2471 H=2471:00 +CPU= 23.983
  04:11:15 STEP 2472 H=2472:00 +CPU= 40.495
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+002472
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+002472
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+002472
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMGGhq0f+002448
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMSHhq0f+002448
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMUAhq0f+002448
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMGGhq0f+002460
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMSHhq0f+002460
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMUAhq0f+002460
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0454_2015050100_123_984_12201098_1_r725644482_102.zip
Uploading the intermediate file: upload_file_102.zip
Uploading trickle at timestep: 8899200
  04:11:42 STEP 2473 H=2473:00 +CPU= 22.958
  04:12:12 STEP 2474 H=2474:00 +CPU= 23.978
  04:12:45 STEP 2475 H=2475:00 +CPU= 32.006
  04:13:06 STEP 2476 H=2476:00 +CPU= 20.843
  04:13:26 STEP 2477 H=2477:00 +CPU= 20.248
  04:13:58 STEP 2478 H=2478:00 +CPU= 31.747
  04:14:18 STEP 2479 H=2479:00 +CPU= 19.873
  04:14:39 STEP 2480 H=2480:00 +CPU= 20.343
  04:15:10 STEP 2481 H=2481:00 +CPU= 31.138
  04:15:32 STEP 2482 H=2482:00 +CPU= 21.477
  04:15:52 STEP 2483 H=2483:00 +CPU= 20.114
  04:16:26 STEP 2484 H=2484:00 +CPU= 33.962
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+002484
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+002484
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+002484
  04:16:46 STEP 2485 H=2485:00 +CPU= 20.097
  04:17:07 STEP 2486 H=2486:00 +CPU= 20.921
  04:18:01 STEP 2487 H=2487:00 +CPU= 35.471
[EC_DRHOOK:philip-All-Series:1:1:2935:2935] [20221231:041828:1672456708:21331.713] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1538] Received signal#15 (SIGTERM) :: 3996MB (heap), 4419MB (maxrss), 0MB (maxstack), 0 (paging), nsigs = 1
[EC_DRHOOK:philip-All-Series:1:1:2935:2935] [20221231:041828:1672456708:21331.713] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1542] Also activating Harakiri-alarm (SIGALRM=14) to expire after 500s elapsed to prevent hangs, nsigs = 1
[EC_DRHOOK:philip-All-Series:1:1:2935:2935] [20221231:041828:1672456708:21331.713] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1544] Harakiri signal handler 'signal_harakiri' for signal#14 (SIGALRM) installed at 0x81f3c0 (old at (nil))
[EC_DRHOOK:philip-All-Series:1:1:2935:2935] [20221231:041828:1672456708:21331.714] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1617] Signal#15 was caused by unrecognized si_code [memaddr=0x1], nsigs = 1
[EC_DRHOOK:philip-All-Series:1:1:2935:2935] [20221231:041828:1672456708:21331.714] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1686] Starting DrHook backtrace for signal#15, nsigs = 1
[EC_DRHOOK:philip-All-Series:1:1:2935:2935] [20221231:041828:1672456708:21331.714] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3843] 3996 MB (maxheap), 4419 MB (maxrss), 0 MB (maxstack)
[EC_DRHOOK:philip-All-Series:1:1:2935:2935] [20221231:041828:1672456708:21331.715] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :  MASTER 
[EC_DRHOOK:philip-All-Series:1:1:2935:2935] [20221231:041828:1672456708:21331.717] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :   CNT0 
[EC_DRHOOK:philip-All-Series:1:1:2935:2935] [20221231:041828:1672456708:21331.717] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :    CNT1 
[EC_DRHOOK:philip-All-Series:1:1:2935:2935] [20221231:041828:1672456708:21331.717] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :     CNT2 
[EC_DRHOOK:philip-All-Series:1:1:2935:2935] [20221231:041828:1672456708:21331.723] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :      CNT3 
[EC_DRHOOK:philip-All-Series:1:1:2935:2935] [20221231:041828:1672456708:21331.723] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :       CNT4 
[EC_DRHOOK:philip-All-Series:1:1:2935:2935] [20221231:041828:1672456708:21331.723] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :        STEPO 
[EC_DRHOOK:philip-All-Series:1:1:2935:2935] [20221231:041828:1672456708:21331.723] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :         SCAN2M 
[EC_DRHOOK:philip-All-Series:1:1:2935:2935] [20221231:041828:1672456708:21331.723] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :          GP_MODEL_HEAP 
[EC_DRHOOK:philip-All-Series:1:1:2935:2935] [20221231:041828:1672456708:21331.723] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :           GP_MODEL 
[EC_DRHOOK:philip-All-Series:1:1:2935:2935] [20221231:041828:1672456708:21331.723] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :            EC_PHYS_DRV 
[EC_DRHOOK:philip-All-Series:1:1:2935:2935] [20221231:041828:1672456708:21331.723] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :             >OMP-PHYSICS              (1001) 
[EC_DRHOOK:philip-All-Series:1:1:2935:2935] [20221231:041828:1672456708:21331.723] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :              EC_PHYS 
[EC_DRHOOK:philip-All-Series:1:1:2935:2935] [20221231:041828:1672456708:21331.723] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :               CALLPAR 
[EC_DRHOOK:philip-All-Series:1:1:2935:2935] [20221231:041828:1672456708:21331.723] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :                CUANCAPE2 
[EC_DRHOOK:philip-All-Series:1:1:2935:2935] [20221231:041829:1672456709:21332.723] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1734] DrHook backtrace done for signal#15, nsigs = 1
[EC_DRHOOK:philip-All-Series:1:1:2935:2935] [20221231:041829:1672456709:21332.723] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1785] Calling previous signal handler at 0x1ce8bb0 for signal#15, nsigs = 1
forrtl: error (78): process killed (SIGTERM)
Image              PC                Routine            Line        Source             
oifs_43r3_model.e  0000000001CE916B  Unknown               Unknown  Unknown
oifs_43r3_model.e  000000000081FFF1  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001DC9090  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001247220  cuancape2_                101  cuancape2.F90
oifs_43r3_model.e  0000000000F14871  callpar_                 1102  callpar.F90
oifs_43r3_model.e  0000000000E4940F  ec_phys_                  670  ec_phys.F90
oifs_43r3_model.e  0000000000E3610C  ec_phys_drv_              599  ec_phys_drv.F90
oifs_43r3_model.e  0000000000BFA3D3  gp_model_                 613  gp_model.F90
oifs_43r3_model.e  00000000012E9482  gp_model_heap_             74  gp_model_heap.F90
oifs_43r3_model.e  0000000000BBC1CE  scan2m_                   535  scan2m.F90
oifs_43r3_model.e  000000000057A664  stepo_                    327  stepo.F90
oifs_43r3_model.e  000000000055DEEC  cnt4_                    1133  cnt4.F90
oifs_43r3_model.e  00000000005471C9  cnt3_                     267  cnt3.F90
oifs_43r3_model.e  0000000000546412  cnt2_                      88  cnt2.F90
oifs_43r3_model.e  0000000000545E28  cnt1_                      92  cnt1.F90
oifs_43r3_model.e  000000000040708D  cnt0_                     146  cnt0.F90
oifs_43r3_model.e  000000000040220F  MAIN__                     96  master.F90
oifs_43r3_model.e  00000000004021A2  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001DCA390  Unknown               Unknown  Unknown
oifs_43r3_model.e  000000000040206E  Unknown               Unknown  Unknown
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 2015050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0454
(argv4) batchid: 984
(argv5) wuid: 12201098
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/5
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/5/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/5/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_462809e40f8a2469bc21950013cb7c4c to: /var/lib/boinc-client/slots/5/oifs_43r3_ps_0454_2015050100_123_984_12201098.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/5/oifs_43r3_ps_0454_2015050100_123_984_12201098.zip
ic_ancil_file: ic_ancil_12201098
ifsdata_file: ifsdata_12201098
climate_data_file: clim_data_12201098
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_f24b10480ff6497a4f4b1b9158d6fc62 to: /var/lib/boinc-client/slots/5/ic_ancil_12201098.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/5/ic_ancil_12201098.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_cdf216a866b78441343dd452083c3828 to: /var/lib/boinc-client/slots/5/ifsdata/ifsdata_12201098.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/5/ifsdata/ifsdata_12201098.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/5/159l_2/clim_data_12201098.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/5/159l_2/clim_data_12201098.zip
Checking for progress XML file: /var/lib/boinc-client/slots/5/progress_file_12201098.xml
Opened progress file ok : /var/lib/boinc-client/slots/5/progress_file_12201098.xml
-- Model is restarting --
Adjusting last_iter, 2488, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/5/progress_file_12201098.xml
last_cpu_time: 67751
upload_file_number: 103
last_iter: 2471
last_upload: 8899200
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0454_2015050100_123_984_12201098_1_r725644482
The child process has been launched with process id: 2915
Executing the command: /var/lib/boinc-client/slots/5/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:philip-All-Series:1:1:2915:2915] [20221231:224641:1672523201:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:philip-All-Series:1:1:2915:2915] [20221231:224641:1672523201:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:philip-All-Series:1:1:2915:2915] [20221231:224641:1672523201:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:philip-All-Series:1:1:2915:2915] [20221231:224641:1672523201:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:philip-All-Series:1:1:2915:2915] [20221231:224641:1672523201:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:philip-All-Series:1:1:2915:2915] [20221231:224641:1672523201:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:philip-All-Series:1:1:2915:2915] [20221231:224641:1672523201:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:philip-All-Series:1:1:2915:2915] [20221231:224641:1672523201:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:philip-All-Series:1:1:2915:2915] [20221231:224641:1672523201:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:philip-All-Series:1:1:2915:2915] [20221231:224641:1672523201:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:philip-All-Series:1:1:2915:2915] [20221231:224641:1672523201:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:philip-All-Series:1:1:2915:2915] [20221231:224641:1672523201:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:philip-All-Series:1:1:2915:2915] [20221231:224641:1672523201:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:philip-All-Series:1:1:2915:2915] [20221231:224641:1672523201:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:philip-All-Series:1:1:2915:2915] [20221231:224641:1672523201:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:philip-All-Series:1:1:2915:2915] [20221231:224641:1672523201:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:philip-All-Series:1:1:2915:2915] [20221231:224641:1672523201:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:philip-All-Series:1:1:2915:2915] [20221231:224641:1672523201:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:philip-All-Series:1:1:2915:2915] [20221231:224641:1672523201:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:philip-All-Series:1:1:2915:2915] [20221231:224641:1672523201:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:philip-All-Series:1:1:2915:2915] [20221231:224641:1672523201:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:philip-All-Series:1:1:2915:2915] [20221231:224641:1672523201:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:philip-All-Series:1:1:2915:2915] [20221231:224641:1672523201:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:philip-All-Series:1:1:2915:2915] [20221231:224641:1672523201:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:philip-All-Series:1:1:2915:2915] [20221231:224641:1672523201:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:philip-All-Series:1:1:2915:2915] [20221231:224641:1672523201:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:philip-All-Series:1:1:2915:2915] [20221231:224641:1672523201:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:philip-All-Series:1:1:2915:2915] [20221231:224641:1672523201:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:philip-All-Series:1:1:2915:2915] [20221231:224641:1672523201:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:philip-All-Series:1:1:2915:2915] [20221231:224641:1672523201:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  22:47:45 STEP 2472 H=2472:00 +CPU= 39.441
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  22:48:20 STEP 2473 H=2473:00 +CPU= 21.450
  22:48:42 STEP 2474 H=2474:00 +CPU= 21.307
  22:49:13 STEP 2475 H=2475:00 +CPU= 30.785
  22:49:35 STEP 2476 H=2476:00 +CPU= 21.169
  22:49:57 STEP 2477 H=2477:00 +CPU= 21.508
  22:50:34 STEP 2478 H=2478:00 +CPU= 34.103
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  22:51:07 STEP 2479 H=2479:00 +CPU= 21.135
  22:51:28 STEP 2480 H=2480:00 +CPU= 20.870
  22:51:59 STEP 2481 H=2481:00 +CPU= 30.708
  22:52:21 STEP 2482 H=2482:00 +CPU= 20.678
  22:52:41 STEP 2483 H=2483:00 +CPU= 20.375
  22:53:16 STEP 2484 H=2484:00 +CPU= 34.460
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+002484
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+002484
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+002484
  22:53:36 STEP 2485 H=2485:00 +CPU= 19.523
  22:53:57 STEP 2486 H=2486:00 +CPU= 20.418
  22:54:26 STEP 2487 H=2487:00 +CPU= 29.237
  22:54:47 STEP 2488 H=2488:00 +CPU= 20.192
  22:55:07 STEP 2489 H=2489:00 +CPU= 19.540
  22:55:38 STEP 2490 H=2490:00 +CPU= 30.944
  22:55:57 STEP 2491 H=2491:00 +CPU= 19.412
  22:56:19 STEP 2492 H=2492:00 +CPU= 20.949
  22:56:50 STEP 2493 H=2493:00 +CPU= 30.730
  22:57:11 STEP 2494 H=2494:00 +CPU= 21.086
  22:57:33 STEP 2495 H=2495:00 +CPU= 21.577
  22:58:10 STEP 2496 H=2496:00 +CPU= 36.165
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+002496
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+002496
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+002496
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMGGhq0f+002472
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMSHhq0f+002472
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMUAhq0f+002472
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMGGhq0f+002484
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMSHhq0f+002484
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201098/ICMUAhq0f+002484
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0454_2015050100_123_984_12201098_1_r725644482_103.zip
Uploading the intermediate file: upload_file_103.zip
  22:58:31 STEP 2497 H=2497:00 +CPU= 20.644
  22:58:52 STEP 2498 H=2498:00 +CPU= 20.761
  22:59:22 STEP 2499 H=2499:00 +CPU= 29.689
  22:59:43 STEP 2500 H=2500:00 +CPU= 20.470
  23:00:03 STEP 2501 H=2501:00 +CPU= 19.920
  23:00:33 STEP 2502 H=2502:00 +CPU= 30.469
  23:00:54 STEP 2503 H=2503:00 +CPU= 20.039
  23:01:14 STEP 2504 H=2504:00 +CPU= 20.038
Abort request received from BOINC client, ending the child process

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 May 2023 16:04:27 1527792 22293755 oifs_43r3_ps_0454_2015050100_123_984_12201098_1 8,899,200 67,367 0.0076
09 May 2023 16:03:31 1527792 22293755 oifs_43r3_ps_0454_2015050100_123_984_12201098_1 8,035,200 61,550 0.0077
09 May 2023 16:03:26 1527792 22293755 oifs_43r3_ps_0454_2015050100_123_984_12201098_1 7,171,200 55,490 0.0077
09 May 2023 16:02:23 1527792 22293755 oifs_43r3_ps_0454_2015050100_123_984_12201098_1 6,134,400 47,091 0.0077
09 May 2023 16:02:13 1527792 22293755 oifs_43r3_ps_0454_2015050100_123_984_12201098_1 5,270,400 40,737 0.0077
09 May 2023 16:02:05 1527792 22293755 oifs_43r3_ps_0454_2015050100_123_984_12201098_1 4,320,000 33,003 0.0076
09 May 2023 15:59:18 1527792 22293755 oifs_43r3_ps_0454_2015050100_123_984_12201098_1 3,283,200 24,062 0.0073
09 May 2023 15:58:02 1527792 22293755 oifs_43r3_ps_0454_2015050100_123_984_12201098_1 1,728,000 12,069 0.0070
09 May 2023 15:57:53 1527792 22293755 oifs_43r3_ps_0454_2015050100_123_984_12201098_1 864,000 5,964 0.0069


©2024 cpdn.org