climateprediction.net home page
Task 22279593

Task 22279593

Name oifs_43r3_ps_0139_2008050100_123_977_12193783_0
Workunit 12193783
Created 23 Dec 2022, 17:39:19 UTC
Sent 27 Dec 2022, 20:51:34 UTC
Report deadline 26 Jan 2023, 20:51:34 UTC
Received 28 Dec 2022, 11:35:42 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 2 (0x00000002) Unknown error code
Computer ID 1523867
Run time 12 hours 27 min 21 sec
CPU time 12 hours 27 min 21 sec
Validate state Invalid
Credit 6,048.00
Device peak FLOPS 5.17 GFLOPS
Application version OpenIFS 43r3 Perturbed Surface v1.05
x86_64-pc-linux-gnu
Peak working set size 4,410.79 MB
Peak swap size 4,978.16 MB
Peak disk usage 3,726.42 MB
Stderr
<core_client_version>7.16.6</core_client_version>
<![CDATA[
<message>
process exited with code 2 (0x2, -254)</message>
<stderr_txt>
cts directory: /var/lib/boinc-client/slots/10/ICMUAhq0f+001164
  07:33:41 STEP 1165 H=1165:00 +CPU= 20.053
  07:34:02 STEP 1166 H=1166:00 +CPU= 20.588
  07:34:35 STEP 1167 H=1167:00 +CPU= 33.195
  07:34:55 STEP 1168 H=1168:00 +CPU= 20.172
  07:35:14 STEP 1169 H=1169:00 +CPU= 19.031
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:35:59 STEP 1170 H=1170:00 +CPU= 32.694
  07:36:21 STEP 1171 H=1171:00 +CPU= 21.942
  07:36:42 STEP 1172 H=1172:00 +CPU= 20.436
  07:37:14 STEP 1173 H=1173:00 +CPU= 31.710
  07:37:33 STEP 1174 H=1174:00 +CPU= 18.660
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:37:55 STEP 1175 H=1175:00 +CPU= 20.166
  07:38:34 STEP 1176 H=1176:00 +CPU= 38.373
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGhq0f+001176
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHhq0f+001176
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAhq0f+001176
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMGGhq0f+001152
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMSHhq0f+001152
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMUAhq0f+001152
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMGGhq0f+001164
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMSHhq0f+001164
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMUAhq0f+001164
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0139_2008050100_123_977_12193783_0_r554019119_48.zip
Uploading the intermediate file: upload_file_48.zip
  07:38:53 STEP 1177 H=1177:00 +CPU= 18.932
  07:39:14 STEP 1178 H=1178:00 +CPU= 21.006
  07:39:46 STEP 1179 H=1179:00 +CPU= 31.792
  07:40:06 STEP 1180 H=1180:00 +CPU= 19.526
  07:40:24 STEP 1181 H=1181:00 +CPU= 18.280
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:40:58 STEP 1182 H=1182:00 +CPU= 31.774
  07:41:18 STEP 1183 H=1183:00 +CPU= 19.961
  07:41:37 STEP 1184 H=1184:00 +CPU= 18.795
  07:42:09 STEP 1185 H=1185:00 +CPU= 31.909
  07:42:31 STEP 1186 H=1186:00 +CPU= 21.114
  07:42:50 STEP 1187 H=1187:00 +CPU= 19.621
  07:43:25 STEP 1188 H=1188:00 +CPU= 34.649
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGhq0f+001188
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHhq0f+001188
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAhq0f+001188
  07:43:45 STEP 1189 H=1189:00 +CPU= 19.344
  07:44:07 STEP 1190 H=1190:00 +CPU= 21.743
  07:44:39 STEP 1191 H=1191:00 +CPU= 31.644
  07:45:00 STEP 1192 H=1192:00 +CPU= 21.008
  07:45:17 STEP 1193 H=1193:00 +CPU= 17.596
  07:45:50 STEP 1194 H=1194:00 +CPU= 32.071
  07:46:11 STEP 1195 H=1195:00 +CPU= 21.545
  07:46:32 STEP 1196 H=1196:00 +CPU= 20.947
  07:47:05 STEP 1197 H=1197:00 +CPU= 32.005
  07:47:27 STEP 1198 H=1198:00 +CPU= 21.870
  07:47:49 STEP 1199 H=1199:00 +CPU= 21.572
  07:48:28 STEP 1200 H=1200:00 +CPU= 39.243
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGhq0f+001200
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHhq0f+001200
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAhq0f+001200
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMGGhq0f+001176
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMSHhq0f+001176
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMUAhq0f+001176
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMGGhq0f+001188
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMSHhq0f+001188
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMUAhq0f+001188
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0139_2008050100_123_977_12193783_0_r554019119_49.zip
Uploading the intermediate file: upload_file_49.zip
  07:48:50 STEP 1201 H=1201:00 +CPU= 21.722
Uploading trickle at timestep: 4320000
  07:49:12 STEP 1202 H=1202:00 +CPU= 22.075
  07:49:46 STEP 1203 H=1203:00 +CPU= 33.572
  07:50:09 STEP 1204 H=1204:00 +CPU= 22.185
  07:50:30 STEP 1205 H=1205:00 +CPU= 21.462
  07:51:05 STEP 1206 H=1206:00 +CPU= 34.507
  07:51:27 STEP 1207 H=1207:00 +CPU= 21.810
  07:51:49 STEP 1208 H=1208:00 +CPU= 21.793
  07:52:23 STEP 1209 H=1209:00 +CPU= 33.882
  07:52:46 STEP 1210 H=1210:00 +CPU= 22.333
  07:53:08 STEP 1211 H=1211:00 +CPU= 21.928
  07:53:46 STEP 1212 H=1212:00 +CPU= 37.471
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGhq0f+001212
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHhq0f+001212
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAhq0f+001212
  07:54:08 STEP 1213 H=1213:00 +CPU= 21.843
  07:54:30 STEP 1214 H=1214:00 +CPU= 22.102
  07:55:05 STEP 1215 H=1215:00 +CPU= 34.896
  07:55:27 STEP 1216 H=1216:00 +CPU= 21.435
  07:55:49 STEP 1217 H=1217:00 +CPU= 21.441
  07:56:24 STEP 1218 H=1218:00 +CPU= 35.415
  07:56:45 STEP 1219 H=1219:00 +CPU= 20.739
  07:57:08 STEP 1220 H=1220:00 +CPU= 22.822
  07:57:42 STEP 1221 H=1221:00 +CPU= 33.243
  07:58:04 STEP 1222 H=1222:00 +CPU= 22.180
  07:58:26 STEP 1223 H=1223:00 +CPU= 21.696
  07:59:06 STEP 1224 H=1224:00 +CPU= 39.166
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGhq0f+001224
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHhq0f+001224
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAhq0f+001224
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMGGhq0f+001200
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMSHhq0f+001200
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMUAhq0f+001200
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMGGhq0f+001212
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMSHhq0f+001212
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMUAhq0f+001212
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0139_2008050100_123_977_12193783_0_r554019119_50.zip
Uploading the intermediate file: upload_file_50.zip
  07:59:27 STEP 1225 H=1225:00 +CPU= 21.229
  07:59:49 STEP 1226 H=1226:00 +CPU= 22.017
  08:00:26 STEP 1227 H=1227:00 +CPU= 35.892
  08:00:51 STEP 1228 H=1228:00 +CPU= 24.516
  08:01:15 STEP 1229 H=1229:00 +CPU= 23.202
  08:01:53 STEP 1230 H=1230:00 +CPU= 38.058
  08:02:19 STEP 1231 H=1231:00 +CPU= 24.845
  08:02:43 STEP 1232 H=1232:00 +CPU= 24.449
  08:03:23 STEP 1233 H=1233:00 +CPU= 38.602
  08:03:47 STEP 1234 H=1234:00 +CPU= 24.618
  08:04:12 STEP 1235 H=1235:00 +CPU= 24.384
  08:04:55 STEP 1236 H=1236:00 +CPU= 42.165
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGhq0f+001236
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHhq0f+001236
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAhq0f+001236
  08:05:20 STEP 1237 H=1237:00 +CPU= 24.541
  08:05:45 STEP 1238 H=1238:00 +CPU= 24.513
  08:06:24 STEP 1239 H=1239:00 +CPU= 38.372
  08:06:49 STEP 1240 H=1240:00 +CPU= 25.152
  08:07:14 STEP 1241 H=1241:00 +CPU= 24.360
  08:07:54 STEP 1242 H=1242:00 +CPU= 39.265
  08:08:18 STEP 1243 H=1243:00 +CPU= 24.030
  08:08:42 STEP 1244 H=1244:00 +CPU= 24.130
  08:09:21 STEP 1245 H=1245:00 +CPU= 38.470
  08:09:47 STEP 1246 H=1246:00 +CPU= 24.955
  08:10:13 STEP 1247 H=1247:00 +CPU= 25.706
  08:10:58 STEP 1248 H=1248:00 +CPU= 44.947
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGhq0f+001248
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHhq0f+001248
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAhq0f+001248
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMGGhq0f+001224
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMSHhq0f+001224
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMUAhq0f+001224
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMGGhq0f+001236
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMSHhq0f+001236
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMUAhq0f+001236
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0139_2008050100_123_977_12193783_0_r554019119_51.zip
Uploading the intermediate file: upload_file_51.zip
  08:11:23 STEP 1249 H=1249:00 +CPU= 24.597
  08:11:49 STEP 1250 H=1250:00 +CPU= 24.806
  08:12:27 STEP 1251 H=1251:00 +CPU= 37.782
  08:12:51 STEP 1252 H=1252:00 +CPU= 23.675
  08:13:19 STEP 1253 H=1253:00 +CPU= 25.091
  08:13:59 STEP 1254 H=1254:00 +CPU= 38.311
  08:14:23 STEP 1255 H=1255:00 +CPU= 23.964
  08:14:49 STEP 1256 H=1256:00 +CPU= 25.045
  08:15:28 STEP 1257 H=1257:00 +CPU= 38.506
  08:15:53 STEP 1258 H=1258:00 +CPU= 24.952
  08:16:18 STEP 1259 H=1259:00 +CPU= 24.592
  08:17:02 STEP 1260 H=1260:00 +CPU= 42.890
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGhq0f+001260
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHhq0f+001260
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAhq0f+001260
  08:17:27 STEP 1261 H=1261:00 +CPU= 24.940
  08:17:53 STEP 1262 H=1262:00 +CPU= 25.065
  08:18:32 STEP 1263 H=1263:00 +CPU= 38.866
  08:18:58 STEP 1264 H=1264:00 +CPU= 25.572
  08:19:24 STEP 1265 H=1265:00 +CPU= 24.827
  08:20:04 STEP 1266 H=1266:00 +CPU= 39.435
  08:20:29 STEP 1267 H=1267:00 +CPU= 25.112
  08:20:54 STEP 1268 H=1268:00 +CPU= 24.707
  08:21:34 STEP 1269 H=1269:00 +CPU= 39.082
  08:22:00 STEP 1270 H=1270:00 +CPU= 25.788
  08:22:26 STEP 1271 H=1271:00 +CPU= 25.550
  08:23:11 STEP 1272 H=1272:00 +CPU= 44.776
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGhq0f+001272
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHhq0f+001272
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAhq0f+001272
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMGGhq0f+001248
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMSHhq0f+001248
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMUAhq0f+001248
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMGGhq0f+001260
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMSHhq0f+001260
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMUAhq0f+001260
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0139_2008050100_123_977_12193783_0_r554019119_52.zip
Uploading the intermediate file: upload_file_52.zip
  08:23:36 STEP 1273 H=1273:00 +CPU= 24.756
  08:24:02 STEP 1274 H=1274:00 +CPU= 25.106
  08:24:41 STEP 1275 H=1275:00 +CPU= 38.281
  08:25:06 STEP 1276 H=1276:00 +CPU= 25.092
  08:25:31 STEP 1277 H=1277:00 +CPU= 24.485
  08:26:11 STEP 1278 H=1278:00 +CPU= 39.346
  08:26:35 STEP 1279 H=1279:00 +CPU= 24.407
  08:27:00 STEP 1280 H=1280:00 +CPU= 24.765
  08:27:39 STEP 1281 H=1281:00 +CPU= 38.344
  08:28:05 STEP 1282 H=1282:00 +CPU= 25.290
  08:28:30 STEP 1283 H=1283:00 +CPU= 24.977
  08:29:14 STEP 1284 H=1284:00 +CPU= 42.892
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGhq0f+001284
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHhq0f+001284
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAhq0f+001284
  08:29:39 STEP 1285 H=1285:00 +CPU= 24.485
  08:30:05 STEP 1286 H=1286:00 +CPU= 25.648
  08:30:43 STEP 1287 H=1287:00 +CPU= 38.247
  08:31:09 STEP 1288 H=1288:00 +CPU= 25.606
  08:31:34 STEP 1289 H=1289:00 +CPU= 24.537
  08:32:15 STEP 1290 H=1290:00 +CPU= 40.143
  08:32:39 STEP 1291 H=1291:00 +CPU= 23.931
  08:33:04 STEP 1292 H=1292:00 +CPU= 24.962
  08:33:43 STEP 1293 H=1293:00 +CPU= 38.090
  08:34:08 STEP 1294 H=1294:00 +CPU= 25.228
  08:34:33 STEP 1295 H=1295:00 +CPU= 24.227
  08:35:18 STEP 1296 H=1296:00 +CPU= 44.714
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGhq0f+001296
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHhq0f+001296
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAhq0f+001296
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMGGhq0f+001272
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMSHhq0f+001272
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMUAhq0f+001272
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMGGhq0f+001284
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMSHhq0f+001284
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMUAhq0f+001284
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0139_2008050100_123_977_12193783_0_r554019119_53.zip
Uploading the intermediate file: upload_file_53.zip
  08:35:42 STEP 1297 H=1297:00 +CPU= 23.874
  08:36:08 STEP 1298 H=1298:00 +CPU= 25.326
  08:36:46 STEP 1299 H=1299:00 +CPU= 37.687
  08:37:12 STEP 1300 H=1300:00 +CPU= 25.395
  08:37:37 STEP 1301 H=1301:00 +CPU= 24.294
  08:38:16 STEP 1302 H=1302:00 +CPU= 38.966
  08:38:41 STEP 1303 H=1303:00 +CPU= 24.483
  08:39:07 STEP 1304 H=1304:00 +CPU= 25.803
  08:39:46 STEP 1305 H=1305:00 +CPU= 38.385
  08:40:11 STEP 1306 H=1306:00 +CPU= 25.001
  08:40:36 STEP 1307 H=1307:00 +CPU= 24.227
  08:41:19 STEP 1308 H=1308:00 +CPU= 43.266
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGhq0f+001308
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHhq0f+001308
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAhq0f+001308
  08:41:44 STEP 1309 H=1309:00 +CPU= 24.079
  08:42:09 STEP 1310 H=1310:00 +CPU= 25.337
  08:42:49 STEP 1311 H=1311:00 +CPU= 38.829
  08:43:16 STEP 1312 H=1312:00 +CPU= 26.262
  08:43:40 STEP 1313 H=1313:00 +CPU= 24.397
  08:44:21 STEP 1314 H=1314:00 +CPU= 40.460
  08:44:46 STEP 1315 H=1315:00 +CPU= 24.221
  08:45:12 STEP 1316 H=1316:00 +CPU= 25.528
  08:45:50 STEP 1317 H=1317:00 +CPU= 37.849
  08:46:16 STEP 1318 H=1318:00 +CPU= 25.236
  08:46:41 STEP 1319 H=1319:00 +CPU= 25.038
  08:47:27 STEP 1320 H=1320:00 +CPU= 45.571
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGhq0f+001320
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHhq0f+001320
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAhq0f+001320
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMGGhq0f+001296
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMSHhq0f+001296
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMUAhq0f+001296
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMGGhq0f+001308
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMSHhq0f+001308
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMUAhq0f+001308
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0139_2008050100_123_977_12193783_0_r554019119_54.zip
Uploading the intermediate file: upload_file_54.zip
  08:47:52 STEP 1321 H=1321:00 +CPU= 24.672
  08:48:17 STEP 1322 H=1322:00 +CPU= 25.057
  08:48:56 STEP 1323 H=1323:00 +CPU= 38.467
  08:49:22 STEP 1324 H=1324:00 +CPU= 25.465
  08:49:46 STEP 1325 H=1325:00 +CPU= 23.963
  08:50:26 STEP 1326 H=1326:00 +CPU= 39.434
  08:50:51 STEP 1327 H=1327:00 +CPU= 23.947
  08:51:16 STEP 1328 H=1328:00 +CPU= 25.073
  08:51:55 STEP 1329 H=1329:00 +CPU= 38.312
  08:52:20 STEP 1330 H=1330:00 +CPU= 25.118
  08:52:45 STEP 1331 H=1331:00 +CPU= 24.036
  08:53:28 STEP 1332 H=1332:00 +CPU= 42.395
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGhq0f+001332
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHhq0f+001332
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAhq0f+001332
  08:53:52 STEP 1333 H=1333:00 +CPU= 24.607
  08:54:18 STEP 1334 H=1334:00 +CPU= 24.950
  08:54:56 STEP 1335 H=1335:00 +CPU= 38.217
  08:55:22 STEP 1336 H=1336:00 +CPU= 25.352
  08:55:47 STEP 1337 H=1337:00 +CPU= 24.438
  08:56:28 STEP 1338 H=1338:00 +CPU= 40.138
  08:56:52 STEP 1339 H=1339:00 +CPU= 24.624
  08:57:17 STEP 1340 H=1340:00 +CPU= 24.623
  08:57:56 STEP 1341 H=1341:00 +CPU= 38.354
  08:58:22 STEP 1342 H=1342:00 +CPU= 25.701
  08:58:48 STEP 1343 H=1343:00 +CPU= 24.932
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 2008050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0139
(argv4) batchid: 977
(argv5) wuid: 12193783
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/10
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/10/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/10/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_391a57b390cca2de4159820b5ce1bd1e to: /var/lib/boinc-client/slots/10/oifs_43r3_ps_0139_2008050100_123_977_12193783.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/10/oifs_43r3_ps_0139_2008050100_123_977_12193783.zip
ic_ancil_file: ic_ancil_12193783
ifsdata_file: ifsdata_12193783
climate_data_file: clim_data_12193783
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_df72498217b481e7e63584f34d0c50b0 to: /var/lib/boinc-client/slots/10/ic_ancil_12193783.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/10/ic_ancil_12193783.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_1411d59fc4ad28ff080461be7601e5b5 to: /var/lib/boinc-client/slots/10/ifsdata/ifsdata_12193783.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/10/ifsdata/ifsdata_12193783.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/10/159l_2/clim_data_12193783.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/10/159l_2/clim_data_12193783.zip
Checking for progress XML file: /var/lib/boinc-client/slots/10/progress_file_12193783.xml
Opened progress file ok : /var/lib/boinc-client/slots/10/progress_file_12193783.xml
-- Model is restarting --
Adjusting last_iter, 1344, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/10/progress_file_12193783.xml
last_cpu_time: 37606
upload_file_number: 55
last_iter: 1343
last_upload: 4752000
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0139_2008050100_123_977_12193783_0_r554019119
The child process has been launched with process id: 19524
Executing the command: /var/lib/boinc-client/slots/10/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:LinuxMint:1:1:19524:19524] [20221228:092849:1672216129:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:LinuxMint:1:1:19524:19524] [20221228:092849:1672216129:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:LinuxMint:1:1:19524:19524] [20221228:092849:1672216129:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:LinuxMint:1:1:19524:19524] [20221228:092849:1672216129:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:LinuxMint:1:1:19524:19524] [20221228:092849:1672216129:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:LinuxMint:1:1:19524:19524] [20221228:092849:1672216129:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:LinuxMint:1:1:19524:19524] [20221228:092849:1672216129:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:LinuxMint:1:1:19524:19524] [20221228:092849:1672216129:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:LinuxMint:1:1:19524:19524] [20221228:092849:1672216129:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:LinuxMint:1:1:19524:19524] [20221228:092849:1672216129:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:LinuxMint:1:1:19524:19524] [20221228:092849:1672216129:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:LinuxMint:1:1:19524:19524] [20221228:092849:1672216129:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:LinuxMint:1:1:19524:19524] [20221228:092849:1672216129:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:LinuxMint:1:1:19524:19524] [20221228:092849:1672216129:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:LinuxMint:1:1:19524:19524] [20221228:092849:1672216129:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:LinuxMint:1:1:19524:19524] [20221228:092849:1672216129:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:LinuxMint:1:1:19524:19524] [20221228:092849:1672216129:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:LinuxMint:1:1:19524:19524] [20221228:092849:1672216129:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:LinuxMint:1:1:19524:19524] [20221228:092849:1672216129:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:LinuxMint:1:1:19524:19524] [20221228:092849:1672216129:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:LinuxMint:1:1:19524:19524] [20221228:092849:1672216129:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:LinuxMint:1:1:19524:19524] [20221228:092849:1672216129:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:LinuxMint:1:1:19524:19524] [20221228:092849:1672216129:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:LinuxMint:1:1:19524:19524] [20221228:092849:1672216129:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:LinuxMint:1:1:19524:19524] [20221228:092849:1672216129:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:LinuxMint:1:1:19524:19524] [20221228:092849:1672216129:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:LinuxMint:1:1:19524:19524] [20221228:092849:1672216129:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:LinuxMint:1:1:19524:19524] [20221228:092849:1672216129:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:LinuxMint:1:1:19524:19524] [20221228:092849:1672216129:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:LinuxMint:1:1:19524:19524] [20221228:092849:1672216129:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  09:29:36 STEP 1344 H=1344:00 +CPU= 43.401
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGhq0f+001344
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHhq0f+001344
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAhq0f+001344
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMGGhq0f+001320
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMSHhq0f+001320
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMUAhq0f+001320
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMGGhq0f+001332
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMSHhq0f+001332
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMUAhq0f+001332
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0139_2008050100_123_977_12193783_0_r554019119_55.zip
Uploading the intermediate file: upload_file_55.zip
  09:30:00 STEP 1345 H=1345:00 +CPU= 24.109
  09:30:24 STEP 1346 H=1346:00 +CPU= 23.865
  09:30:59 STEP 1347 H=1347:00 +CPU= 34.633
  09:31:25 STEP 1348 H=1348:00 +CPU= 25.645
  09:31:49 STEP 1349 H=1349:00 +CPU= 23.923
  09:32:30 STEP 1350 H=1350:00 +CPU= 40.131
  09:32:55 STEP 1351 H=1351:00 +CPU= 24.305
  09:33:20 STEP 1352 H=1352:00 +CPU= 25.241
  09:34:00 STEP 1353 H=1353:00 +CPU= 38.825
  09:34:25 STEP 1354 H=1354:00 +CPU= 24.449
  09:34:46 STEP 1355 H=1355:00 +CPU= 21.372
  09:35:26 STEP 1356 H=1356:00 +CPU= 39.398
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGhq0f+001356
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHhq0f+001356
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAhq0f+001356
  09:35:50 STEP 1357 H=1357:00 +CPU= 23.393
  09:36:15 STEP 1358 H=1358:00 +CPU= 24.862
  09:36:53 STEP 1359 H=1359:00 +CPU= 38.001
  09:37:19 STEP 1360 H=1360:00 +CPU= 25.096
  09:37:44 STEP 1361 H=1361:00 +CPU= 24.755
  09:38:23 STEP 1362 H=1362:00 +CPU= 38.840
  09:38:47 STEP 1363 H=1363:00 +CPU= 23.468
  09:39:12 STEP 1364 H=1364:00 +CPU= 24.521
  09:39:50 STEP 1365 H=1365:00 +CPU= 37.668
  09:40:15 STEP 1366 H=1366:00 +CPU= 24.793
  09:40:40 STEP 1367 H=1367:00 +CPU= 25.053
  09:41:25 STEP 1368 H=1368:00 +CPU= 44.526
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGhq0f+001368
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHhq0f+001368
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAhq0f+001368
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMGGhq0f+001344
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMSHhq0f+001344
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMUAhq0f+001344
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMGGhq0f+001356
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMSHhq0f+001356
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMUAhq0f+001356
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0139_2008050100_123_977_12193783_0_r554019119_56.zip
Uploading the intermediate file: upload_file_56.zip
  09:41:50 STEP 1369 H=1369:00 +CPU= 24.147
  09:42:15 STEP 1370 H=1370:00 +CPU= 24.864
  09:42:53 STEP 1371 H=1371:00 +CPU= 38.008
  09:43:19 STEP 1372 H=1372:00 +CPU= 24.941
  09:43:43 STEP 1373 H=1373:00 +CPU= 23.787
  09:44:22 STEP 1374 H=1374:00 +CPU= 38.873
  09:44:46 STEP 1375 H=1375:00 +CPU= 24.176
  09:45:10 STEP 1376 H=1376:00 +CPU= 22.882
  09:45:43 STEP 1377 H=1377:00 +CPU= 32.759
  09:46:07 STEP 1378 H=1378:00 +CPU= 23.660
  09:46:32 STEP 1379 H=1379:00 +CPU= 24.801
  09:47:15 STEP 1380 H=1380:00 +CPU= 42.492
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGhq0f+001380
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHhq0f+001380
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAhq0f+001380
  09:47:39 STEP 1381 H=1381:00 +CPU= 23.896
  09:48:05 STEP 1382 H=1382:00 +CPU= 25.431
  09:48:43 STEP 1383 H=1383:00 +CPU= 38.278
  09:49:09 STEP 1384 H=1384:00 +CPU= 25.444
  09:49:33 STEP 1385 H=1385:00 +CPU= 24.142
  09:50:13 STEP 1386 H=1386:00 +CPU= 38.989
  09:50:37 STEP 1387 H=1387:00 +CPU= 23.703
  09:51:01 STEP 1388 H=1388:00 +CPU= 24.116
  09:51:40 STEP 1389 H=1389:00 +CPU= 38.250
  09:52:05 STEP 1390 H=1390:00 +CPU= 24.874
  09:52:30 STEP 1391 H=1391:00 +CPU= 24.802
  09:53:15 STEP 1392 H=1392:00 +CPU= 44.382
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGhq0f+001392
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHhq0f+001392
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAhq0f+001392
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMGGhq0f+001368
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMSHhq0f+001368
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMUAhq0f+001368
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMGGhq0f+001380
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMSHhq0f+001380
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMUAhq0f+001380
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0139_2008050100_123_977_12193783_0_r554019119_57.zip
Uploading the intermediate file: upload_file_57.zip
  09:53:39 STEP 1393 H=1393:00 +CPU= 24.136
  09:54:05 STEP 1394 H=1394:00 +CPU= 25.069
  09:54:42 STEP 1395 H=1395:00 +CPU= 37.211
  09:55:08 STEP 1396 H=1396:00 +CPU= 25.056
  09:55:32 STEP 1397 H=1397:00 +CPU= 23.756
  09:56:11 STEP 1398 H=1398:00 +CPU= 38.989
  09:56:36 STEP 1399 H=1399:00 +CPU= 24.211
  09:57:01 STEP 1400 H=1400:00 +CPU= 24.985
  09:57:39 STEP 1401 H=1401:00 +CPU= 37.397
  09:58:04 STEP 1402 H=1402:00 +CPU= 25.217
  09:58:29 STEP 1403 H=1403:00 +CPU= 24.131
  09:59:11 STEP 1404 H=1404:00 +CPU= 41.359
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGhq0f+001404
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHhq0f+001404
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAhq0f+001404
  09:59:35 STEP 1405 H=1405:00 +CPU= 24.500
  10:00:00 STEP 1406 H=1406:00 +CPU= 24.439
  10:00:39 STEP 1407 H=1407:00 +CPU= 38.091
  10:01:04 STEP 1408 H=1408:00 +CPU= 25.194
  10:01:28 STEP 1409 H=1409:00 +CPU= 24.199
  10:02:08 STEP 1410 H=1410:00 +CPU= 38.821
  10:02:32 STEP 1411 H=1411:00 +CPU= 24.229
  10:02:57 STEP 1412 H=1412:00 +CPU= 24.776
  10:03:36 STEP 1413 H=1413:00 +CPU= 38.527
  10:04:02 STEP 1414 H=1414:00 +CPU= 25.250
  10:04:27 STEP 1415 H=1415:00 +CPU= 24.579
  10:05:12 STEP 1416 H=1416:00 +CPU= 45.086
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGhq0f+001416
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHhq0f+001416
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAhq0f+001416
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMGGhq0f+001392
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMSHhq0f+001392
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMUAhq0f+001392
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMGGhq0f+001404
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMSHhq0f+001404
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMUAhq0f+001404
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0139_2008050100_123_977_12193783_0_r554019119_58.zip
Uploading the intermediate file: upload_file_58.zip
  10:05:37 STEP 1417 H=1417:00 +CPU= 24.482
  10:06:02 STEP 1418 H=1418:00 +CPU= 25.069
  10:06:41 STEP 1419 H=1419:00 +CPU= 37.906
  10:07:06 STEP 1420 H=1420:00 +CPU= 24.887
  10:07:30 STEP 1421 H=1421:00 +CPU= 23.836
  10:08:10 STEP 1422 H=1422:00 +CPU= 39.389
  10:08:34 STEP 1423 H=1423:00 +CPU= 23.907
  10:08:59 STEP 1424 H=1424:00 +CPU= 24.550
  10:09:38 STEP 1425 H=1425:00 +CPU= 38.913
  10:10:04 STEP 1426 H=1426:00 +CPU= 25.304
  10:10:28 STEP 1427 H=1427:00 +CPU= 23.910
  10:11:11 STEP 1428 H=1428:00 +CPU= 42.709
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGhq0f+001428
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHhq0f+001428
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAhq0f+001428
  10:11:36 STEP 1429 H=1429:00 +CPU= 24.300
  10:12:01 STEP 1430 H=1430:00 +CPU= 25.386
  10:12:39 STEP 1431 H=1431:00 +CPU= 37.573
  10:13:05 STEP 1432 H=1432:00 +CPU= 25.175
  10:13:29 STEP 1433 H=1433:00 +CPU= 23.622
  10:14:08 STEP 1434 H=1434:00 +CPU= 38.649
  10:14:32 STEP 1435 H=1435:00 +CPU= 24.086
  10:14:57 STEP 1436 H=1436:00 +CPU= 24.793
  10:15:35 STEP 1437 H=1437:00 +CPU= 37.582
  10:16:00 STEP 1438 H=1438:00 +CPU= 24.981
  10:16:26 STEP 1439 H=1439:00 +CPU= 25.316
  10:17:12 STEP 1440 H=1440:00 +CPU= 45.310
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGhq0f+001440
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHhq0f+001440
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAhq0f+001440
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMGGhq0f+001416
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMSHhq0f+001416
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMUAhq0f+001416
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMGGhq0f+001428
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMSHhq0f+001428
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMUAhq0f+001428
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0139_2008050100_123_977_12193783_0_r554019119_59.zip
Uploading the intermediate file: upload_file_59.zip
  10:17:37 STEP 1441 H=1441:00 +CPU= 24.461
  10:18:01 STEP 1442 H=1442:00 +CPU= 24.078
  10:18:40 STEP 1443 H=1443:00 +CPU= 38.111
  10:19:02 STEP 1444 H=1444:00 +CPU= 22.396
  10:19:24 STEP 1445 H=1445:00 +CPU= 21.157
  10:19:59 STEP 1446 H=1446:00 +CPU= 34.910
  10:20:24 STEP 1447 H=1447:00 +CPU= 25.065
  10:20:50 STEP 1448 H=1448:00 +CPU= 25.240
  10:21:29 STEP 1449 H=1449:00 +CPU= 39.079
  10:21:54 STEP 1450 H=1450:00 +CPU= 24.947
  10:22:19 STEP 1451 H=1451:00 +CPU= 23.854
  10:23:01 STEP 1452 H=1452:00 +CPU= 42.046
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGhq0f+001452
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHhq0f+001452
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAhq0f+001452
  10:23:26 STEP 1453 H=1453:00 +CPU= 24.753
  10:23:51 STEP 1454 H=1454:00 +CPU= 24.508
  10:24:30 STEP 1455 H=1455:00 +CPU= 38.775
  10:24:55 STEP 1456 H=1456:00 +CPU= 24.635
  10:25:19 STEP 1457 H=1457:00 +CPU= 23.519
  10:25:57 STEP 1458 H=1458:00 +CPU= 38.390
  10:26:22 STEP 1459 H=1459:00 +CPU= 23.878
  10:26:47 STEP 1460 H=1460:00 +CPU= 25.052
  10:27:26 STEP 1461 H=1461:00 +CPU= 38.439
  10:27:51 STEP 1462 H=1462:00 +CPU= 24.719
  10:28:16 STEP 1463 H=1463:00 +CPU= 25.183
  10:29:01 STEP 1464 H=1464:00 +CPU= 44.439
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGhq0f+001464
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHhq0f+001464
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAhq0f+001464
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMGGhq0f+001440
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMSHhq0f+001440
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMUAhq0f+001440
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMGGhq0f+001452
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMSHhq0f+001452
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMUAhq0f+001452
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0139_2008050100_123_977_12193783_0_r554019119_60.zip
Uploading the intermediate file: upload_file_60.zip
  10:29:26 STEP 1465 H=1465:00 +CPU= 24.782
  10:29:52 STEP 1466 H=1466:00 +CPU= 25.689
Quit request received from BOINC client, ending the child process
..Failed, model did not complete successfully
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 2008050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0139
(argv4) batchid: 977
(argv5) wuid: 12193783
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/10
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/10/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/10/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_391a57b390cca2de4159820b5ce1bd1e to: /var/lib/boinc-client/slots/10/oifs_43r3_ps_0139_2008050100_123_977_12193783.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/10/oifs_43r3_ps_0139_2008050100_123_977_12193783.zip
ic_ancil_file: ic_ancil_12193783
ifsdata_file: ifsdata_12193783
climate_data_file: clim_data_12193783
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_df72498217b481e7e63584f34d0c50b0 to: /var/lib/boinc-client/slots/10/ic_ancil_12193783.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/10/ic_ancil_12193783.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_1411d59fc4ad28ff080461be7601e5b5 to: /var/lib/boinc-client/slots/10/ifsdata/ifsdata_12193783.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/10/ifsdata/ifsdata_12193783.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/10/159l_2/clim_data_12193783.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/10/159l_2/clim_data_12193783.zip
Checking for progress XML file: /var/lib/boinc-client/slots/10/progress_file_12193783.xml
Opened progress file ok : /var/lib/boinc-client/slots/10/progress_file_12193783.xml
-- Model is restarting --
Adjusting last_iter, 1466, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/10/progress_file_12193783.xml
last_cpu_time: 41226
upload_file_number: 61
last_iter: 1463
last_upload: 5270400
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0139_2008050100_123_977_12193783_0_r554019119
The child process has been launched with process id: 19835
Executing the command: /var/lib/boinc-client/slots/10/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:LinuxMint:1:1:19835:19835] [20221228:103101:1672219861:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:LinuxMint:1:1:19835:19835] [20221228:103101:1672219861:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:LinuxMint:1:1:19835:19835] [20221228:103101:1672219861:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:LinuxMint:1:1:19835:19835] [20221228:103101:1672219861:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:LinuxMint:1:1:19835:19835] [20221228:103101:1672219861:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:LinuxMint:1:1:19835:19835] [20221228:103101:1672219861:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:LinuxMint:1:1:19835:19835] [20221228:103101:1672219861:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:LinuxMint:1:1:19835:19835] [20221228:103101:1672219861:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:LinuxMint:1:1:19835:19835] [20221228:103101:1672219861:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:LinuxMint:1:1:19835:19835] [20221228:103101:1672219861:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:LinuxMint:1:1:19835:19835] [20221228:103101:1672219861:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:LinuxMint:1:1:19835:19835] [20221228:103101:1672219861:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:LinuxMint:1:1:19835:19835] [20221228:103101:1672219861:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:LinuxMint:1:1:19835:19835] [20221228:103101:1672219861:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:LinuxMint:1:1:19835:19835] [20221228:103101:1672219861:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:LinuxMint:1:1:19835:19835] [20221228:103101:1672219861:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:LinuxMint:1:1:19835:19835] [20221228:103101:1672219861:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:LinuxMint:1:1:19835:19835] [20221228:103101:1672219861:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:LinuxMint:1:1:19835:19835] [20221228:103101:1672219861:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:LinuxMint:1:1:19835:19835] [20221228:103101:1672219861:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:LinuxMint:1:1:19835:19835] [20221228:103101:1672219861:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:LinuxMint:1:1:19835:19835] [20221228:103101:1672219861:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:LinuxMint:1:1:19835:19835] [20221228:103101:1672219861:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:LinuxMint:1:1:19835:19835] [20221228:103101:1672219861:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:LinuxMint:1:1:19835:19835] [20221228:103101:1672219861:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:LinuxMint:1:1:19835:19835] [20221228:103101:1672219861:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:LinuxMint:1:1:19835:19835] [20221228:103101:1672219861:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:LinuxMint:1:1:19835:19835] [20221228:103101:1672219861:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:LinuxMint:1:1:19835:19835] [20221228:103101:1672219861:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:LinuxMint:1:1:19835:19835] [20221228:103101:1672219861:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  10:31:48 STEP 1464 H=1464:00 +CPU= 42.944
  10:32:11 STEP 1465 H=1465:00 +CPU= 23.330
  10:32:35 STEP 1466 H=1466:00 +CPU= 23.016
  10:33:10 STEP 1467 H=1467:00 +CPU= 35.340
  10:33:35 STEP 1468 H=1468:00 +CPU= 24.742
  10:33:59 STEP 1469 H=1469:00 +CPU= 23.751
  10:34:39 STEP 1470 H=1470:00 +CPU= 38.732
  10:35:03 STEP 1471 H=1471:00 +CPU= 23.975
  10:35:28 STEP 1472 H=1472:00 +CPU= 24.588
  10:36:06 STEP 1473 H=1473:00 +CPU= 37.765
  10:36:31 STEP 1474 H=1474:00 +CPU= 25.414
  10:36:56 STEP 1475 H=1475:00 +CPU= 24.003
  10:37:38 STEP 1476 H=1476:00 +CPU= 42.251
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGhq0f+001476
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHhq0f+001476
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAhq0f+001476
  10:38:03 STEP 1477 H=1477:00 +CPU= 24.319
  10:38:29 STEP 1478 H=1478:00 +CPU= 25.209
  10:39:07 STEP 1479 H=1479:00 +CPU= 38.114
  10:39:32 STEP 1480 H=1480:00 +CPU= 24.947
  10:39:56 STEP 1481 H=1481:00 +CPU= 23.393
  10:40:36 STEP 1482 H=1482:00 +CPU= 39.628
  10:41:00 STEP 1483 H=1483:00 +CPU= 23.972
  10:41:25 STEP 1484 H=1484:00 +CPU= 24.824
  10:42:04 STEP 1485 H=1485:00 +CPU= 38.440
  10:42:30 STEP 1486 H=1486:00 +CPU= 25.076
  10:42:55 STEP 1487 H=1487:00 +CPU= 25.406
  10:43:42 STEP 1488 H=1488:00 +CPU= 46.192
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGhq0f+001488
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHhq0f+001488
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAhq0f+001488
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMGGhq0f+001464
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMSHhq0f+001464
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMUAhq0f+001464
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMGGhq0f+001476
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMSHhq0f+001476
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMUAhq0f+001476
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0139_2008050100_123_977_12193783_0_r554019119_61.zip
Uploading the intermediate file: upload_file_61.zip
  10:44:07 STEP 1489 H=1489:00 +CPU= 24.570
  10:44:32 STEP 1490 H=1490:00 +CPU= 24.656
  10:45:10 STEP 1491 H=1491:00 +CPU= 37.584
  10:45:35 STEP 1492 H=1492:00 +CPU= 24.487
  10:45:59 STEP 1493 H=1493:00 +CPU= 24.378
  10:46:39 STEP 1494 H=1494:00 +CPU= 38.937
  10:47:04 STEP 1495 H=1495:00 +CPU= 24.987
  10:47:30 STEP 1496 H=1496:00 +CPU= 25.177
  10:48:08 STEP 1497 H=1497:00 +CPU= 37.932
  10:48:33 STEP 1498 H=1498:00 +CPU= 24.467
  10:48:57 STEP 1499 H=1499:00 +CPU= 24.107
  10:49:40 STEP 1500 H=1500:00 +CPU= 42.156
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGhq0f+001500
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHhq0f+001500
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAhq0f+001500
  10:50:05 STEP 1501 H=1501:00 +CPU= 24.739
  10:50:31 STEP 1502 H=1502:00 +CPU= 25.303
  10:51:09 STEP 1503 H=1503:00 +CPU= 37.558
  10:51:34 STEP 1504 H=1504:00 +CPU= 25.102
  10:51:58 STEP 1505 H=1505:00 +CPU= 23.901
  10:52:37 STEP 1506 H=1506:00 +CPU= 38.777
  10:53:02 STEP 1507 H=1507:00 +CPU= 24.116
  10:53:28 STEP 1508 H=1508:00 +CPU= 25.351
  10:54:06 STEP 1509 H=1509:00 +CPU= 37.846
  10:54:30 STEP 1510 H=1510:00 +CPU= 24.102
  10:54:56 STEP 1511 H=1511:00 +CPU= 25.340
  10:55:41 STEP 1512 H=1512:00 +CPU= 44.323
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGhq0f+001512
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHhq0f+001512
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAhq0f+001512
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMGGhq0f+001488
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMSHhq0f+001488
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMUAhq0f+001488
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMGGhq0f+001500
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMSHhq0f+001500
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMUAhq0f+001500
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0139_2008050100_123_977_12193783_0_r554019119_62.zip
Uploading the intermediate file: upload_file_62.zip
  10:56:06 STEP 1513 H=1513:00 +CPU= 24.549
  10:56:31 STEP 1514 H=1514:00 +CPU= 25.082
  10:57:10 STEP 1515 H=1515:00 +CPU= 38.595
  10:57:35 STEP 1516 H=1516:00 +CPU= 24.324
  10:57:59 STEP 1517 H=1517:00 +CPU= 24.089
  10:58:38 STEP 1518 H=1518:00 +CPU= 38.869
  10:59:04 STEP 1519 H=1519:00 +CPU= 25.328
  10:59:29 STEP 1520 H=1520:00 +CPU= 24.839
  11:00:08 STEP 1521 H=1521:00 +CPU= 38.449
  11:00:33 STEP 1522 H=1522:00 +CPU= 24.741
  11:00:58 STEP 1523 H=1523:00 +CPU= 24.417
  11:01:41 STEP 1524 H=1524:00 +CPU= 42.367
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGhq0f+001524
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHhq0f+001524
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAhq0f+001524
  11:02:06 STEP 1525 H=1525:00 +CPU= 24.744
  11:02:30 STEP 1526 H=1526:00 +CPU= 24.310
  11:03:09 STEP 1527 H=1527:00 +CPU= 38.443
  11:03:35 STEP 1528 H=1528:00 +CPU= 24.993
  11:03:58 STEP 1529 H=1529:00 +CPU= 23.505
  11:04:37 STEP 1530 H=1530:00 +CPU= 38.425
  11:05:01 STEP 1531 H=1531:00 +CPU= 23.531
  11:05:26 STEP 1532 H=1532:00 +CPU= 24.983
  11:06:05 STEP 1533 H=1533:00 +CPU= 38.047
  11:06:30 STEP 1534 H=1534:00 +CPU= 25.193
  11:06:55 STEP 1535 H=1535:00 +CPU= 24.554
  11:07:40 STEP 1536 H=1536:00 +CPU= 44.415
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGhq0f+001536
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHhq0f+001536
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAhq0f+001536
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMGGhq0f+001512
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMSHhq0f+001512
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMUAhq0f+001512
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMGGhq0f+001524
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMSHhq0f+001524
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMUAhq0f+001524
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0139_2008050100_123_977_12193783_0_r554019119_63.zip
Uploading the intermediate file: upload_file_63.zip
  11:08:05 STEP 1537 H=1537:00 +CPU= 24.398
  11:08:30 STEP 1538 H=1538:00 +CPU= 24.539
  11:09:08 STEP 1539 H=1539:00 +CPU= 38.222
  11:09:34 STEP 1540 H=1540:00 +CPU= 25.592
  11:09:59 STEP 1541 H=1541:00 +CPU= 24.353
  11:10:38 STEP 1542 H=1542:00 +CPU= 38.488
  11:11:02 STEP 1543 H=1543:00 +CPU= 23.833
  11:11:27 STEP 1544 H=1544:00 +CPU= 25.189
  11:12:06 STEP 1545 H=1545:00 +CPU= 38.477
  11:12:31 STEP 1546 H=1546:00 +CPU= 24.540
  11:12:56 STEP 1547 H=1547:00 +CPU= 24.475
  11:13:38 STEP 1548 H=1548:00 +CPU= 41.926
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGhq0f+001548
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHhq0f+001548
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAhq0f+001548
  11:14:02 STEP 1549 H=1549:00 +CPU= 23.502
  11:14:27 STEP 1550 H=1550:00 +CPU= 25.211
  11:15:03 STEP 1551 H=1551:00 +CPU= 35.208
  11:15:26 STEP 1552 H=1552:00 +CPU= 22.391
  11:15:45 STEP 1553 H=1553:00 +CPU= 19.635
  11:16:26 STEP 1554 H=1554:00 +CPU= 39.955
  11:16:50 STEP 1555 H=1555:00 +CPU= 23.861
  11:17:12 STEP 1556 H=1556:00 +CPU= 21.779
  11:17:46 STEP 1557 H=1557:00 +CPU= 33.258
  11:18:12 STEP 1558 H=1558:00 +CPU= 25.890
  11:18:36 STEP 1559 H=1559:00 +CPU= 24.365
  11:19:19 STEP 1560 H=1560:00 +CPU= 41.245
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGhq0f+001560
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHhq0f+001560
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAhq0f+001560
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMGGhq0f+001536
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMSHhq0f+001536
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMUAhq0f+001536
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMGGhq0f+001548
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMSHhq0f+001548
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMUAhq0f+001548
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0139_2008050100_123_977_12193783_0_r554019119_64.zip
Uploading the intermediate file: upload_file_64.zip
  11:19:39 STEP 1561 H=1561:00 +CPU= 20.341
  11:20:02 STEP 1562 H=1562:00 +CPU= 22.182
  11:20:38 STEP 1563 H=1563:00 +CPU= 36.071
  11:20:59 STEP 1564 H=1564:00 +CPU= 20.999
  11:21:22 STEP 1565 H=1565:00 +CPU= 22.263
  11:21:57 STEP 1566 H=1566:00 +CPU= 35.069
  11:22:20 STEP 1567 H=1567:00 +CPU= 22.332
  11:22:40 STEP 1568 H=1568:00 +CPU= 19.888
  11:23:09 STEP 1569 H=1569:00 +CPU= 29.512
  11:23:30 STEP 1570 H=1570:00 +CPU= 20.558
  11:23:52 STEP 1571 H=1571:00 +CPU= 21.424
  11:24:30 STEP 1572 H=1572:00 +CPU= 37.548
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGhq0f+001572
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHhq0f+001572
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAhq0f+001572
  11:24:51 STEP 1573 H=1573:00 +CPU= 21.487
  11:25:14 STEP 1574 H=1574:00 +CPU= 22.193
  11:25:48 STEP 1575 H=1575:00 +CPU= 33.671
  11:26:11 STEP 1576 H=1576:00 +CPU= 23.100
  11:26:33 STEP 1577 H=1577:00 +CPU= 22.186
  11:27:08 STEP 1578 H=1578:00 +CPU= 34.182
  11:27:30 STEP 1579 H=1579:00 +CPU= 21.829
  11:27:53 STEP 1580 H=1580:00 +CPU= 22.540
  11:28:27 STEP 1581 H=1581:00 +CPU= 34.135
  11:28:49 STEP 1582 H=1582:00 +CPU= 21.636
  11:29:12 STEP 1583 H=1583:00 +CPU= 23.421
  11:29:53 STEP 1584 H=1584:00 +CPU= 39.956
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGhq0f+001584
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHhq0f+001584
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAhq0f+001584
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMGGhq0f+001560
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMSHhq0f+001560
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMUAhq0f+001560
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMGGhq0f+001572
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMSHhq0f+001572
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783/ICMUAhq0f+001572
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0139_2008050100_123_977_12193783_0_r554019119_65.zip
Uploading the intermediate file: upload_file_65.zip
  11:30:14 STEP 1585 H=1585:00 +CPU= 21.138
  11:30:36 STEP 1586 H=1586:00 +CPU= 21.902
  11:31:11 STEP 1587 H=1587:00 +CPU= 35.029
  11:31:34 STEP 1588 H=1588:00 +CPU= 21.880
  11:31:55 STEP 1589 H=1589:00 +CPU= 21.020
Quit request received from BOINC client, ending the child process
..Failed, model did not complete successfully
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 2008050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0139
(argv4) batchid: 977
(argv5) wuid: 12193783
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/10
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193783
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/10/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/10/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_391a57b390cca2de4159820b5ce1bd1e to: /var/lib/boinc-client/slots/10/oifs_43r3_ps_0139_2008050100_123_977_12193783.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/10/oifs_43r3_ps_0139_2008050100_123_977_12193783.zip
ic_ancil_file: ic_ancil_12193783
ifsdata_file: ifsdata_12193783
climate_data_file: clim_data_12193783
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_df72498217b481e7e63584f34d0c50b0 to: /var/lib/boinc-client/slots/10/ic_ancil_12193783.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/10/ic_ancil_12193783.zip
/var/lib/boinc-client/slots/10/ICMGGhq0fINIT  bad CRC 439d1c31  (should be 9ccecd67)
..Unzipping the IC ancils file failed

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 May 2023 15:59:44 1523867 22279593 oifs_43r3_ps_0139_2008050100_123_977_12193783_0 4,320,000 33,420 0.0077
09 May 2023 15:59:39 1523867 22279593 oifs_43r3_ps_0139_2008050100_123_977_12193783_0 3,456,000 27,599 0.0080
09 May 2023 15:59:31 1523867 22279593 oifs_43r3_ps_0139_2008050100_123_977_12193783_0 2,592,000 20,575 0.0079
09 May 2023 15:59:24 1523867 22279593 oifs_43r3_ps_0139_2008050100_123_977_12193783_0 1,728,000 13,477 0.0078
09 May 2023 15:59:18 1523867 22279593 oifs_43r3_ps_0139_2008050100_123_977_12193783_0 864,000 6,343 0.0073


©2024 cpdn.org