climateprediction.net home page
Task 22446368

Task 22446368

Name oifs_43r3_bl_a3f4_2016092300_20_1018_12292759_0
Workunit 12292759
Created 13 Jun 2024, 2:26:24 UTC
Sent 13 Jun 2024, 16:18:12 UTC
Report deadline 12 Aug 2024, 16:18:12 UTC
Received 14 Jun 2024, 11:28:42 UTC
Server state Over
Outcome Success
Client state Done
Exit status 0 (0x00000000)
Computer ID 1549780
Run time 11 hours 28 min 32 sec
CPU time 11 hours 22 min 58 sec
Validate state Valid
Credit 2,635.55
Device peak FLOPS 5.04 GFLOPS
Application version OpenIFS 43r3 Baroclinic Lifecycle v1.13
x86_64-pc-linux-gnu
Peak working set size 5,000.73 MB
Peak swap size 5,981.13 MB
Peak disk usage 1,751.26 MB
Stderr
<core_client_version>7.22.1</core_client_version>
<![CDATA[
<stderr_txt>
.zip
  09:19:02 STEP 1249 H= 312:15 +CPU= 23.109
  09:19:24 STEP 1250 H= 312:30 +CPU= 21.716
  09:19:46 STEP 1251 H= 312:45 +CPU= 22.259
  09:20:09 STEP 1252 H= 313:00 +CPU= 22.287
  09:20:29 STEP 1253 H= 313:15 +CPU= 20.727
  09:20:53 STEP 1254 H= 313:30 +CPU= 22.981
  09:21:14 STEP 1255 H= 313:45 +CPU= 21.015
  09:21:34 STEP 1256 H= 314:00 +CPU= 20.715
  09:21:57 STEP 1257 H= 314:15 +CPU= 22.440
  09:22:18 STEP 1258 H= 314:30 +CPU= 21.330
  09:22:40 STEP 1259 H= 314:45 +CPU= 21.418
  09:23:08 STEP 1260 H= 315:00 +CPU= 27.712
Moving to projects directory: /home/ij2/slots/15/ICMGGh7zg+001260
Moving to projects directory: /home/ij2/slots/15/ICMSHh7zg+001260
Moving to projects directory: /home/ij2/slots/15/ICMUAh7zg+001260
  09:23:30 STEP 1261 H= 315:15 +CPU= 22.415
  09:23:52 STEP 1262 H= 315:30 +CPU= 21.867
  09:24:14 STEP 1263 H= 315:45 +CPU= 22.432
  09:24:37 STEP 1264 H= 316:00 +CPU= 22.971
  09:24:59 STEP 1265 H= 316:15 +CPU= 21.130
  09:25:20 STEP 1266 H= 316:30 +CPU= 20.983
  09:25:41 STEP 1267 H= 316:45 +CPU= 21.532
  09:26:03 STEP 1268 H= 317:00 +CPU= 22.130
  09:26:25 STEP 1269 H= 317:15 +CPU= 21.854
  09:26:47 STEP 1270 H= 317:30 +CPU= 21.706
  09:27:09 STEP 1271 H= 317:45 +CPU= 22.229
  09:27:38 STEP 1272 H= 318:00 +CPU= 28.136
Moving to projects directory: /home/ij2/slots/15/ICMGGh7zg+001272
Moving to projects directory: /home/ij2/slots/15/ICMSHh7zg+001272
Moving to projects directory: /home/ij2/slots/15/ICMUAh7zg+001272
  09:27:59 STEP 1273 H= 318:15 +CPU= 21.609
  09:28:21 STEP 1274 H= 318:30 +CPU= 21.929
  09:28:43 STEP 1275 H= 318:45 +CPU= 22.147
  09:29:05 STEP 1276 H= 319:00 +CPU= 21.217
  09:29:28 STEP 1277 H= 319:15 +CPU= 22.744
  09:29:48 STEP 1278 H= 319:30 +CPU= 20.913
  09:30:10 STEP 1279 H= 319:45 +CPU= 21.930
  09:30:32 STEP 1280 H= 320:00 +CPU= 21.178
  09:30:52 STEP 1281 H= 320:15 +CPU= 20.379
  09:31:17 STEP 1282 H= 320:30 +CPU= 23.188
  09:31:39 STEP 1283 H= 320:45 +CPU= 22.035
  09:32:07 STEP 1284 H= 321:00 +CPU= 27.662
Moving to projects directory: /home/ij2/slots/15/ICMGGh7zg+001284
Moving to projects directory: /home/ij2/slots/15/ICMSHh7zg+001284
Moving to projects directory: /home/ij2/slots/15/ICMUAh7zg+001284
  09:32:26 STEP 1285 H= 321:15 +CPU= 19.350
  09:32:47 STEP 1286 H= 321:30 +CPU= 20.525
  09:33:07 STEP 1287 H= 321:45 +CPU= 20.814
  09:33:31 STEP 1288 H= 322:00 +CPU= 23.118
  09:33:52 STEP 1289 H= 322:15 +CPU= 21.577
  09:34:14 STEP 1290 H= 322:30 +CPU= 21.262
  09:34:35 STEP 1291 H= 322:45 +CPU= 21.413
  09:34:58 STEP 1292 H= 323:00 +CPU= 22.734
  09:35:20 STEP 1293 H= 323:15 +CPU= 22.334
  09:35:42 STEP 1294 H= 323:30 +CPU= 21.779
  09:36:04 STEP 1295 H= 323:45 +CPU= 21.724
  09:36:34 STEP 1296 H= 324:00 +CPU= 29.896
Moving to projects directory: /home/ij2/slots/15/ICMGGh7zg+001296
Moving to projects directory: /home/ij2/slots/15/ICMSHh7zg+001296
Moving to projects directory: /home/ij2/slots/15/ICMUAh7zg+001296
  09:36:55 STEP 1297 H= 324:15 +CPU= 20.617
  09:37:17 STEP 1298 H= 324:30 +CPU= 21.449
  09:37:37 STEP 1299 H= 324:45 +CPU= 20.536
  09:37:58 STEP 1300 H= 325:00 +CPU= 21.060
  09:38:20 STEP 1301 H= 325:15 +CPU= 21.193
  09:38:42 STEP 1302 H= 325:30 +CPU= 22.158
  09:39:05 STEP 1303 H= 325:45 +CPU= 22.741
  09:39:26 STEP 1304 H= 326:00 +CPU= 20.847
  09:39:46 STEP 1305 H= 326:15 +CPU= 20.698
  09:40:07 STEP 1306 H= 326:30 +CPU= 20.438
  09:40:29 STEP 1307 H= 326:45 +CPU= 21.956
  09:40:56 STEP 1308 H= 327:00 +CPU= 27.500
Moving to projects directory: /home/ij2/slots/15/ICMGGh7zg+001308
Moving to projects directory: /home/ij2/slots/15/ICMSHh7zg+001308
Moving to projects directory: /home/ij2/slots/15/ICMUAh7zg+001308
  09:41:18 STEP 1309 H= 327:15 +CPU= 21.464
  09:41:40 STEP 1310 H= 327:30 +CPU= 21.818
  09:42:01 STEP 1311 H= 327:45 +CPU= 21.135
  09:42:24 STEP 1312 H= 328:00 +CPU= 22.553
  09:42:44 STEP 1313 H= 328:15 +CPU= 20.178
  09:43:05 STEP 1314 H= 328:30 +CPU= 21.711
  09:43:26 STEP 1315 H= 328:45 +CPU= 20.429
  09:43:48 STEP 1316 H= 329:00 +CPU= 22.138
  09:44:10 STEP 1317 H= 329:15 +CPU= 21.352
  09:44:30 STEP 1318 H= 329:30 +CPU= 20.411
  09:44:53 STEP 1319 H= 329:45 +CPU= 22.618
  09:45:20 STEP 1320 H= 330:00 +CPU= 27.546
Moving to projects directory: /home/ij2/slots/15/ICMGGh7zg+001320
Moving to projects directory: /home/ij2/slots/15/ICMSHh7zg+001320
Moving to projects directory: /home/ij2/slots/15/ICMUAh7zg+001320
  09:45:42 STEP 1321 H= 330:15 +CPU= 21.472
  09:46:04 STEP 1322 H= 330:30 +CPU= 22.426
  09:46:25 STEP 1323 H= 330:45 +CPU= 20.548
  09:46:47 STEP 1324 H= 331:00 +CPU= 21.743
  09:47:08 STEP 1325 H= 331:15 +CPU= 21.475
  09:47:29 STEP 1326 H= 331:30 +CPU= 20.563
  09:47:50 STEP 1327 H= 331:45 +CPU= 20.851
  09:48:12 STEP 1328 H= 332:00 +CPU= 22.436
  09:48:34 STEP 1329 H= 332:15 +CPU= 21.650
  09:48:54 STEP 1330 H= 332:30 +CPU= 20.485
  09:49:15 STEP 1331 H= 332:45 +CPU= 20.492
  09:49:42 STEP 1332 H= 333:00 +CPU= 27.548
Moving to projects directory: /home/ij2/slots/15/ICMGGh7zg+001332
Moving to projects directory: /home/ij2/slots/15/ICMSHh7zg+001332
Moving to projects directory: /home/ij2/slots/15/ICMUAh7zg+001332
  09:50:05 STEP 1333 H= 333:15 +CPU= 22.313
  09:50:25 STEP 1334 H= 333:30 +CPU= 20.287
  09:50:47 STEP 1335 H= 333:45 +CPU= 21.568
  09:51:09 STEP 1336 H= 334:00 +CPU= 22.712
  09:51:30 STEP 1337 H= 334:15 +CPU= 20.444
  09:51:52 STEP 1338 H= 334:30 +CPU= 21.632
  09:52:12 STEP 1339 H= 334:45 +CPU= 20.853
  09:52:33 STEP 1340 H= 335:00 +CPU= 20.958
  09:52:55 STEP 1341 H= 335:15 +CPU= 21.859
  09:53:16 STEP 1342 H= 335:30 +CPU= 20.748
  09:53:39 STEP 1343 H= 335:45 +CPU= 22.810
  09:54:10 STEP 1344 H= 336:00 +CPU= 30.676
Moving to projects directory: /home/ij2/slots/15/ICMGGh7zg+001344
Moving to projects directory: /home/ij2/slots/15/ICMSHh7zg+001344
Moving to projects directory: /home/ij2/slots/15/ICMUAh7zg+001344
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMGGh7zg+001248
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMSHh7zg+001248
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMUAh7zg+001248
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMGGh7zg+001260
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMSHh7zg+001260
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMUAh7zg+001260
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMGGh7zg+001272
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMSHh7zg+001272
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMUAh7zg+001272
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMGGh7zg+001284
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMSHh7zg+001284
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMUAh7zg+001284
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMGGh7zg+001296
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMSHh7zg+001296
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMUAh7zg+001296
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMGGh7zg+001308
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMSHh7zg+001308
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMUAh7zg+001308
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMGGh7zg+001320
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMSHh7zg+001320
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMUAh7zg+001320
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMGGh7zg+001332
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMSHh7zg+001332
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMUAh7zg+001332
Zipping up the intermediate file: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_a3f4_2016092300_20_1018_12292759_0_r1107509013_13.zip
Uploading the intermediate file: upload_file_13.zip
  09:54:32 STEP 1345 H= 336:15 +CPU= 21.542
  09:54:53 STEP 1346 H= 336:30 +CPU= 21.244
  09:55:14 STEP 1347 H= 336:45 +CPU= 21.283
  09:55:36 STEP 1348 H= 337:00 +CPU= 22.019
  09:55:57 STEP 1349 H= 337:15 +CPU= 21.102
  09:56:18 STEP 1350 H= 337:30 +CPU= 20.808
  09:56:39 STEP 1351 H= 337:45 +CPU= 20.326
  09:57:00 STEP 1352 H= 338:00 +CPU= 21.634
  09:57:22 STEP 1353 H= 338:15 +CPU= 21.291
  09:57:44 STEP 1354 H= 338:30 +CPU= 22.555
  09:58:06 STEP 1355 H= 338:45 +CPU= 21.371
  09:58:33 STEP 1356 H= 339:00 +CPU= 26.754
Moving to projects directory: /home/ij2/slots/15/ICMGGh7zg+001356
Moving to projects directory: /home/ij2/slots/15/ICMSHh7zg+001356
Moving to projects directory: /home/ij2/slots/15/ICMUAh7zg+001356
  09:58:54 STEP 1357 H= 339:15 +CPU= 21.123
  09:59:16 STEP 1358 H= 339:30 +CPU= 22.550
  09:59:37 STEP 1359 H= 339:45 +CPU= 21.082
  09:59:59 STEP 1360 H= 340:00 +CPU= 21.921
  10:00:20 STEP 1361 H= 340:15 +CPU= 20.931
  10:00:43 STEP 1362 H= 340:30 +CPU= 22.453
  10:01:06 STEP 1363 H= 340:45 +CPU= 22.970
  10:01:28 STEP 1364 H= 341:00 +CPU= 22.157
  10:01:51 STEP 1365 H= 341:15 +CPU= 22.637
  10:02:12 STEP 1366 H= 341:30 +CPU= 20.742
  10:02:33 STEP 1367 H= 341:45 +CPU= 21.803
  10:03:00 STEP 1368 H= 342:00 +CPU= 26.883
Moving to projects directory: /home/ij2/slots/15/ICMGGh7zg+001368
Moving to projects directory: /home/ij2/slots/15/ICMSHh7zg+001368
Moving to projects directory: /home/ij2/slots/15/ICMUAh7zg+001368
  10:03:22 STEP 1369 H= 342:15 +CPU= 21.203
  10:03:42 STEP 1370 H= 342:30 +CPU= 20.559
  10:04:03 STEP 1371 H= 342:45 +CPU= 21.289
  10:04:26 STEP 1372 H= 343:00 +CPU= 22.398
  10:04:46 STEP 1373 H= 343:15 +CPU= 20.206
  10:05:09 STEP 1374 H= 343:30 +CPU= 22.446
  10:05:29 STEP 1375 H= 343:45 +CPU= 19.908
  10:05:51 STEP 1376 H= 344:00 +CPU= 22.684
  10:06:12 STEP 1377 H= 344:15 +CPU= 20.746
  10:06:33 STEP 1378 H= 344:30 +CPU= 20.862
  10:06:56 STEP 1379 H= 344:45 +CPU= 22.848
  10:07:22 STEP 1380 H= 345:00 +CPU= 26.118
Moving to projects directory: /home/ij2/slots/15/ICMGGh7zg+001380
Moving to projects directory: /home/ij2/slots/15/ICMSHh7zg+001380
Moving to projects directory: /home/ij2/slots/15/ICMUAh7zg+001380
  10:07:43 STEP 1381 H= 345:15 +CPU= 21.117
  10:08:04 STEP 1382 H= 345:30 +CPU= 20.740
  10:08:27 STEP 1383 H= 345:45 +CPU= 22.494
  10:08:50 STEP 1384 H= 346:00 +CPU= 23.205
  10:09:10 STEP 1385 H= 346:15 +CPU= 20.356
  10:09:34 STEP 1386 H= 346:30 +CPU= 23.461
  10:09:54 STEP 1387 H= 346:45 +CPU= 20.416
  10:10:15 STEP 1388 H= 347:00 +CPU= 20.468
  10:10:37 STEP 1389 H= 347:15 +CPU= 22.207
  10:10:58 STEP 1390 H= 347:30 +CPU= 20.989
  10:11:19 STEP 1391 H= 347:45 +CPU= 21.494
  10:11:49 STEP 1392 H= 348:00 +CPU= 28.812
Moving to projects directory: /home/ij2/slots/15/ICMGGh7zg+001392
Moving to projects directory: /home/ij2/slots/15/ICMSHh7zg+001392
Moving to projects directory: /home/ij2/slots/15/ICMUAh7zg+001392
  10:12:08 STEP 1393 H= 348:15 +CPU= 19.620
  10:12:31 STEP 1394 H= 348:30 +CPU= 22.792
  10:12:52 STEP 1395 H= 348:45 +CPU= 21.435
  10:13:14 STEP 1396 H= 349:00 +CPU= 21.338
  10:13:35 STEP 1397 H= 349:15 +CPU= 20.944
  10:13:55 STEP 1398 H= 349:30 +CPU= 20.265
  10:14:15 STEP 1399 H= 349:45 +CPU= 20.078
  10:14:36 STEP 1400 H= 350:00 +CPU= 20.220
  10:14:57 STEP 1401 H= 350:15 +CPU= 21.713
  10:15:18 STEP 1402 H= 350:30 +CPU= 20.916
  10:15:40 STEP 1403 H= 350:45 +CPU= 21.509
  10:16:07 STEP 1404 H= 351:00 +CPU= 26.863
Moving to projects directory: /home/ij2/slots/15/ICMGGh7zg+001404
Moving to projects directory: /home/ij2/slots/15/ICMSHh7zg+001404
Moving to projects directory: /home/ij2/slots/15/ICMUAh7zg+001404
  10:16:28 STEP 1405 H= 351:15 +CPU= 20.945
  10:16:48 STEP 1406 H= 351:30 +CPU= 20.244
  10:17:06 STEP 1407 H= 351:45 +CPU= 18.401
  10:17:25 STEP 1408 H= 352:00 +CPU= 19.017
  10:17:44 STEP 1409 H= 352:15 +CPU= 18.115
  10:18:03 STEP 1410 H= 352:30 +CPU= 18.918
  10:18:20 STEP 1411 H= 352:45 +CPU= 17.482
  10:18:40 STEP 1412 H= 353:00 +CPU= 19.505
  10:18:57 STEP 1413 H= 353:15 +CPU= 17.510
  10:19:16 STEP 1414 H= 353:30 +CPU= 18.459
  10:19:35 STEP 1415 H= 353:45 +CPU= 19.822
  10:19:59 STEP 1416 H= 354:00 +CPU= 23.130
Moving to projects directory: /home/ij2/slots/15/ICMGGh7zg+001416
Moving to projects directory: /home/ij2/slots/15/ICMSHh7zg+001416
Moving to projects directory: /home/ij2/slots/15/ICMUAh7zg+001416
  10:20:17 STEP 1417 H= 354:15 +CPU= 18.546
  10:20:35 STEP 1418 H= 354:30 +CPU= 18.007
  10:20:54 STEP 1419 H= 354:45 +CPU= 18.519
  10:21:13 STEP 1420 H= 355:00 +CPU= 18.890
  10:21:33 STEP 1421 H= 355:15 +CPU= 19.960
  10:21:52 STEP 1422 H= 355:30 +CPU= 19.753
  10:22:11 STEP 1423 H= 355:45 +CPU= 18.193
  10:22:29 STEP 1424 H= 356:00 +CPU= 18.731
  10:22:48 STEP 1425 H= 356:15 +CPU= 18.672
  10:23:07 STEP 1426 H= 356:30 +CPU= 18.823
  10:23:26 STEP 1427 H= 356:45 +CPU= 18.541
  10:23:49 STEP 1428 H= 357:00 +CPU= 23.362
Moving to projects directory: /home/ij2/slots/15/ICMGGh7zg+001428
Moving to projects directory: /home/ij2/slots/15/ICMSHh7zg+001428
Moving to projects directory: /home/ij2/slots/15/ICMUAh7zg+001428
  10:24:07 STEP 1429 H= 357:15 +CPU= 17.769
  10:24:25 STEP 1430 H= 357:30 +CPU= 18.337
  10:24:43 STEP 1431 H= 357:45 +CPU= 17.865
  10:25:02 STEP 1432 H= 358:00 +CPU= 18.432
  10:25:20 STEP 1433 H= 358:15 +CPU= 18.915
  10:25:38 STEP 1434 H= 358:30 +CPU= 17.622
  10:25:56 STEP 1435 H= 358:45 +CPU= 18.196
  10:26:15 STEP 1436 H= 359:00 +CPU= 18.282
  10:26:33 STEP 1437 H= 359:15 +CPU= 18.046
  10:26:52 STEP 1438 H= 359:30 +CPU= 18.885
  10:27:10 STEP 1439 H= 359:45 +CPU= 18.328
  10:27:36 STEP 1440 H= 360:00 +CPU= 25.720
Moving to projects directory: /home/ij2/slots/15/ICMGGh7zg+001440
Moving to projects directory: /home/ij2/slots/15/ICMSHh7zg+001440
Moving to projects directory: /home/ij2/slots/15/ICMUAh7zg+001440
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMGGh7zg+001344
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMSHh7zg+001344
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMUAh7zg+001344
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMGGh7zg+001356
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMSHh7zg+001356
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMUAh7zg+001356
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMGGh7zg+001368
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMSHh7zg+001368
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMUAh7zg+001368
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMGGh7zg+001380
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMSHh7zg+001380
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMUAh7zg+001380
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMGGh7zg+001392
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMSHh7zg+001392
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMUAh7zg+001392
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMGGh7zg+001404
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMSHh7zg+001404
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMUAh7zg+001404
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMGGh7zg+001416
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMSHh7zg+001416
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMUAh7zg+001416
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMGGh7zg+001428
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMSHh7zg+001428
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMUAh7zg+001428
Zipping up the intermediate file: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_a3f4_2016092300_20_1018_12292759_0_r1107509013_14.zip
Uploading the intermediate file: upload_file_14.zip
  10:27:54 STEP 1441 H= 360:15 +CPU= 18.271
  10:28:12 STEP 1442 H= 360:30 +CPU= 17.711
  10:28:30 STEP 1443 H= 360:45 +CPU= 18.125
  10:28:49 STEP 1444 H= 361:00 +CPU= 18.392
  10:29:07 STEP 1445 H= 361:15 +CPU= 17.848
  10:29:24 STEP 1446 H= 361:30 +CPU= 17.324
  10:29:42 STEP 1447 H= 361:45 +CPU= 18.207
  10:30:01 STEP 1448 H= 362:00 +CPU= 18.512
  10:30:19 STEP 1449 H= 362:15 +CPU= 18.600
  10:30:37 STEP 1450 H= 362:30 +CPU= 17.656
  10:30:55 STEP 1451 H= 362:45 +CPU= 18.316
  10:31:18 STEP 1452 H= 363:00 +CPU= 22.343
Moving to projects directory: /home/ij2/slots/15/ICMGGh7zg+001452
Moving to projects directory: /home/ij2/slots/15/ICMSHh7zg+001452
Moving to projects directory: /home/ij2/slots/15/ICMUAh7zg+001452
  10:31:35 STEP 1453 H= 363:15 +CPU= 17.638
  10:31:53 STEP 1454 H= 363:30 +CPU= 17.243
  10:32:11 STEP 1455 H= 363:45 +CPU= 18.123
  10:32:30 STEP 1456 H= 364:00 +CPU= 18.647
  10:32:47 STEP 1457 H= 364:15 +CPU= 17.222
  10:33:04 STEP 1458 H= 364:30 +CPU= 17.490
  10:33:22 STEP 1459 H= 364:45 +CPU= 17.575
  10:33:40 STEP 1460 H= 365:00 +CPU= 17.750
  10:33:57 STEP 1461 H= 365:15 +CPU= 17.499
  10:34:15 STEP 1462 H= 365:30 +CPU= 17.467
  10:34:33 STEP 1463 H= 365:45 +CPU= 17.832
  10:34:55 STEP 1464 H= 366:00 +CPU= 22.158
Moving to projects directory: /home/ij2/slots/15/ICMGGh7zg+001464
Moving to projects directory: /home/ij2/slots/15/ICMSHh7zg+001464
Moving to projects directory: /home/ij2/slots/15/ICMUAh7zg+001464
  10:35:12 STEP 1465 H= 366:15 +CPU= 17.380
  10:35:30 STEP 1466 H= 366:30 +CPU= 17.710
  10:35:48 STEP 1467 H= 366:45 +CPU= 17.928
  10:36:06 STEP 1468 H= 367:00 +CPU= 18.203
  10:36:24 STEP 1469 H= 367:15 +CPU= 17.568
  10:36:42 STEP 1470 H= 367:30 +CPU= 17.912
  10:36:59 STEP 1471 H= 367:45 +CPU= 17.481
  10:37:17 STEP 1472 H= 368:00 +CPU= 18.044
  10:37:35 STEP 1473 H= 368:15 +CPU= 17.511
  10:37:53 STEP 1474 H= 368:30 +CPU= 17.938
  10:38:11 STEP 1475 H= 368:45 +CPU= 18.216
  10:38:34 STEP 1476 H= 369:00 +CPU= 22.282
Moving to projects directory: /home/ij2/slots/15/ICMGGh7zg+001476
Moving to projects directory: /home/ij2/slots/15/ICMSHh7zg+001476
Moving to projects directory: /home/ij2/slots/15/ICMUAh7zg+001476
  10:38:51 STEP 1477 H= 369:15 +CPU= 17.151
  10:39:08 STEP 1478 H= 369:30 +CPU= 17.057
  10:39:25 STEP 1479 H= 369:45 +CPU= 17.173
  10:39:43 STEP 1480 H= 370:00 +CPU= 17.716
  10:40:00 STEP 1481 H= 370:15 +CPU= 17.735
  10:40:18 STEP 1482 H= 370:30 +CPU= 17.413
  10:40:36 STEP 1483 H= 370:45 +CPU= 17.700
  10:40:53 STEP 1484 H= 371:00 +CPU= 17.772
  10:41:11 STEP 1485 H= 371:15 +CPU= 17.605
  10:41:28 STEP 1486 H= 371:30 +CPU= 17.150
  10:41:46 STEP 1487 H= 371:45 +CPU= 18.286
  10:42:13 STEP 1488 H= 372:00 +CPU= 25.158
Moving to projects directory: /home/ij2/slots/15/ICMGGh7zg+001488
Moving to projects directory: /home/ij2/slots/15/ICMSHh7zg+001488
Moving to projects directory: /home/ij2/slots/15/ICMUAh7zg+001488
  10:42:30 STEP 1489 H= 372:15 +CPU= 17.447
  10:42:47 STEP 1490 H= 372:30 +CPU= 16.858
  10:43:05 STEP 1491 H= 372:45 +CPU= 17.513
  10:43:23 STEP 1492 H= 373:00 +CPU= 18.206
  10:43:41 STEP 1493 H= 373:15 +CPU= 17.461
  10:43:58 STEP 1494 H= 373:30 +CPU= 17.333
  10:44:16 STEP 1495 H= 373:45 +CPU= 17.753
  10:44:34 STEP 1496 H= 374:00 +CPU= 18.652
  10:44:52 STEP 1497 H= 374:15 +CPU= 17.432
  10:45:09 STEP 1498 H= 374:30 +CPU= 17.270
  10:45:26 STEP 1499 H= 374:45 +CPU= 17.324
  10:45:49 STEP 1500 H= 375:00 +CPU= 22.754
Moving to projects directory: /home/ij2/slots/15/ICMGGh7zg+001500
Moving to projects directory: /home/ij2/slots/15/ICMSHh7zg+001500
Moving to projects directory: /home/ij2/slots/15/ICMUAh7zg+001500
  10:46:07 STEP 1501 H= 375:15 +CPU= 17.293
  10:46:24 STEP 1502 H= 375:30 +CPU= 17.560
  10:46:42 STEP 1503 H= 375:45 +CPU= 17.627
  10:47:01 STEP 1504 H= 376:00 +CPU= 19.123
  10:47:19 STEP 1505 H= 376:15 +CPU= 17.660
  10:47:36 STEP 1506 H= 376:30 +CPU= 17.370
  10:47:53 STEP 1507 H= 376:45 +CPU= 17.029
  10:48:11 STEP 1508 H= 377:00 +CPU= 18.159
  10:48:29 STEP 1509 H= 377:15 +CPU= 17.247
  10:48:45 STEP 1510 H= 377:30 +CPU= 16.809
  10:49:03 STEP 1511 H= 377:45 +CPU= 17.409
  10:49:25 STEP 1512 H= 378:00 +CPU= 21.995
Moving to projects directory: /home/ij2/slots/15/ICMGGh7zg+001512
Moving to projects directory: /home/ij2/slots/15/ICMSHh7zg+001512
Moving to projects directory: /home/ij2/slots/15/ICMUAh7zg+001512
  10:49:43 STEP 1513 H= 378:15 +CPU= 17.814
  10:50:00 STEP 1514 H= 378:30 +CPU= 17.784
  10:50:18 STEP 1515 H= 378:45 +CPU= 17.895
  10:50:37 STEP 1516 H= 379:00 +CPU= 18.389
  10:50:54 STEP 1517 H= 379:15 +CPU= 17.649
  10:51:12 STEP 1518 H= 379:30 +CPU= 17.888
  10:51:30 STEP 1519 H= 379:45 +CPU= 17.333
  10:51:48 STEP 1520 H= 380:00 +CPU= 18.424
  10:52:06 STEP 1521 H= 380:15 +CPU= 17.635
  10:52:24 STEP 1522 H= 380:30 +CPU= 17.830
  10:52:41 STEP 1523 H= 380:45 +CPU= 17.774
  10:53:03 STEP 1524 H= 381:00 +CPU= 22.056
Moving to projects directory: /home/ij2/slots/15/ICMGGh7zg+001524
Moving to projects directory: /home/ij2/slots/15/ICMSHh7zg+001524
Moving to projects directory: /home/ij2/slots/15/ICMUAh7zg+001524
  10:53:21 STEP 1525 H= 381:15 +CPU= 17.732
  10:53:39 STEP 1526 H= 381:30 +CPU= 17.343
  10:53:56 STEP 1527 H= 381:45 +CPU= 17.652
  10:54:15 STEP 1528 H= 382:00 +CPU= 18.302
  10:54:32 STEP 1529 H= 382:15 +CPU= 17.205
  10:54:49 STEP 1530 H= 382:30 +CPU= 16.799
  10:55:06 STEP 1531 H= 382:45 +CPU= 17.535
  10:55:25 STEP 1532 H= 383:00 +CPU= 18.342
  10:55:42 STEP 1533 H= 383:15 +CPU= 17.652
  10:56:00 STEP 1534 H= 383:30 +CPU= 17.699
  10:56:17 STEP 1535 H= 383:45 +CPU= 17.188
  10:56:43 STEP 1536 H= 384:00 +CPU= 25.246
Moving to projects directory: /home/ij2/slots/15/ICMGGh7zg+001536
Moving to projects directory: /home/ij2/slots/15/ICMSHh7zg+001536
Moving to projects directory: /home/ij2/slots/15/ICMUAh7zg+001536
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMGGh7zg+001440
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMSHh7zg+001440
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMUAh7zg+001440
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMGGh7zg+001452
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMSHh7zg+001452
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMUAh7zg+001452
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMGGh7zg+001464
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMSHh7zg+001464
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMUAh7zg+001464
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMGGh7zg+001476
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMSHh7zg+001476
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMUAh7zg+001476
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMGGh7zg+001488
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMSHh7zg+001488
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMUAh7zg+001488
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMGGh7zg+001500
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMSHh7zg+001500
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMUAh7zg+001500
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMGGh7zg+001512
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMSHh7zg+001512
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMUAh7zg+001512
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMGGh7zg+001524
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMSHh7zg+001524
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMUAh7zg+001524
Zipping up the intermediate file: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_a3f4_2016092300_20_1018_12292759_0_r1107509013_15.zip
Uploading the intermediate file: upload_file_15.zip
  10:57:00 STEP 1537 H= 384:15 +CPU= 17.049
  10:57:18 STEP 1538 H= 384:30 +CPU= 17.586
  10:57:35 STEP 1539 H= 384:45 +CPU= 17.483
  10:57:53 STEP 1540 H= 385:00 +CPU= 17.732
  10:58:10 STEP 1541 H= 385:15 +CPU= 17.477
  10:58:28 STEP 1542 H= 385:30 +CPU= 17.250
  10:58:45 STEP 1543 H= 385:45 +CPU= 17.312
  10:59:03 STEP 1544 H= 386:00 +CPU= 17.693
  10:59:20 STEP 1545 H= 386:15 +CPU= 17.510
  10:59:38 STEP 1546 H= 386:30 +CPU= 17.564
  10:59:55 STEP 1547 H= 386:45 +CPU= 17.025
  11:00:17 STEP 1548 H= 387:00 +CPU= 22.274
Moving to projects directory: /home/ij2/slots/15/ICMGGh7zg+001548
Moving to projects directory: /home/ij2/slots/15/ICMSHh7zg+001548
Moving to projects directory: /home/ij2/slots/15/ICMUAh7zg+001548
  11:00:35 STEP 1549 H= 387:15 +CPU= 17.658
  11:00:53 STEP 1550 H= 387:30 +CPU= 17.877
  11:01:11 STEP 1551 H= 387:45 +CPU= 17.643
  11:01:28 STEP 1552 H= 388:00 +CPU= 17.863
  11:01:45 STEP 1553 H= 388:15 +CPU= 17.032
  11:02:03 STEP 1554 H= 388:30 +CPU= 17.731
  11:02:20 STEP 1555 H= 388:45 +CPU= 17.068
  11:02:38 STEP 1556 H= 389:00 +CPU= 18.130
  11:02:56 STEP 1557 H= 389:15 +CPU= 17.459
  11:03:13 STEP 1558 H= 389:30 +CPU= 17.292
  11:03:31 STEP 1559 H= 389:45 +CPU= 17.239
  11:03:52 STEP 1560 H= 390:00 +CPU= 21.797
Moving to projects directory: /home/ij2/slots/15/ICMGGh7zg+001560
Moving to projects directory: /home/ij2/slots/15/ICMSHh7zg+001560
Moving to projects directory: /home/ij2/slots/15/ICMUAh7zg+001560
  11:04:10 STEP 1561 H= 390:15 +CPU= 17.203
  11:04:27 STEP 1562 H= 390:30 +CPU= 17.750
  11:04:45 STEP 1563 H= 390:45 +CPU= 17.431
  11:05:10 STEP 1564 H= 391:00 +CPU= 17.873
  11:05:28 STEP 1565 H= 391:15 +CPU= 17.106
  11:05:46 STEP 1566 H= 391:30 +CPU= 17.792
  11:06:03 STEP 1567 H= 391:45 +CPU= 17.583
  11:06:21 STEP 1568 H= 392:00 +CPU= 17.972
  11:06:39 STEP 1569 H= 392:15 +CPU= 17.910
  11:06:56 STEP 1570 H= 392:30 +CPU= 17.006
  11:07:13 STEP 1571 H= 392:45 +CPU= 17.389
  11:07:36 STEP 1572 H= 393:00 +CPU= 22.069
Moving to projects directory: /home/ij2/slots/15/ICMGGh7zg+001572
Moving to projects directory: /home/ij2/slots/15/ICMSHh7zg+001572
Moving to projects directory: /home/ij2/slots/15/ICMUAh7zg+001572
  11:07:53 STEP 1573 H= 393:15 +CPU= 17.226
  11:08:10 STEP 1574 H= 393:30 +CPU= 17.138
  11:08:28 STEP 1575 H= 393:45 +CPU= 17.531
  11:08:46 STEP 1576 H= 394:00 +CPU= 18.034
  11:09:03 STEP 1577 H= 394:15 +CPU= 17.353
  11:09:20 STEP 1578 H= 394:30 +CPU= 17.064
  11:09:37 STEP 1579 H= 394:45 +CPU= 17.265
  11:09:55 STEP 1580 H= 395:00 +CPU= 17.681
  11:10:13 STEP 1581 H= 395:15 +CPU= 17.405
  11:10:30 STEP 1582 H= 395:30 +CPU= 17.441
  11:10:48 STEP 1583 H= 395:45 +CPU= 17.616
  11:11:13 STEP 1584 H= 396:00 +CPU= 25.017
Moving to projects directory: /home/ij2/slots/15/ICMGGh7zg+001584
Moving to projects directory: /home/ij2/slots/15/ICMSHh7zg+001584
Moving to projects directory: /home/ij2/slots/15/ICMUAh7zg+001584
  11:11:31 STEP 1585 H= 396:15 +CPU= 17.178
  11:11:48 STEP 1586 H= 396:30 +CPU= 17.578
  11:12:05 STEP 1587 H= 396:45 +CPU= 16.905
  11:12:23 STEP 1588 H= 397:00 +CPU= 17.652
  11:12:40 STEP 1589 H= 397:15 +CPU= 17.493
  11:12:57 STEP 1590 H= 397:30 +CPU= 17.127
  11:13:14 STEP 1591 H= 397:45 +CPU= 16.895
  11:13:32 STEP 1592 H= 398:00 +CPU= 17.867
  11:13:49 STEP 1593 H= 398:15 +CPU= 17.178
  11:14:07 STEP 1594 H= 398:30 +CPU= 17.005
  11:14:24 STEP 1595 H= 398:45 +CPU= 17.154
  11:14:45 STEP 1596 H= 399:00 +CPU= 21.651
Moving to projects directory: /home/ij2/slots/15/ICMGGh7zg+001596
Moving to projects directory: /home/ij2/slots/15/ICMSHh7zg+001596
Moving to projects directory: /home/ij2/slots/15/ICMUAh7zg+001596
  11:15:02 STEP 1597 H= 399:15 +CPU= 16.970
  11:15:20 STEP 1598 H= 399:30 +CPU= 17.319
  11:15:37 STEP 1599 H= 399:45 +CPU= 17.237
  11:15:55 STEP 1600 H= 400:00 +CPU= 17.821
  11:16:12 STEP 1601 H= 400:15 +CPU= 17.150
  11:16:29 STEP 1602 H= 400:30 +CPU= 17.438
  11:16:47 STEP 1603 H= 400:45 +CPU= 17.440
  11:17:05 STEP 1604 H= 401:00 +CPU= 17.617
  11:17:22 STEP 1605 H= 401:15 +CPU= 17.231
  11:17:39 STEP 1606 H= 401:30 +CPU= 16.929
  11:17:56 STEP 1607 H= 401:45 +CPU= 17.165
  11:18:18 STEP 1608 H= 402:00 +CPU= 21.921
Moving to projects directory: /home/ij2/slots/15/ICMGGh7zg+001608
Moving to projects directory: /home/ij2/slots/15/ICMSHh7zg+001608
Moving to projects directory: /home/ij2/slots/15/ICMUAh7zg+001608
  11:18:35 STEP 1609 H= 402:15 +CPU= 17.363
  11:18:52 STEP 1610 H= 402:30 +CPU= 17.284
  11:19:10 STEP 1611 H= 402:45 +CPU= 17.322
  11:19:28 STEP 1612 H= 403:00 +CPU= 17.906
  11:19:45 STEP 1613 H= 403:15 +CPU= 17.107
  11:20:02 STEP 1614 H= 403:30 +CPU= 17.351
  11:20:19 STEP 1615 H= 403:45 +CPU= 16.956
  11:20:37 STEP 1616 H= 404:00 +CPU= 17.910
  11:20:54 STEP 1617 H= 404:15 +CPU= 17.130
  11:21:11 STEP 1618 H= 404:30 +CPU= 17.087
  11:21:29 STEP 1619 H= 404:45 +CPU= 17.202
  11:21:50 STEP 1620 H= 405:00 +CPU= 21.636
Moving to projects directory: /home/ij2/slots/15/ICMGGh7zg+001620
Moving to projects directory: /home/ij2/slots/15/ICMSHh7zg+001620
Moving to projects directory: /home/ij2/slots/15/ICMUAh7zg+001620
  11:22:07 STEP 1621 H= 405:15 +CPU= 17.012
  11:22:24 STEP 1622 H= 405:30 +CPU= 17.086
  11:22:42 STEP 1623 H= 405:45 +CPU= 17.209
  11:22:59 STEP 1624 H= 406:00 +CPU= 17.420
  11:23:16 STEP 1625 H= 406:15 +CPU= 17.046
  11:23:33 STEP 1626 H= 406:30 +CPU= 17.146
  11:23:50 STEP 1627 H= 406:45 +CPU= 17.174
  11:24:08 STEP 1628 H= 407:00 +CPU= 17.699
  11:24:26 STEP 1629 H= 407:15 +CPU= 17.466
  11:24:43 STEP 1630 H= 407:30 +CPU= 17.434
  11:25:00 STEP 1631 H= 407:45 +CPU= 17.203
  11:25:25 STEP 1632 H= 408:00 +CPU= 24.532
Moving to projects directory: /home/ij2/slots/15/ICMGGh7zg+001632
Moving to projects directory: /home/ij2/slots/15/ICMSHh7zg+001632
Moving to projects directory: /home/ij2/slots/15/ICMUAh7zg+001632
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMGGh7zg+001536
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMSHh7zg+001536
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMUAh7zg+001536
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMGGh7zg+001548
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMSHh7zg+001548
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMUAh7zg+001548
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMGGh7zg+001560
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMSHh7zg+001560
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMUAh7zg+001560
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMGGh7zg+001572
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMSHh7zg+001572
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMUAh7zg+001572
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMGGh7zg+001584
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMSHh7zg+001584
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMUAh7zg+001584
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMGGh7zg+001596
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMSHh7zg+001596
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMUAh7zg+001596
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMGGh7zg+001608
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMSHh7zg+001608
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMUAh7zg+001608
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMGGh7zg+001620
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMSHh7zg+001620
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMUAh7zg+001620
Zipping up the intermediate file: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_a3f4_2016092300_20_1018_12292759_0_r1107509013_16.zip
Uploading the intermediate file: upload_file_16.zip
  11:25:42 STEP 1633 H= 408:15 +CPU= 17.391
  11:26:00 STEP 1634 H= 408:30 +CPU= 17.325
  11:26:17 STEP 1635 H= 408:45 +CPU= 16.896
  11:26:34 STEP 1636 H= 409:00 +CPU= 17.223
  11:26:50 STEP 1637 H= 409:15 +CPU= 16.363
  11:27:07 STEP 1638 H= 409:30 +CPU= 17.033
  11:27:24 STEP 1639 H= 409:45 +CPU= 16.479
  11:27:41 STEP 1640 H= 410:00 +CPU= 17.533
  11:27:58 STEP 1641 H= 410:15 +CPU= 16.679
  11:28:14 STEP 1642 H= 410:30 +CPU= 16.218
  11:28:31 STEP 1643 H= 410:45 +CPU= 16.495
  11:28:51 STEP 1644 H= 411:00 +CPU= 20.604
Moving to projects directory: /home/ij2/slots/15/ICMGGh7zg+001644
Moving to projects directory: /home/ij2/slots/15/ICMSHh7zg+001644
Moving to projects directory: /home/ij2/slots/15/ICMUAh7zg+001644
  11:29:08 STEP 1645 H= 411:15 +CPU= 16.166
  11:29:24 STEP 1646 H= 411:30 +CPU= 16.339
  11:29:40 STEP 1647 H= 411:45 +CPU= 16.161
  11:29:57 STEP 1648 H= 412:00 +CPU= 16.654
  11:30:13 STEP 1649 H= 412:15 +CPU= 16.164
  11:30:29 STEP 1650 H= 412:30 +CPU= 16.217
  11:30:45 STEP 1651 H= 412:45 +CPU= 16.274
  11:31:03 STEP 1652 H= 413:00 +CPU= 17.020
  11:31:19 STEP 1653 H= 413:15 +CPU= 16.091
  11:31:35 STEP 1654 H= 413:30 +CPU= 16.094
  11:31:51 STEP 1655 H= 413:45 +CPU= 16.039
  11:32:11 STEP 1656 H= 414:00 +CPU= 20.570
Moving to projects directory: /home/ij2/slots/15/ICMGGh7zg+001656
Moving to projects directory: /home/ij2/slots/15/ICMSHh7zg+001656
Moving to projects directory: /home/ij2/slots/15/ICMUAh7zg+001656
  11:32:28 STEP 1657 H= 414:15 +CPU= 16.101
  11:32:44 STEP 1658 H= 414:30 +CPU= 16.064
  11:33:00 STEP 1659 H= 414:45 +CPU= 16.066
  11:33:16 STEP 1660 H= 415:00 +CPU= 16.460
  11:33:32 STEP 1661 H= 415:15 +CPU= 16.085
  11:33:48 STEP 1662 H= 415:30 +CPU= 16.015
  11:34:04 STEP 1663 H= 415:45 +CPU= 16.021
  11:34:21 STEP 1664 H= 416:00 +CPU= 16.543
  11:34:37 STEP 1665 H= 416:15 +CPU= 16.085
  11:34:53 STEP 1666 H= 416:30 +CPU= 16.062
  11:35:09 STEP 1667 H= 416:45 +CPU= 16.059
  11:35:30 STEP 1668 H= 417:00 +CPU= 20.527
Moving to projects directory: /home/ij2/slots/15/ICMGGh7zg+001668
Moving to projects directory: /home/ij2/slots/15/ICMSHh7zg+001668
Moving to projects directory: /home/ij2/slots/15/ICMUAh7zg+001668
  11:35:46 STEP 1669 H= 417:15 +CPU= 16.089
  11:36:02 STEP 1670 H= 417:30 +CPU= 16.066
  11:36:18 STEP 1671 H= 417:45 +CPU= 16.077
  11:36:34 STEP 1672 H= 418:00 +CPU= 16.507
  11:36:50 STEP 1673 H= 418:15 +CPU= 16.017
  11:37:06 STEP 1674 H= 418:30 +CPU= 16.049
  11:37:22 STEP 1675 H= 418:45 +CPU= 16.030
  11:37:39 STEP 1676 H= 419:00 +CPU= 16.779
  11:37:55 STEP 1677 H= 419:15 +CPU= 15.994
  11:38:11 STEP 1678 H= 419:30 +CPU= 16.001
  11:38:27 STEP 1679 H= 419:45 +CPU= 15.994
  11:38:50 STEP 1680 H= 420:00 +CPU= 22.796
Moving to projects directory: /home/ij2/slots/15/ICMGGh7zg+001680
Moving to projects directory: /home/ij2/slots/15/ICMSHh7zg+001680
Moving to projects directory: /home/ij2/slots/15/ICMUAh7zg+001680
  11:39:06 STEP 1681 H= 420:15 +CPU= 16.066
  11:39:22 STEP 1682 H= 420:30 +CPU= 16.079
  11:39:38 STEP 1683 H= 420:45 +CPU= 16.005
  11:39:55 STEP 1684 H= 421:00 +CPU= 16.525
  11:40:11 STEP 1685 H= 421:15 +CPU= 15.992
  11:40:27 STEP 1686 H= 421:30 +CPU= 15.974
  11:40:43 STEP 1687 H= 421:45 +CPU= 16.014
  11:41:00 STEP 1688 H= 422:00 +CPU= 16.486
  11:41:16 STEP 1689 H= 422:15 +CPU= 16.065
  11:41:32 STEP 1690 H= 422:30 +CPU= 16.099
  11:41:48 STEP 1691 H= 422:45 +CPU= 16.074
  11:42:08 STEP 1692 H= 423:00 +CPU= 20.558
Moving to projects directory: /home/ij2/slots/15/ICMGGh7zg+001692
Moving to projects directory: /home/ij2/slots/15/ICMSHh7zg+001692
Moving to projects directory: /home/ij2/slots/15/ICMUAh7zg+001692
  11:42:24 STEP 1693 H= 423:15 +CPU= 16.067
  11:42:41 STEP 1694 H= 423:30 +CPU= 16.285
  11:42:57 STEP 1695 H= 423:45 +CPU= 16.035
  11:43:13 STEP 1696 H= 424:00 +CPU= 16.545
  11:43:29 STEP 1697 H= 424:15 +CPU= 16.052
  11:43:45 STEP 1698 H= 424:30 +CPU= 16.039
  11:44:01 STEP 1699 H= 424:45 +CPU= 15.991
  11:44:18 STEP 1700 H= 425:00 +CPU= 16.486
  11:44:34 STEP 1701 H= 425:15 +CPU= 15.990
  11:44:50 STEP 1702 H= 425:30 +CPU= 16.015
  11:45:06 STEP 1703 H= 425:45 +CPU= 16.035
  11:45:27 STEP 1704 H= 426:00 +CPU= 20.557
Moving to projects directory: /home/ij2/slots/15/ICMGGh7zg+001704
Moving to projects directory: /home/ij2/slots/15/ICMSHh7zg+001704
Moving to projects directory: /home/ij2/slots/15/ICMUAh7zg+001704
  11:45:43 STEP 1705 H= 426:15 +CPU= 16.015
  11:45:59 STEP 1706 H= 426:30 +CPU= 16.044
  11:46:15 STEP 1707 H= 426:45 +CPU= 16.049
  11:46:31 STEP 1708 H= 427:00 +CPU= 16.564
  11:46:47 STEP 1709 H= 427:15 +CPU= 16.106
  11:47:03 STEP 1710 H= 427:30 +CPU= 16.073
  11:47:20 STEP 1711 H= 427:45 +CPU= 16.106
  11:47:36 STEP 1712 H= 428:00 +CPU= 16.642
  11:47:52 STEP 1713 H= 428:15 +CPU= 16.110
  11:48:08 STEP 1714 H= 428:30 +CPU= 16.119
  11:48:25 STEP 1715 H= 428:45 +CPU= 16.096
  11:48:45 STEP 1716 H= 429:00 +CPU= 20.598
Moving to projects directory: /home/ij2/slots/15/ICMGGh7zg+001716
Moving to projects directory: /home/ij2/slots/15/ICMSHh7zg+001716
Moving to projects directory: /home/ij2/slots/15/ICMUAh7zg+001716
  11:49:01 STEP 1717 H= 429:15 +CPU= 16.093
  11:49:17 STEP 1718 H= 429:30 +CPU= 16.092
  11:49:33 STEP 1719 H= 429:45 +CPU= 16.053
  11:49:50 STEP 1720 H= 430:00 +CPU= 16.587
  11:50:06 STEP 1721 H= 430:15 +CPU= 16.072
  11:50:22 STEP 1722 H= 430:30 +CPU= 16.096
  11:50:38 STEP 1723 H= 430:45 +CPU= 16.096
  11:50:55 STEP 1724 H= 431:00 +CPU= 16.570
  11:51:11 STEP 1725 H= 431:15 +CPU= 16.030
  11:51:27 STEP 1726 H= 431:30 +CPU= 16.117
  11:51:43 STEP 1727 H= 431:45 +CPU= 16.222
  11:52:06 STEP 1728 H= 432:00 +CPU= 22.916
Moving to projects directory: /home/ij2/slots/15/ICMGGh7zg+001728
Moving to projects directory: /home/ij2/slots/15/ICMSHh7zg+001728
Moving to projects directory: /home/ij2/slots/15/ICMUAh7zg+001728
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMGGh7zg+001632
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMSHh7zg+001632
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMUAh7zg+001632
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMGGh7zg+001644
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMSHh7zg+001644
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMUAh7zg+001644
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMGGh7zg+001656
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMSHh7zg+001656
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMUAh7zg+001656
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMGGh7zg+001668
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMSHh7zg+001668
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMUAh7zg+001668
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMGGh7zg+001680
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMSHh7zg+001680
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMUAh7zg+001680
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMGGh7zg+001692
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMSHh7zg+001692
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMUAh7zg+001692
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMGGh7zg+001704
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMSHh7zg+001704
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMUAh7zg+001704
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMGGh7zg+001716
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMSHh7zg+001716
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMUAh7zg+001716
Zipping up the intermediate file: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_a3f4_2016092300_20_1018_12292759_0_r1107509013_17.zip
Uploading the intermediate file: upload_file_17.zip
  11:52:22 STEP 1729 H= 432:15 +CPU= 16.047
  11:52:39 STEP 1730 H= 432:30 +CPU= 16.086
  11:52:55 STEP 1731 H= 432:45 +CPU= 16.130
  11:53:11 STEP 1732 H= 433:00 +CPU= 16.613
  11:53:27 STEP 1733 H= 433:15 +CPU= 16.099
  11:53:44 STEP 1734 H= 433:30 +CPU= 16.093
  11:54:00 STEP 1735 H= 433:45 +CPU= 16.084
  11:54:16 STEP 1736 H= 434:00 +CPU= 16.585
  11:54:32 STEP 1737 H= 434:15 +CPU= 16.122
  11:54:48 STEP 1738 H= 434:30 +CPU= 16.067
  11:55:05 STEP 1739 H= 434:45 +CPU= 16.072
  11:55:25 STEP 1740 H= 435:00 +CPU= 20.451
Moving to projects directory: /home/ij2/slots/15/ICMGGh7zg+001740
Moving to projects directory: /home/ij2/slots/15/ICMSHh7zg+001740
Moving to projects directory: /home/ij2/slots/15/ICMUAh7zg+001740
  11:55:41 STEP 1741 H= 435:15 +CPU= 16.088
  11:55:57 STEP 1742 H= 435:30 +CPU= 16.057
  11:56:13 STEP 1743 H= 435:45 +CPU= 16.059
  11:56:30 STEP 1744 H= 436:00 +CPU= 16.532
  11:56:46 STEP 1745 H= 436:15 +CPU= 16.130
  11:57:02 STEP 1746 H= 436:30 +CPU= 16.104
  11:57:18 STEP 1747 H= 436:45 +CPU= 16.077
  11:57:35 STEP 1748 H= 437:00 +CPU= 16.613
  11:57:51 STEP 1749 H= 437:15 +CPU= 16.110
  11:58:07 STEP 1750 H= 437:30 +CPU= 16.079
  11:58:23 STEP 1751 H= 437:45 +CPU= 16.056
  11:58:44 STEP 1752 H= 438:00 +CPU= 20.559
Moving to projects directory: /home/ij2/slots/15/ICMGGh7zg+001752
Moving to projects directory: /home/ij2/slots/15/ICMSHh7zg+001752
Moving to projects directory: /home/ij2/slots/15/ICMUAh7zg+001752
  11:59:00 STEP 1753 H= 438:15 +CPU= 15.988
  11:59:16 STEP 1754 H= 438:30 +CPU= 16.099
  11:59:32 STEP 1755 H= 438:45 +CPU= 16.078
  11:59:48 STEP 1756 H= 439:00 +CPU= 16.556
  12:00:04 STEP 1757 H= 439:15 +CPU= 16.061
  12:00:20 STEP 1758 H= 439:30 +CPU= 16.086
  12:00:37 STEP 1759 H= 439:45 +CPU= 16.059
  12:00:53 STEP 1760 H= 440:00 +CPU= 16.534
  12:01:09 STEP 1761 H= 440:15 +CPU= 16.013
  12:01:25 STEP 1762 H= 440:30 +CPU= 16.045
  12:01:41 STEP 1763 H= 440:45 +CPU= 16.017
  12:02:02 STEP 1764 H= 441:00 +CPU= 20.453
Moving to projects directory: /home/ij2/slots/15/ICMGGh7zg+001764
Moving to projects directory: /home/ij2/slots/15/ICMSHh7zg+001764
Moving to projects directory: /home/ij2/slots/15/ICMUAh7zg+001764
  12:02:18 STEP 1765 H= 441:15 +CPU= 16.002
  12:02:34 STEP 1766 H= 441:30 +CPU= 16.064
  12:02:50 STEP 1767 H= 441:45 +CPU= 16.034
  12:03:06 STEP 1768 H= 442:00 +CPU= 16.542
  12:03:22 STEP 1769 H= 442:15 +CPU= 16.062
  12:03:38 STEP 1770 H= 442:30 +CPU= 16.070
  12:03:55 STEP 1771 H= 442:45 +CPU= 16.019
  12:04:11 STEP 1772 H= 443:00 +CPU= 16.584
  12:04:27 STEP 1773 H= 443:15 +CPU= 16.063
  12:04:43 STEP 1774 H= 443:30 +CPU= 16.051
  12:04:59 STEP 1775 H= 443:45 +CPU= 16.049
  12:05:22 STEP 1776 H= 444:00 +CPU= 22.876
Moving to projects directory: /home/ij2/slots/15/ICMGGh7zg+001776
Moving to projects directory: /home/ij2/slots/15/ICMSHh7zg+001776
Moving to projects directory: /home/ij2/slots/15/ICMUAh7zg+001776
  12:05:39 STEP 1777 H= 444:15 +CPU= 16.327
  12:05:55 STEP 1778 H= 444:30 +CPU= 16.031
  12:06:11 STEP 1779 H= 444:45 +CPU= 16.024
  12:06:27 STEP 1780 H= 445:00 +CPU= 16.484
  12:06:43 STEP 1781 H= 445:15 +CPU= 15.987
  12:06:59 STEP 1782 H= 445:30 +CPU= 15.961
  12:07:15 STEP 1783 H= 445:45 +CPU= 15.970
  12:07:32 STEP 1784 H= 446:00 +CPU= 16.491
  12:07:48 STEP 1785 H= 446:15 +CPU= 15.973
  12:08:04 STEP 1786 H= 446:30 +CPU= 15.990
  12:08:20 STEP 1787 H= 446:45 +CPU= 16.030
  12:08:40 STEP 1788 H= 447:00 +CPU= 20.443
Moving to projects directory: /home/ij2/slots/15/ICMGGh7zg+001788
Moving to projects directory: /home/ij2/slots/15/ICMSHh7zg+001788
Moving to projects directory: /home/ij2/slots/15/ICMUAh7zg+001788
  12:08:56 STEP 1789 H= 447:15 +CPU= 16.110
  12:09:12 STEP 1790 H= 447:30 +CPU= 16.001
  12:09:28 STEP 1791 H= 447:45 +CPU= 16.043
  12:09:45 STEP 1792 H= 448:00 +CPU= 16.534
  12:10:01 STEP 1793 H= 448:15 +CPU= 16.018
  12:10:17 STEP 1794 H= 448:30 +CPU= 16.058
  12:10:33 STEP 1795 H= 448:45 +CPU= 16.041
  12:10:50 STEP 1796 H= 449:00 +CPU= 16.505
  12:11:06 STEP 1797 H= 449:15 +CPU= 16.003
  12:11:22 STEP 1798 H= 449:30 +CPU= 15.963
  12:11:37 STEP 1799 H= 449:45 +CPU= 15.977
  12:11:58 STEP 1800 H= 450:00 +CPU= 20.548
Moving to projects directory: /home/ij2/slots/15/ICMGGh7zg+001800
Moving to projects directory: /home/ij2/slots/15/ICMSHh7zg+001800
Moving to projects directory: /home/ij2/slots/15/ICMUAh7zg+001800
  12:12:14 STEP 1801 H= 450:15 +CPU= 16.033
  12:12:30 STEP 1802 H= 450:30 +CPU= 15.998
  12:12:46 STEP 1803 H= 450:45 +CPU= 15.950
  12:13:03 STEP 1804 H= 451:00 +CPU= 16.489
  12:13:19 STEP 1805 H= 451:15 +CPU= 16.017
  12:13:35 STEP 1806 H= 451:30 +CPU= 16.022
  12:13:51 STEP 1807 H= 451:45 +CPU= 16.070
  12:14:07 STEP 1808 H= 452:00 +CPU= 16.537
  12:14:23 STEP 1809 H= 452:15 +CPU= 15.986
  12:14:39 STEP 1810 H= 452:30 +CPU= 16.004
  12:14:55 STEP 1811 H= 452:45 +CPU= 16.048
  12:15:16 STEP 1812 H= 453:00 +CPU= 20.549
Moving to projects directory: /home/ij2/slots/15/ICMGGh7zg+001812
Moving to projects directory: /home/ij2/slots/15/ICMSHh7zg+001812
Moving to projects directory: /home/ij2/slots/15/ICMUAh7zg+001812
  12:15:32 STEP 1813 H= 453:15 +CPU= 16.046
  12:15:48 STEP 1814 H= 453:30 +CPU= 16.062
  12:16:04 STEP 1815 H= 453:45 +CPU= 16.071
  12:16:21 STEP 1816 H= 454:00 +CPU= 16.490
  12:16:37 STEP 1817 H= 454:15 +CPU= 15.975
  12:16:52 STEP 1818 H= 454:30 +CPU= 15.950
  12:17:08 STEP 1819 H= 454:45 +CPU= 15.984
  12:17:25 STEP 1820 H= 455:00 +CPU= 16.466
  12:17:41 STEP 1821 H= 455:15 +CPU= 15.977
  12:17:57 STEP 1822 H= 455:30 +CPU= 15.982
  12:18:13 STEP 1823 H= 455:45 +CPU= 15.993
  12:18:36 STEP 1824 H= 456:00 +CPU= 22.868
Moving to projects directory: /home/ij2/slots/15/ICMGGh7zg+001824
Moving to projects directory: /home/ij2/slots/15/ICMSHh7zg+001824
Moving to projects directory: /home/ij2/slots/15/ICMUAh7zg+001824
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMGGh7zg+001728
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMSHh7zg+001728
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMUAh7zg+001728
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMGGh7zg+001740
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMSHh7zg+001740
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMUAh7zg+001740
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMGGh7zg+001752
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMSHh7zg+001752
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMUAh7zg+001752
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMGGh7zg+001764
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMSHh7zg+001764
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMUAh7zg+001764
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMGGh7zg+001776
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMSHh7zg+001776
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMUAh7zg+001776
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMGGh7zg+001788
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMSHh7zg+001788
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMUAh7zg+001788
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMGGh7zg+001800
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMSHh7zg+001800
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMUAh7zg+001800
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMGGh7zg+001812
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMSHh7zg+001812
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMUAh7zg+001812
Zipping up the intermediate file: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_a3f4_2016092300_20_1018_12292759_0_r1107509013_18.zip
Uploading the intermediate file: upload_file_18.zip
  12:18:52 STEP 1825 H= 456:15 +CPU= 16.085
  12:19:08 STEP 1826 H= 456:30 +CPU= 15.949
  12:19:24 STEP 1827 H= 456:45 +CPU= 16.031
  12:19:41 STEP 1828 H= 457:00 +CPU= 16.567
  12:19:57 STEP 1829 H= 457:15 +CPU= 15.999
  12:20:13 STEP 1830 H= 457:30 +CPU= 16.029
  12:20:29 STEP 1831 H= 457:45 +CPU= 16.002
  12:20:45 STEP 1832 H= 458:00 +CPU= 16.527
  12:21:01 STEP 1833 H= 458:15 +CPU= 15.989
  12:21:17 STEP 1834 H= 458:30 +CPU= 16.002
  12:21:33 STEP 1835 H= 458:45 +CPU= 15.979
  12:21:54 STEP 1836 H= 459:00 +CPU= 20.555
Moving to projects directory: /home/ij2/slots/15/ICMGGh7zg+001836
Moving to projects directory: /home/ij2/slots/15/ICMSHh7zg+001836
Moving to projects directory: /home/ij2/slots/15/ICMUAh7zg+001836
  12:22:10 STEP 1837 H= 459:15 +CPU= 16.218
  12:22:26 STEP 1838 H= 459:30 +CPU= 16.021
  12:22:42 STEP 1839 H= 459:45 +CPU= 16.000
  12:22:59 STEP 1840 H= 460:00 +CPU= 16.450
  12:23:15 STEP 1841 H= 460:15 +CPU= 15.967
  12:23:31 STEP 1842 H= 460:30 +CPU= 15.950
  12:23:47 STEP 1843 H= 460:45 +CPU= 16.014
  12:24:03 STEP 1844 H= 461:00 +CPU= 16.502
  12:24:19 STEP 1845 H= 461:15 +CPU= 15.998
  12:24:35 STEP 1846 H= 461:30 +CPU= 16.001
  12:24:51 STEP 1847 H= 461:45 +CPU= 16.009
  12:25:12 STEP 1848 H= 462:00 +CPU= 20.481
Moving to projects directory: /home/ij2/slots/15/ICMGGh7zg+001848
Moving to projects directory: /home/ij2/slots/15/ICMSHh7zg+001848
Moving to projects directory: /home/ij2/slots/15/ICMUAh7zg+001848
  12:25:28 STEP 1849 H= 462:15 +CPU= 16.028
  12:25:44 STEP 1850 H= 462:30 +CPU= 16.000
  12:26:00 STEP 1851 H= 462:45 +CPU= 16.023
  12:26:16 STEP 1852 H= 463:00 +CPU= 16.500
  12:26:32 STEP 1853 H= 463:15 +CPU= 16.041
  12:26:48 STEP 1854 H= 463:30 +CPU= 15.963
  12:27:04 STEP 1855 H= 463:45 +CPU= 15.977
  12:27:21 STEP 1856 H= 464:00 +CPU= 16.431
  12:27:37 STEP 1857 H= 464:15 +CPU= 15.943
  12:27:53 STEP 1858 H= 464:30 +CPU= 15.938
  12:28:08 STEP 1859 H= 464:45 +CPU= 15.949
  12:28:29 STEP 1860 H= 465:00 +CPU= 20.345
Moving to projects directory: /home/ij2/slots/15/ICMGGh7zg+001860
Moving to projects directory: /home/ij2/slots/15/ICMSHh7zg+001860
Moving to projects directory: /home/ij2/slots/15/ICMUAh7zg+001860
  12:28:45 STEP 1861 H= 465:15 +CPU= 16.004
  12:29:01 STEP 1862 H= 465:30 +CPU= 15.930
  12:29:17 STEP 1863 H= 465:45 +CPU= 15.970
  12:29:33 STEP 1864 H= 466:00 +CPU= 16.443
  12:29:49 STEP 1865 H= 466:15 +CPU= 16.002
  12:30:05 STEP 1866 H= 466:30 +CPU= 15.950
  12:30:21 STEP 1867 H= 466:45 +CPU= 16.046
  12:30:38 STEP 1868 H= 467:00 +CPU= 16.540
  12:30:54 STEP 1869 H= 467:15 +CPU= 16.011
  12:31:10 STEP 1870 H= 467:30 +CPU= 16.007
  12:31:26 STEP 1871 H= 467:45 +CPU= 15.960
  12:31:49 STEP 1872 H= 468:00 +CPU= 22.836
Moving to projects directory: /home/ij2/slots/15/ICMGGh7zg+001872
Moving to projects directory: /home/ij2/slots/15/ICMSHh7zg+001872
Moving to projects directory: /home/ij2/slots/15/ICMUAh7zg+001872
  12:32:05 STEP 1873 H= 468:15 +CPU= 16.071
  12:32:21 STEP 1874 H= 468:30 +CPU= 16.033
  12:32:37 STEP 1875 H= 468:45 +CPU= 16.025
  12:32:54 STEP 1876 H= 469:00 +CPU= 16.542
  12:33:10 STEP 1877 H= 469:15 +CPU= 16.020
  12:33:26 STEP 1878 H= 469:30 +CPU= 16.217
  12:33:42 STEP 1879 H= 469:45 +CPU= 16.013
  12:33:58 STEP 1880 H= 470:00 +CPU= 16.558
  12:34:14 STEP 1881 H= 470:15 +CPU= 15.999
  12:34:30 STEP 1882 H= 470:30 +CPU= 15.964
  12:34:46 STEP 1883 H= 470:45 +CPU= 16.061
  12:35:07 STEP 1884 H= 471:00 +CPU= 20.512
Moving to projects directory: /home/ij2/slots/15/ICMGGh7zg+001884
Moving to projects directory: /home/ij2/slots/15/ICMSHh7zg+001884
Moving to projects directory: /home/ij2/slots/15/ICMUAh7zg+001884
  12:35:23 STEP 1885 H= 471:15 +CPU= 16.223
  12:35:39 STEP 1886 H= 471:30 +CPU= 15.979
  12:35:55 STEP 1887 H= 471:45 +CPU= 15.985
  12:36:12 STEP 1888 H= 472:00 +CPU= 16.502
  12:36:28 STEP 1889 H= 472:15 +CPU= 15.992
  12:36:44 STEP 1890 H= 472:30 +CPU= 15.966
  12:37:00 STEP 1891 H= 472:45 +CPU= 16.012
  12:37:16 STEP 1892 H= 473:00 +CPU= 16.480
  12:37:32 STEP 1893 H= 473:15 +CPU= 15.933
  12:37:48 STEP 1894 H= 473:30 +CPU= 15.920
  12:38:04 STEP 1895 H= 473:45 +CPU= 15.918
  12:38:24 STEP 1896 H= 474:00 +CPU= 20.381
Moving to projects directory: /home/ij2/slots/15/ICMGGh7zg+001896
Moving to projects directory: /home/ij2/slots/15/ICMSHh7zg+001896
Moving to projects directory: /home/ij2/slots/15/ICMUAh7zg+001896
  12:38:40 STEP 1897 H= 474:15 +CPU= 15.934
  12:38:56 STEP 1898 H= 474:30 +CPU= 15.949
  12:39:12 STEP 1899 H= 474:45 +CPU= 15.969
  12:39:29 STEP 1900 H= 475:00 +CPU= 16.450
  12:39:45 STEP 1901 H= 475:15 +CPU= 15.978
  12:40:01 STEP 1902 H= 475:30 +CPU= 16.006
  12:40:17 STEP 1903 H= 475:45 +CPU= 16.039
  12:40:33 STEP 1904 H= 476:00 +CPU= 16.498
  12:40:49 STEP 1905 H= 476:15 +CPU= 16.016
  12:41:05 STEP 1906 H= 476:30 +CPU= 16.024
  12:41:21 STEP 1907 H= 476:45 +CPU= 15.997
  12:41:42 STEP 1908 H= 477:00 +CPU= 20.470
Moving to projects directory: /home/ij2/slots/15/ICMGGh7zg+001908
Moving to projects directory: /home/ij2/slots/15/ICMSHh7zg+001908
Moving to projects directory: /home/ij2/slots/15/ICMUAh7zg+001908
  12:41:58 STEP 1909 H= 477:15 +CPU= 16.006
  12:42:14 STEP 1910 H= 477:30 +CPU= 16.003
  12:42:30 STEP 1911 H= 477:45 +CPU= 15.962
  12:42:46 STEP 1912 H= 478:00 +CPU= 16.418
  12:43:02 STEP 1913 H= 478:15 +CPU= 15.919
  12:43:18 STEP 1914 H= 478:30 +CPU= 15.965
  12:43:34 STEP 1915 H= 478:45 +CPU= 15.928
  12:43:51 STEP 1916 H= 479:00 +CPU= 16.526
  12:44:07 STEP 1917 H= 479:15 +CPU= 16.011
  12:44:23 STEP 1918 H= 479:30 +CPU= 16.028
  12:44:39 STEP 1919 H= 479:45 +CPU= 16.022
  12:45:01 STEP 1920 H= 480:00 +CPU= 21.733
..The child process terminated with status: 0
>>> Printing last 70 lines from file: NODE.001_01

I   VARIABLE   I      Initial value         I        Current value        I
I-------------------------------------------------------------------------I
I  OMEGA (P/S) I      0.7684820014E-21      I       0.1263903337E-07      I
ICloud fractionI      0.0000000000E+00      I       0.1982866514E-01      I
I  Relat. Hum. I      0.4371502982E+00      I       0.4719108245E+00      I
I  PASS 01     I      0.0000000000E+00      I       0.4544545583E+02      I
DDH------------------------------------------------------------------------
  
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
 NSTEP =  1920 SCAN2M_HPOS  P
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ICMGGh7zg+001920 MODE=a
 IO-STREAM CLOSED - ICMGGh7zg+001920
 MPI-TASK:   1 -    315551700 BYTES IN      7 RECORDS TRANSFERRED IN    0.0003 SECONDS******** Mbytes/s, TOTAL TIME=    0.0070( 4.3%)
 12:45:01 STEP 1920 H= 480:00 +CPU= 21.733
 END CNT3
 NSTEP =  1920 CNT0   000000000  15.457  15.457  15.552   0.994               0               0               0               0               0
IO-STREAM STATISTICS, TOTAL NO OF REC/BYTES READ            0/           0 WRITTEN        57546/************
 -TASK-OPENED-OPEN-RECS IN -KBYTE IN    -RECS OUT-KBYTE OUT-WALL   -WALL IN-WALL
  OU-TOT IN -TOT OUT
   1   1008    0        0            0    57546 ************    79.5     0.0    79.4     0.0   161.8
===-=== START OF TIMING STATISTICS ===-===
 
STATS FOR ALL TASKS
 NUM ROUTINE                                     CALLS  MEAN(ms)   MAX(ms)   FRAC(%)  UNBAL(%)
   0 CNT0     - COMPLETE EXECUTION                   1 ********* *********    100.00      0.00
   1 CNT4     - FORWARD INTEGRATION                  1 ********* *********     99.99      0.00
   8 SCAN2M - GRID-POINT DYNAMICS                 2082   15009.1   15009.1     47.93      0.00
   9 SPCM     - SPECTRAL COMP.                    1920     552.4     552.4      1.63      0.00
  10 SCAN2M - PHYSICS                             1921   10061.2   10061.2     29.64      0.00
  11 IOPACK   - OUTPUT P.P. RESULTS                161     758.5     758.5      0.19      0.00
  12 SPNORM   - SPECTRAL NORM COMP.                 83      42.1      42.1      0.01      0.00
  14 SUINIF                                          1    5140.7    5140.7      0.01      0.00
  17 GRIDFPOS IN CNT4                              161      23.2      23.2      0.01      0.00
  18 SUSPECG                                         1    2608.7    2608.7      0.00      0.00
  19 SUSPEC                                          1    2615.9    2615.9      0.00      0.00
  24 SUGRIDU                                         1    2020.4    2020.4      0.00      0.00
  25 SPECRT                                          1     226.4     226.4      0.00      0.00
  26 SUGRIDF                                         1     277.9     277.9      0.00      0.00
  27 RESTART FILES - WRITING                        40    3052.3    3052.3      0.19      0.00
  28 RESTART FILES - READING                         1       0.0       0.0      0.00      0.00
  29 SU4FPOS IN CNT4                               161       0.2       0.2      0.00      0.00
  30 DYNFPOS IN CNT4                               161    4853.2    4853.2      1.20      0.00
  31 POSDDH IN STEPO                                 9      31.9      31.9      0.00      0.00
  37 CPGLAG   - SL COMPUTATIONS                   1921    4518.1    4518.1     13.31      0.00
  39 SU0YOMB                                         1     588.2     588.2      0.00      0.00
  51 SCAN2M   - SL COMM. PART 1                   1921      18.9      18.9      0.06      0.00
  54 SPCM     - M TO S/S TO M TRANSP.             1920     184.3     184.3      0.54      0.00
  55 SPCIMPF  - S TO M/M TO S TRANSP.             1920      46.4      46.4      0.14      0.00
  56 SPNORM   - SPECTRAL NORM COMM.                 83       2.3       2.3      0.00      0.00
 102 LTINV_CTL   - INVERSE LEGENDRE TRANSFORM     4165     292.3     292.3      1.87      0.00
 103 LTDIR_CTL   - DIRECT LEGENDRE TRANSFORM      4006     184.7     184.7      1.13      0.00
 106 FTDIR_CTL   - DIRECT FOURIER TRANSFORM       4006      92.3      92.3      0.57      0.00
 107 FTINV_CTL   - INVERSE FOURIER TRANSFORM      4165     102.9     102.9      0.66      0.00
 140 SULEG       - COMP. OF LEGENDRE POL.            1      46.9      46.9      0.00      0.00
 152 LTINV_CTL   - M TO L TRANSPOSITION           4165      59.1      59.1      0.38      0.00
 153 LTDIR_CTL   - L TO M TRANSPOSITION           4006      29.5      29.5      0.18      0.00
 157 FTINV_CTL   - L TO G TRANSPOSITION           4165      64.7      64.7      0.41      0.00
 158 FTDIR_CTL   - G TO L TRANSPOSITION           4006      31.4      31.4      0.19      0.00
 400 GSTATS                                     223224       0.0       0.0      0.00      0.00
 401 GSTATS HOOK                                207696       0.0       0.0      0.00      0.00
TOTAL MEASURED IMBALANCE =       0.0 SECONDS,  0.0 PERCENT
TOTAL WALLCLOCK TIME    65203.4 CPU TIME   40978.2 VECTOR TIME    40978.2


===-=== END   OF TIMING STATISTICS ===-===
FORECAST DAYS PER DAY       26.5
  *** END CNT0 *** 
------------------------------------------------
Moving to projects directory: /home/ij2/slots/15/ICMGGh7zg+001920
Moving to projects directory: /home/ij2/slots/15/ICMSHh7zg+001920
Moving to projects directory: /home/ij2/slots/15/ICMUAh7zg+001920
Adding to the zip: /home/ij2/slots/15/NODE.001_01
Adding to the zip: /home/ij2/slots/15/ifs.stat
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMUAh7zg+001836
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMUAh7zg+001860
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMUAh7zg+001884
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMSHh7zg+001836
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMGGh7zg+001836
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMSHh7zg+001884
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMSHh7zg+001860
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMGGh7zg+001860
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMGGh7zg+001884
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMUAh7zg+001848
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMGGh7zg+001848
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMSHh7zg+001848
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMSHh7zg+001824
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMGGh7zg+001824
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMSHh7zg+001896
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMSHh7zg+001872
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMSHh7zg+001908
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMGGh7zg+001908
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMGGh7zg+001872
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMGGh7zg+001896
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMUAh7zg+001824
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMUAh7zg+001872
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMUAh7zg+001908
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMUAh7zg+001896
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMGGh7zg+001920
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMSHh7zg+001920
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292759/ICMUAh7zg+001920
Zipping up the final file: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_a3f4_2016092300_20_1018_12292759_0_r1107509013_19.zip
Uploading the final file: upload_file_19.zip
Uploading trickle at timestep: 1727100
12:48:31 (1267): called boinc_finish(0)

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
14 Jun 2024 11:29:12 1549780 22446368 oifs_43r3_bl_a3f4_2016092300_20_1018_12292759_0 1,727,100 40,978 0.0237
13 Jun 2024 22:50:54 1549780 22446368 oifs_43r3_bl_a3f4_2016092300_20_1018_12292759_0 864,000 22,179 0.0257


©2024 cpdn.org