climateprediction.net home page
Task 22449458

Task 22449458

Name oifs_43r3_bl_a1fy_2016092300_20_1018_12290197_1
Workunit 12290197
Created 16 Jun 2024, 14:21:41 UTC
Sent 16 Jun 2024, 14:22:38 UTC
Report deadline 15 Aug 2024, 14:22:38 UTC
Received 17 Jun 2024, 6:51:52 UTC
Server state Over
Outcome Success
Client state Done
Exit status 0 (0x00000000)
Computer ID 1549780
Run time 15 hours 29 min 48 sec
CPU time 15 hours 24 min 13 sec
Validate state Valid
Credit 2,635.55
Device peak FLOPS 5.04 GFLOPS
Application version OpenIFS 43r3 Baroclinic Lifecycle v1.13
x86_64-pc-linux-gnu
Peak working set size 5,016.12 MB
Peak swap size 5,981.13 MB
Peak disk usage 2,024.93 MB
Stderr
<core_client_version>7.22.1</core_client_version>
<![CDATA[
<stderr_txt>
_file_12.zip
  02:24:51 STEP 1249 H= 312:15 +CPU= 28.326
  02:25:19 STEP 1250 H= 312:30 +CPU= 27.330
  02:25:44 STEP 1251 H= 312:45 +CPU= 25.240
  02:26:14 STEP 1252 H= 313:00 +CPU= 29.499
  02:26:42 STEP 1253 H= 313:15 +CPU= 28.741
  02:27:09 STEP 1254 H= 313:30 +CPU= 26.870
  02:27:38 STEP 1255 H= 313:45 +CPU= 28.804
  02:28:08 STEP 1256 H= 314:00 +CPU= 29.865
  02:28:35 STEP 1257 H= 314:15 +CPU= 26.968
  02:29:02 STEP 1258 H= 314:30 +CPU= 26.704
  02:29:31 STEP 1259 H= 314:45 +CPU= 29.417
  02:30:07 STEP 1260 H= 315:00 +CPU= 35.705
Moving to projects directory: /home/ij2/slots/21/ICMGGh7zg+001260
Moving to projects directory: /home/ij2/slots/21/ICMSHh7zg+001260
Moving to projects directory: /home/ij2/slots/21/ICMUAh7zg+001260
  02:30:35 STEP 1261 H= 315:15 +CPU= 27.723
  02:31:04 STEP 1262 H= 315:30 +CPU= 28.946
  02:31:32 STEP 1263 H= 315:45 +CPU= 27.741
  02:32:01 STEP 1264 H= 316:00 +CPU= 29.313
  02:32:30 STEP 1265 H= 316:15 +CPU= 28.890
  02:32:57 STEP 1266 H= 316:30 +CPU= 27.245
  02:33:26 STEP 1267 H= 316:45 +CPU= 28.939
  02:33:56 STEP 1268 H= 317:00 +CPU= 29.922
  02:34:24 STEP 1269 H= 317:15 +CPU= 27.961
  02:34:49 STEP 1270 H= 317:30 +CPU= 24.123
  02:35:17 STEP 1271 H= 317:45 +CPU= 27.807
  02:35:51 STEP 1272 H= 318:00 +CPU= 34.783
Moving to projects directory: /home/ij2/slots/21/ICMGGh7zg+001272
Moving to projects directory: /home/ij2/slots/21/ICMSHh7zg+001272
Moving to projects directory: /home/ij2/slots/21/ICMUAh7zg+001272
  02:36:20 STEP 1273 H= 318:15 +CPU= 28.601
  02:36:48 STEP 1274 H= 318:30 +CPU= 28.389
  02:37:16 STEP 1275 H= 318:45 +CPU= 27.976
  02:37:45 STEP 1276 H= 319:00 +CPU= 28.784
  02:38:14 STEP 1277 H= 319:15 +CPU= 28.929
  02:38:43 STEP 1278 H= 319:30 +CPU= 28.168
  02:39:12 STEP 1279 H= 319:45 +CPU= 28.924
  02:39:40 STEP 1280 H= 320:00 +CPU= 28.058
  02:40:05 STEP 1281 H= 320:15 +CPU= 24.968
  02:40:33 STEP 1282 H= 320:30 +CPU= 28.100
  02:41:02 STEP 1283 H= 320:45 +CPU= 28.807
  02:41:38 STEP 1284 H= 321:00 +CPU= 36.393
Moving to projects directory: /home/ij2/slots/21/ICMGGh7zg+001284
Moving to projects directory: /home/ij2/slots/21/ICMSHh7zg+001284
Moving to projects directory: /home/ij2/slots/21/ICMUAh7zg+001284
  02:42:07 STEP 1285 H= 321:15 +CPU= 29.025
  02:42:32 STEP 1286 H= 321:30 +CPU= 24.611
  02:43:01 STEP 1287 H= 321:45 +CPU= 28.728
  02:43:29 STEP 1288 H= 322:00 +CPU= 28.760
  02:43:58 STEP 1289 H= 322:15 +CPU= 28.378
  02:44:27 STEP 1290 H= 322:30 +CPU= 28.761
  02:44:56 STEP 1291 H= 322:45 +CPU= 29.095
  02:45:25 STEP 1292 H= 323:00 +CPU= 29.212
  02:45:54 STEP 1293 H= 323:15 +CPU= 29.216
  02:46:23 STEP 1294 H= 323:30 +CPU= 28.346
  02:46:49 STEP 1295 H= 323:45 +CPU= 25.838
  02:47:30 STEP 1296 H= 324:00 +CPU= 40.552
Moving to projects directory: /home/ij2/slots/21/ICMGGh7zg+001296
Moving to projects directory: /home/ij2/slots/21/ICMSHh7zg+001296
Moving to projects directory: /home/ij2/slots/21/ICMUAh7zg+001296
  02:47:59 STEP 1297 H= 324:15 +CPU= 29.631
  02:48:29 STEP 1298 H= 324:30 +CPU= 29.355
  02:48:58 STEP 1299 H= 324:45 +CPU= 29.152
  02:49:29 STEP 1300 H= 325:00 +CPU= 30.440
  02:49:58 STEP 1301 H= 325:15 +CPU= 28.889
  02:50:26 STEP 1302 H= 325:30 +CPU= 28.935
  02:50:55 STEP 1303 H= 325:45 +CPU= 28.726
  02:51:23 STEP 1304 H= 326:00 +CPU= 27.362
  02:51:48 STEP 1305 H= 326:15 +CPU= 25.233
  02:52:17 STEP 1306 H= 326:30 +CPU= 29.447
  02:52:46 STEP 1307 H= 326:45 +CPU= 28.539
  02:53:23 STEP 1308 H= 327:00 +CPU= 36.791
Moving to projects directory: /home/ij2/slots/21/ICMGGh7zg+001308
Moving to projects directory: /home/ij2/slots/21/ICMSHh7zg+001308
Moving to projects directory: /home/ij2/slots/21/ICMUAh7zg+001308
  02:53:51 STEP 1309 H= 327:15 +CPU= 28.262
  02:54:19 STEP 1310 H= 327:30 +CPU= 27.938
  02:54:46 STEP 1311 H= 327:45 +CPU= 26.575
  02:55:14 STEP 1312 H= 328:00 +CPU= 28.281
  02:55:43 STEP 1313 H= 328:15 +CPU= 28.756
  02:56:11 STEP 1314 H= 328:30 +CPU= 27.634
  02:56:40 STEP 1315 H= 328:45 +CPU= 28.990
  02:57:10 STEP 1316 H= 329:00 +CPU= 29.985
  02:57:39 STEP 1317 H= 329:15 +CPU= 29.013
  02:58:08 STEP 1318 H= 329:30 +CPU= 29.025
  02:58:34 STEP 1319 H= 329:45 +CPU= 26.576
  02:59:11 STEP 1320 H= 330:00 +CPU= 37.170
Moving to projects directory: /home/ij2/slots/21/ICMGGh7zg+001320
Moving to projects directory: /home/ij2/slots/21/ICMSHh7zg+001320
Moving to projects directory: /home/ij2/slots/21/ICMUAh7zg+001320
  02:59:40 STEP 1321 H= 330:15 +CPU= 28.679
  03:00:10 STEP 1322 H= 330:30 +CPU= 29.282
  03:00:36 STEP 1323 H= 330:45 +CPU= 26.214
  03:01:05 STEP 1324 H= 331:00 +CPU= 29.397
  03:01:33 STEP 1325 H= 331:15 +CPU= 28.162
  03:02:02 STEP 1326 H= 331:30 +CPU= 29.016
  03:02:32 STEP 1327 H= 331:45 +CPU= 29.105
  03:02:59 STEP 1328 H= 332:00 +CPU= 27.305
  03:03:28 STEP 1329 H= 332:15 +CPU= 29.254
  03:03:55 STEP 1330 H= 332:30 +CPU= 26.971
  03:04:24 STEP 1331 H= 332:45 +CPU= 29.035
  03:05:01 STEP 1332 H= 333:00 +CPU= 36.538
Moving to projects directory: /home/ij2/slots/21/ICMGGh7zg+001332
Moving to projects directory: /home/ij2/slots/21/ICMSHh7zg+001332
Moving to projects directory: /home/ij2/slots/21/ICMUAh7zg+001332
  03:05:30 STEP 1333 H= 333:15 +CPU= 29.053
  03:05:59 STEP 1334 H= 333:30 +CPU= 29.058
  03:06:27 STEP 1335 H= 333:45 +CPU= 28.038
  03:06:55 STEP 1336 H= 334:00 +CPU= 27.504
  03:07:21 STEP 1337 H= 334:15 +CPU= 25.866
  03:07:49 STEP 1338 H= 334:30 +CPU= 27.782
  03:08:17 STEP 1339 H= 334:45 +CPU= 28.674
  03:08:46 STEP 1340 H= 335:00 +CPU= 28.850
  03:09:15 STEP 1341 H= 335:15 +CPU= 28.505
  03:09:42 STEP 1342 H= 335:30 +CPU= 27.713
  03:10:11 STEP 1343 H= 335:45 +CPU= 28.699
  03:10:51 STEP 1344 H= 336:00 +CPU= 39.588
Moving to projects directory: /home/ij2/slots/21/ICMGGh7zg+001344
Moving to projects directory: /home/ij2/slots/21/ICMSHh7zg+001344
Moving to projects directory: /home/ij2/slots/21/ICMUAh7zg+001344
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMGGh7zg+001248
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMSHh7zg+001248
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMUAh7zg+001248
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMGGh7zg+001260
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMSHh7zg+001260
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMUAh7zg+001260
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMGGh7zg+001272
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMSHh7zg+001272
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMUAh7zg+001272
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMGGh7zg+001284
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMSHh7zg+001284
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMUAh7zg+001284
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMGGh7zg+001296
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMSHh7zg+001296
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMUAh7zg+001296
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMGGh7zg+001308
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMSHh7zg+001308
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMUAh7zg+001308
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMGGh7zg+001320
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMSHh7zg+001320
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMUAh7zg+001320
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMGGh7zg+001332
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMSHh7zg+001332
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMUAh7zg+001332
Zipping up the intermediate file: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_a1fy_2016092300_20_1018_12290197_1_r509449324_13.zip
Uploading the intermediate file: upload_file_13.zip
  03:11:20 STEP 1345 H= 336:15 +CPU= 29.156
  03:11:49 STEP 1346 H= 336:30 +CPU= 28.918
  03:12:18 STEP 1347 H= 336:45 +CPU= 28.658
  03:12:47 STEP 1348 H= 337:00 +CPU= 28.725
  03:13:16 STEP 1349 H= 337:15 +CPU= 29.471
  03:13:45 STEP 1350 H= 337:30 +CPU= 29.026
  03:14:14 STEP 1351 H= 337:45 +CPU= 29.030
  03:14:45 STEP 1352 H= 338:00 +CPU= 30.152
  03:15:14 STEP 1353 H= 338:15 +CPU= 29.053
  03:15:43 STEP 1354 H= 338:30 +CPU= 29.030
  03:16:12 STEP 1355 H= 338:45 +CPU= 29.480
  03:16:47 STEP 1356 H= 339:00 +CPU= 35.097
Moving to projects directory: /home/ij2/slots/21/ICMGGh7zg+001356
Moving to projects directory: /home/ij2/slots/21/ICMSHh7zg+001356
Moving to projects directory: /home/ij2/slots/21/ICMUAh7zg+001356
  03:17:16 STEP 1357 H= 339:15 +CPU= 28.105
  03:17:42 STEP 1358 H= 339:30 +CPU= 26.522
  03:18:11 STEP 1359 H= 339:45 +CPU= 29.046
  03:18:40 STEP 1360 H= 340:00 +CPU= 28.588
  03:19:09 STEP 1361 H= 340:15 +CPU= 29.094
  03:19:38 STEP 1362 H= 340:30 +CPU= 29.050
  03:20:06 STEP 1363 H= 340:45 +CPU= 27.585
  03:20:31 STEP 1364 H= 341:00 +CPU= 25.452
  03:20:59 STEP 1365 H= 341:15 +CPU= 27.195
  03:21:28 STEP 1366 H= 341:30 +CPU= 29.290
  03:21:57 STEP 1367 H= 341:45 +CPU= 29.206
  03:22:32 STEP 1368 H= 342:00 +CPU= 35.168
Moving to projects directory: /home/ij2/slots/21/ICMGGh7zg+001368
Moving to projects directory: /home/ij2/slots/21/ICMSHh7zg+001368
Moving to projects directory: /home/ij2/slots/21/ICMUAh7zg+001368
  03:23:00 STEP 1369 H= 342:15 +CPU= 27.714
  03:23:30 STEP 1370 H= 342:30 +CPU= 29.293
  03:23:58 STEP 1371 H= 342:45 +CPU= 28.215
  03:24:27 STEP 1372 H= 343:00 +CPU= 29.224
  03:24:56 STEP 1373 H= 343:15 +CPU= 28.764
  03:25:25 STEP 1374 H= 343:30 +CPU= 29.064
  03:25:54 STEP 1375 H= 343:45 +CPU= 29.098
  03:26:23 STEP 1376 H= 344:00 +CPU= 28.397
  03:26:50 STEP 1377 H= 344:15 +CPU= 27.743
  03:27:20 STEP 1378 H= 344:30 +CPU= 29.103
  03:27:48 STEP 1379 H= 344:45 +CPU= 27.901
  03:28:25 STEP 1380 H= 345:00 +CPU= 37.090
Moving to projects directory: /home/ij2/slots/21/ICMGGh7zg+001380
Moving to projects directory: /home/ij2/slots/21/ICMSHh7zg+001380
Moving to projects directory: /home/ij2/slots/21/ICMUAh7zg+001380
  03:28:53 STEP 1381 H= 345:15 +CPU= 28.733
  03:29:21 STEP 1382 H= 345:30 +CPU= 27.160
  03:29:50 STEP 1383 H= 345:45 +CPU= 29.435
  03:30:19 STEP 1384 H= 346:00 +CPU= 28.562
  03:30:45 STEP 1385 H= 346:15 +CPU= 26.503
  03:31:14 STEP 1386 H= 346:30 +CPU= 28.868
  03:31:42 STEP 1387 H= 346:45 +CPU= 28.034
  03:32:10 STEP 1388 H= 347:00 +CPU= 28.116
  03:32:39 STEP 1389 H= 347:15 +CPU= 28.851
  03:33:08 STEP 1390 H= 347:30 +CPU= 28.976
  03:33:36 STEP 1391 H= 347:45 +CPU= 27.549
  03:34:17 STEP 1392 H= 348:00 +CPU= 40.520
Moving to projects directory: /home/ij2/slots/21/ICMGGh7zg+001392
Moving to projects directory: /home/ij2/slots/21/ICMSHh7zg+001392
Moving to projects directory: /home/ij2/slots/21/ICMUAh7zg+001392
  03:34:46 STEP 1393 H= 348:15 +CPU= 28.958
  03:35:10 STEP 1394 H= 348:30 +CPU= 24.211
  03:35:37 STEP 1395 H= 348:45 +CPU= 26.505
  03:36:05 STEP 1396 H= 349:00 +CPU= 28.726
  03:36:35 STEP 1397 H= 349:15 +CPU= 29.308
  03:37:05 STEP 1398 H= 349:30 +CPU= 29.822
  03:37:30 STEP 1399 H= 349:45 +CPU= 25.102
  03:38:00 STEP 1400 H= 350:00 +CPU= 30.018
  03:38:29 STEP 1401 H= 350:15 +CPU= 29.048
  03:38:58 STEP 1402 H= 350:30 +CPU= 29.277
  03:39:28 STEP 1403 H= 350:45 +CPU= 29.475
  03:40:05 STEP 1404 H= 351:00 +CPU= 36.697
Moving to projects directory: /home/ij2/slots/21/ICMGGh7zg+001404
Moving to projects directory: /home/ij2/slots/21/ICMSHh7zg+001404
Moving to projects directory: /home/ij2/slots/21/ICMUAh7zg+001404
  03:40:34 STEP 1405 H= 351:15 +CPU= 28.919
  03:41:02 STEP 1406 H= 351:30 +CPU= 28.314
  03:41:30 STEP 1407 H= 351:45 +CPU= 28.261
  03:42:00 STEP 1408 H= 352:00 +CPU= 29.420
  03:42:28 STEP 1409 H= 352:15 +CPU= 28.574
  03:42:57 STEP 1410 H= 352:30 +CPU= 28.882
  03:43:26 STEP 1411 H= 352:45 +CPU= 28.109
  03:43:54 STEP 1412 H= 353:00 +CPU= 28.669
  03:44:22 STEP 1413 H= 353:15 +CPU= 27.956
  03:44:51 STEP 1414 H= 353:30 +CPU= 28.936
  03:45:20 STEP 1415 H= 353:45 +CPU= 28.997
  03:45:57 STEP 1416 H= 354:00 +CPU= 36.337
Moving to projects directory: /home/ij2/slots/21/ICMGGh7zg+001416
Moving to projects directory: /home/ij2/slots/21/ICMSHh7zg+001416
Moving to projects directory: /home/ij2/slots/21/ICMUAh7zg+001416
  03:46:25 STEP 1417 H= 354:15 +CPU= 28.234
  03:46:52 STEP 1418 H= 354:30 +CPU= 27.049
  03:47:19 STEP 1419 H= 354:45 +CPU= 27.213
  03:47:48 STEP 1420 H= 355:00 +CPU= 28.600
  03:48:17 STEP 1421 H= 355:15 +CPU= 28.992
  03:48:46 STEP 1422 H= 355:30 +CPU= 28.749
  03:49:13 STEP 1423 H= 355:45 +CPU= 27.516
  03:49:43 STEP 1424 H= 356:00 +CPU= 29.526
  03:50:10 STEP 1425 H= 356:15 +CPU= 27.013
  03:50:37 STEP 1426 H= 356:30 +CPU= 27.161
  03:51:04 STEP 1427 H= 356:45 +CPU= 27.207
  03:51:40 STEP 1428 H= 357:00 +CPU= 35.538
Moving to projects directory: /home/ij2/slots/21/ICMGGh7zg+001428
Moving to projects directory: /home/ij2/slots/21/ICMSHh7zg+001428
Moving to projects directory: /home/ij2/slots/21/ICMUAh7zg+001428
  03:52:07 STEP 1429 H= 357:15 +CPU= 27.219
  03:52:36 STEP 1430 H= 357:30 +CPU= 28.920
  03:53:05 STEP 1431 H= 357:45 +CPU= 28.650
  03:53:33 STEP 1432 H= 358:00 +CPU= 27.648
  03:54:02 STEP 1433 H= 358:15 +CPU= 28.811
  03:54:30 STEP 1434 H= 358:30 +CPU= 28.156
  03:54:59 STEP 1435 H= 358:45 +CPU= 29.435
  03:55:29 STEP 1436 H= 359:00 +CPU= 29.764
  03:55:58 STEP 1437 H= 359:15 +CPU= 28.925
  03:56:27 STEP 1438 H= 359:30 +CPU= 29.110
  03:56:57 STEP 1439 H= 359:45 +CPU= 29.589
  03:57:37 STEP 1440 H= 360:00 +CPU= 39.359
Moving to projects directory: /home/ij2/slots/21/ICMGGh7zg+001440
Moving to projects directory: /home/ij2/slots/21/ICMSHh7zg+001440
Moving to projects directory: /home/ij2/slots/21/ICMUAh7zg+001440
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMGGh7zg+001344
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMSHh7zg+001344
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMUAh7zg+001344
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMGGh7zg+001356
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMSHh7zg+001356
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMUAh7zg+001356
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMGGh7zg+001368
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMSHh7zg+001368
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMUAh7zg+001368
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMGGh7zg+001380
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMSHh7zg+001380
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMUAh7zg+001380
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMGGh7zg+001392
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMSHh7zg+001392
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMUAh7zg+001392
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMGGh7zg+001404
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMSHh7zg+001404
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMUAh7zg+001404
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMGGh7zg+001416
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMSHh7zg+001416
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMUAh7zg+001416
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMGGh7zg+001428
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMSHh7zg+001428
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMUAh7zg+001428
Zipping up the intermediate file: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_a1fy_2016092300_20_1018_12290197_1_r509449324_14.zip
Uploading the intermediate file: upload_file_14.zip
  03:58:06 STEP 1441 H= 360:15 +CPU= 29.166
  03:58:34 STEP 1442 H= 360:30 +CPU= 28.175
  03:59:02 STEP 1443 H= 360:45 +CPU= 27.999
  03:59:32 STEP 1444 H= 361:00 +CPU= 29.862
  04:00:01 STEP 1445 H= 361:15 +CPU= 29.200
  04:00:30 STEP 1446 H= 361:30 +CPU= 28.375
  04:00:58 STEP 1447 H= 361:45 +CPU= 28.700
  04:01:28 STEP 1448 H= 362:00 +CPU= 29.744
  04:01:56 STEP 1449 H= 362:15 +CPU= 27.214
  04:02:24 STEP 1450 H= 362:30 +CPU= 28.813
  04:02:53 STEP 1451 H= 362:45 +CPU= 28.334
  04:03:28 STEP 1452 H= 363:00 +CPU= 35.254
Moving to projects directory: /home/ij2/slots/21/ICMGGh7zg+001452
Moving to projects directory: /home/ij2/slots/21/ICMSHh7zg+001452
Moving to projects directory: /home/ij2/slots/21/ICMUAh7zg+001452
  04:03:53 STEP 1453 H= 363:15 +CPU= 24.843
  04:04:22 STEP 1454 H= 363:30 +CPU= 28.725
  04:04:50 STEP 1455 H= 363:45 +CPU= 28.126
  04:05:18 STEP 1456 H= 364:00 +CPU= 28.523
  04:05:47 STEP 1457 H= 364:15 +CPU= 28.958
  04:06:16 STEP 1458 H= 364:30 +CPU= 28.901
  04:06:45 STEP 1459 H= 364:45 +CPU= 28.917
  04:07:15 STEP 1460 H= 365:00 +CPU= 29.715
  04:07:43 STEP 1461 H= 365:15 +CPU= 28.036
  04:08:12 STEP 1462 H= 365:30 +CPU= 28.308
  04:08:38 STEP 1463 H= 365:45 +CPU= 26.324
  04:09:13 STEP 1464 H= 366:00 +CPU= 34.740
Moving to projects directory: /home/ij2/slots/21/ICMGGh7zg+001464
Moving to projects directory: /home/ij2/slots/21/ICMSHh7zg+001464
Moving to projects directory: /home/ij2/slots/21/ICMUAh7zg+001464
  04:09:42 STEP 1465 H= 366:15 +CPU= 28.896
  04:10:10 STEP 1466 H= 366:30 +CPU= 28.508
  04:10:39 STEP 1467 H= 366:45 +CPU= 28.659
  04:11:09 STEP 1468 H= 367:00 +CPU= 29.715
  04:11:38 STEP 1469 H= 367:15 +CPU= 28.962
  04:12:07 STEP 1470 H= 367:30 +CPU= 28.991
  04:12:36 STEP 1471 H= 367:45 +CPU= 29.358
  04:13:06 STEP 1472 H= 368:00 +CPU= 29.622
  04:13:34 STEP 1473 H= 368:15 +CPU= 27.939
  04:14:03 STEP 1474 H= 368:30 +CPU= 28.869
  04:14:32 STEP 1475 H= 368:45 +CPU= 29.186
  04:15:06 STEP 1476 H= 369:00 +CPU= 34.305
Moving to projects directory: /home/ij2/slots/21/ICMGGh7zg+001476
Moving to projects directory: /home/ij2/slots/21/ICMSHh7zg+001476
Moving to projects directory: /home/ij2/slots/21/ICMUAh7zg+001476
  04:15:35 STEP 1477 H= 369:15 +CPU= 28.009
  04:16:03 STEP 1478 H= 369:30 +CPU= 28.008
  04:16:29 STEP 1479 H= 369:45 +CPU= 26.439
  04:16:59 STEP 1480 H= 370:00 +CPU= 29.902
  04:17:27 STEP 1481 H= 370:15 +CPU= 27.929
  04:17:56 STEP 1482 H= 370:30 +CPU= 28.978
  04:18:25 STEP 1483 H= 370:45 +CPU= 29.071
  04:18:55 STEP 1484 H= 371:00 +CPU= 29.886
  04:19:24 STEP 1485 H= 371:15 +CPU= 28.766
  04:19:53 STEP 1486 H= 371:30 +CPU= 29.208
  04:20:21 STEP 1487 H= 371:45 +CPU= 27.626
  04:21:02 STEP 1488 H= 372:00 +CPU= 40.149
Moving to projects directory: /home/ij2/slots/21/ICMGGh7zg+001488
Moving to projects directory: /home/ij2/slots/21/ICMSHh7zg+001488
Moving to projects directory: /home/ij2/slots/21/ICMUAh7zg+001488
  04:21:31 STEP 1489 H= 372:15 +CPU= 28.968
  04:21:59 STEP 1490 H= 372:30 +CPU= 28.865
  04:22:26 STEP 1491 H= 372:45 +CPU= 26.689
  04:22:54 STEP 1492 H= 373:00 +CPU= 28.057
  04:23:23 STEP 1493 H= 373:15 +CPU= 29.027
  04:23:53 STEP 1494 H= 373:30 +CPU= 28.986
  04:24:21 STEP 1495 H= 373:45 +CPU= 28.908
  04:24:51 STEP 1496 H= 374:00 +CPU= 29.067
  04:25:20 STEP 1497 H= 374:15 +CPU= 28.905
  04:25:48 STEP 1498 H= 374:30 +CPU= 28.773
  04:26:18 STEP 1499 H= 374:45 +CPU= 29.490
  04:26:55 STEP 1500 H= 375:00 +CPU= 36.826
Moving to projects directory: /home/ij2/slots/21/ICMGGh7zg+001500
Moving to projects directory: /home/ij2/slots/21/ICMSHh7zg+001500
Moving to projects directory: /home/ij2/slots/21/ICMUAh7zg+001500
  04:27:23 STEP 1501 H= 375:15 +CPU= 27.845
  04:27:51 STEP 1502 H= 375:30 +CPU= 28.550
  04:28:20 STEP 1503 H= 375:45 +CPU= 28.363
  04:28:48 STEP 1504 H= 376:00 +CPU= 28.480
  04:29:17 STEP 1505 H= 376:15 +CPU= 28.611
  04:29:41 STEP 1506 H= 376:30 +CPU= 24.392
  04:30:05 STEP 1507 H= 376:45 +CPU= 23.419
  04:30:31 STEP 1508 H= 377:00 +CPU= 25.735
  04:31:00 STEP 1509 H= 377:15 +CPU= 29.245
  04:31:29 STEP 1510 H= 377:30 +CPU= 29.496
  04:31:59 STEP 1511 H= 377:45 +CPU= 29.151
  04:32:35 STEP 1512 H= 378:00 +CPU= 36.193
Moving to projects directory: /home/ij2/slots/21/ICMGGh7zg+001512
Moving to projects directory: /home/ij2/slots/21/ICMSHh7zg+001512
Moving to projects directory: /home/ij2/slots/21/ICMUAh7zg+001512
  04:33:02 STEP 1513 H= 378:15 +CPU= 27.260
  04:33:31 STEP 1514 H= 378:30 +CPU= 28.927
  04:33:59 STEP 1515 H= 378:45 +CPU= 27.530
  04:34:27 STEP 1516 H= 379:00 +CPU= 28.494
  04:34:55 STEP 1517 H= 379:15 +CPU= 28.048
  04:35:23 STEP 1518 H= 379:30 +CPU= 27.957
  04:35:50 STEP 1519 H= 379:45 +CPU= 26.045
  04:36:19 STEP 1520 H= 380:00 +CPU= 28.926
  04:36:48 STEP 1521 H= 380:15 +CPU= 28.920
  04:37:15 STEP 1522 H= 380:30 +CPU= 27.069
  04:37:42 STEP 1523 H= 380:45 +CPU= 27.748
  04:38:16 STEP 1524 H= 381:00 +CPU= 33.746
Moving to projects directory: /home/ij2/slots/21/ICMGGh7zg+001524
Moving to projects directory: /home/ij2/slots/21/ICMSHh7zg+001524
Moving to projects directory: /home/ij2/slots/21/ICMUAh7zg+001524
  04:38:44 STEP 1525 H= 381:15 +CPU= 28.109
  04:39:14 STEP 1526 H= 381:30 +CPU= 29.199
  04:39:43 STEP 1527 H= 381:45 +CPU= 28.977
  04:40:10 STEP 1528 H= 382:00 +CPU= 27.645
  04:40:39 STEP 1529 H= 382:15 +CPU= 28.854
  04:41:07 STEP 1530 H= 382:30 +CPU= 27.383
  04:41:36 STEP 1531 H= 382:45 +CPU= 28.929
  04:42:06 STEP 1532 H= 383:00 +CPU= 30.135
  04:42:34 STEP 1533 H= 383:15 +CPU= 28.431
  04:42:59 STEP 1534 H= 383:30 +CPU= 24.662
  04:43:28 STEP 1535 H= 383:45 +CPU= 28.687
  04:44:08 STEP 1536 H= 384:00 +CPU= 39.828
Moving to projects directory: /home/ij2/slots/21/ICMGGh7zg+001536
Moving to projects directory: /home/ij2/slots/21/ICMSHh7zg+001536
Moving to projects directory: /home/ij2/slots/21/ICMUAh7zg+001536
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMGGh7zg+001440
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMSHh7zg+001440
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMUAh7zg+001440
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMGGh7zg+001452
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMSHh7zg+001452
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMUAh7zg+001452
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMGGh7zg+001464
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMSHh7zg+001464
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMUAh7zg+001464
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMGGh7zg+001476
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMSHh7zg+001476
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMUAh7zg+001476
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMGGh7zg+001488
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMSHh7zg+001488
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMUAh7zg+001488
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMGGh7zg+001500
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMSHh7zg+001500
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMUAh7zg+001500
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMGGh7zg+001512
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMSHh7zg+001512
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMUAh7zg+001512
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMGGh7zg+001524
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMSHh7zg+001524
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMUAh7zg+001524
Zipping up the intermediate file: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_a1fy_2016092300_20_1018_12290197_1_r509449324_15.zip
Uploading the intermediate file: upload_file_15.zip
  04:44:35 STEP 1537 H= 384:15 +CPU= 27.448
  04:45:04 STEP 1538 H= 384:30 +CPU= 28.865
  04:45:34 STEP 1539 H= 384:45 +CPU= 29.293
  04:46:04 STEP 1540 H= 385:00 +CPU= 29.994
  04:46:32 STEP 1541 H= 385:15 +CPU= 28.333
  04:47:02 STEP 1542 H= 385:30 +CPU= 29.435
  04:47:30 STEP 1543 H= 385:45 +CPU= 28.002
  04:47:59 STEP 1544 H= 386:00 +CPU= 28.930
  04:48:25 STEP 1545 H= 386:15 +CPU= 26.553
  04:48:53 STEP 1546 H= 386:30 +CPU= 27.726
  04:49:19 STEP 1547 H= 386:45 +CPU= 25.527
  04:49:56 STEP 1548 H= 387:00 +CPU= 37.310
Moving to projects directory: /home/ij2/slots/21/ICMGGh7zg+001548
Moving to projects directory: /home/ij2/slots/21/ICMSHh7zg+001548
Moving to projects directory: /home/ij2/slots/21/ICMUAh7zg+001548
  04:50:25 STEP 1549 H= 387:15 +CPU= 29.151
  04:50:55 STEP 1550 H= 387:30 +CPU= 29.333
  04:51:21 STEP 1551 H= 387:45 +CPU= 26.294
  04:51:50 STEP 1552 H= 388:00 +CPU= 29.221
  04:52:19 STEP 1553 H= 388:15 +CPU= 28.927
  04:52:48 STEP 1554 H= 388:30 +CPU= 28.911
  04:53:17 STEP 1555 H= 388:45 +CPU= 28.887
  04:53:47 STEP 1556 H= 389:00 +CPU= 29.586
  04:54:16 STEP 1557 H= 389:15 +CPU= 28.609
  04:54:43 STEP 1558 H= 389:30 +CPU= 27.612
  04:55:12 STEP 1559 H= 389:45 +CPU= 28.881
  04:55:48 STEP 1560 H= 390:00 +CPU= 36.045
Moving to projects directory: /home/ij2/slots/21/ICMGGh7zg+001560
Moving to projects directory: /home/ij2/slots/21/ICMSHh7zg+001560
Moving to projects directory: /home/ij2/slots/21/ICMUAh7zg+001560
  04:56:17 STEP 1561 H= 390:15 +CPU= 29.058
  04:56:45 STEP 1562 H= 390:30 +CPU= 27.810
  04:57:10 STEP 1563 H= 390:45 +CPU= 24.947
  04:57:37 STEP 1564 H= 391:00 +CPU= 27.015
  04:58:05 STEP 1565 H= 391:15 +CPU= 27.958
  04:58:33 STEP 1566 H= 391:30 +CPU= 27.271
  04:58:59 STEP 1567 H= 391:45 +CPU= 26.166
  04:59:29 STEP 1568 H= 392:00 +CPU= 29.732
  04:59:58 STEP 1569 H= 392:15 +CPU= 29.020
  05:00:27 STEP 1570 H= 392:30 +CPU= 29.075
  05:00:56 STEP 1571 H= 392:45 +CPU= 29.032
  05:01:33 STEP 1572 H= 393:00 +CPU= 36.959
Moving to projects directory: /home/ij2/slots/21/ICMGGh7zg+001572
Moving to projects directory: /home/ij2/slots/21/ICMSHh7zg+001572
Moving to projects directory: /home/ij2/slots/21/ICMUAh7zg+001572
  05:02:02 STEP 1573 H= 393:15 +CPU= 29.151
  05:02:32 STEP 1574 H= 393:30 +CPU= 29.364
  05:02:58 STEP 1575 H= 393:45 +CPU= 26.468
  05:03:26 STEP 1576 H= 394:00 +CPU= 28.205
  05:03:56 STEP 1577 H= 394:15 +CPU= 29.364
  05:04:25 STEP 1578 H= 394:30 +CPU= 29.023
  05:04:52 STEP 1579 H= 394:45 +CPU= 27.516
  05:05:21 STEP 1580 H= 395:00 +CPU= 28.455
  05:05:50 STEP 1581 H= 395:15 +CPU= 29.194
  05:06:20 STEP 1582 H= 395:30 +CPU= 29.361
  05:06:48 STEP 1583 H= 395:45 +CPU= 28.644
  05:07:28 STEP 1584 H= 396:00 +CPU= 39.003
Moving to projects directory: /home/ij2/slots/21/ICMGGh7zg+001584
Moving to projects directory: /home/ij2/slots/21/ICMSHh7zg+001584
Moving to projects directory: /home/ij2/slots/21/ICMUAh7zg+001584
  05:07:55 STEP 1585 H= 396:15 +CPU= 27.043
  05:08:21 STEP 1586 H= 396:30 +CPU= 25.727
  05:08:47 STEP 1587 H= 396:45 +CPU= 26.328
  05:09:15 STEP 1588 H= 397:00 +CPU= 27.909
  05:09:43 STEP 1589 H= 397:15 +CPU= 28.461
  05:10:12 STEP 1590 H= 397:30 +CPU= 28.862
  05:10:41 STEP 1591 H= 397:45 +CPU= 29.002
  05:11:11 STEP 1592 H= 398:00 +CPU= 29.881
  05:11:40 STEP 1593 H= 398:15 +CPU= 28.888
  05:12:09 STEP 1594 H= 398:30 +CPU= 28.969
  05:12:38 STEP 1595 H= 398:45 +CPU= 28.984
  05:13:13 STEP 1596 H= 399:00 +CPU= 34.818
Moving to projects directory: /home/ij2/slots/21/ICMGGh7zg+001596
Moving to projects directory: /home/ij2/slots/21/ICMSHh7zg+001596
Moving to projects directory: /home/ij2/slots/21/ICMUAh7zg+001596
  05:13:43 STEP 1597 H= 399:15 +CPU= 29.147
  05:14:11 STEP 1598 H= 399:30 +CPU= 28.523
  05:14:37 STEP 1599 H= 399:45 +CPU= 26.222
  05:15:07 STEP 1600 H= 400:00 +CPU= 29.185
  05:15:36 STEP 1601 H= 400:15 +CPU= 28.942
  05:16:04 STEP 1602 H= 400:30 +CPU= 28.147
  05:16:31 STEP 1603 H= 400:45 +CPU= 27.070
  05:17:00 STEP 1604 H= 401:00 +CPU= 29.054
  05:17:29 STEP 1605 H= 401:15 +CPU= 28.843
  05:17:58 STEP 1606 H= 401:30 +CPU= 28.939
  05:18:25 STEP 1607 H= 401:45 +CPU= 26.581
  05:19:01 STEP 1608 H= 402:00 +CPU= 36.179
Moving to projects directory: /home/ij2/slots/21/ICMGGh7zg+001608
Moving to projects directory: /home/ij2/slots/21/ICMSHh7zg+001608
Moving to projects directory: /home/ij2/slots/21/ICMUAh7zg+001608
  05:19:28 STEP 1609 H= 402:15 +CPU= 27.586
  05:19:55 STEP 1610 H= 402:30 +CPU= 27.055
  05:20:23 STEP 1611 H= 402:45 +CPU= 26.974
  05:20:53 STEP 1612 H= 403:00 +CPU= 29.991
  05:21:21 STEP 1613 H= 403:15 +CPU= 28.844
  05:21:50 STEP 1614 H= 403:30 +CPU= 28.641
  05:22:19 STEP 1615 H= 403:45 +CPU= 28.425
  05:22:45 STEP 1616 H= 404:00 +CPU= 26.720
  05:23:15 STEP 1617 H= 404:15 +CPU= 29.570
  05:23:45 STEP 1618 H= 404:30 +CPU= 30.081
  05:24:14 STEP 1619 H= 404:45 +CPU= 28.590
  05:24:49 STEP 1620 H= 405:00 +CPU= 34.948
Moving to projects directory: /home/ij2/slots/21/ICMGGh7zg+001620
Moving to projects directory: /home/ij2/slots/21/ICMSHh7zg+001620
Moving to projects directory: /home/ij2/slots/21/ICMUAh7zg+001620
  05:25:18 STEP 1621 H= 405:15 +CPU= 29.535
  05:25:47 STEP 1622 H= 405:30 +CPU= 28.960
  05:26:15 STEP 1623 H= 405:45 +CPU= 28.119
  05:26:45 STEP 1624 H= 406:00 +CPU= 29.649
  05:27:14 STEP 1625 H= 406:15 +CPU= 28.767
  05:27:43 STEP 1626 H= 406:30 +CPU= 28.892
  05:28:12 STEP 1627 H= 406:45 +CPU= 28.763
  05:28:40 STEP 1628 H= 407:00 +CPU= 28.710
  05:29:09 STEP 1629 H= 407:15 +CPU= 28.790
  05:29:38 STEP 1630 H= 407:30 +CPU= 28.885
  05:30:07 STEP 1631 H= 407:45 +CPU= 29.172
  05:30:48 STEP 1632 H= 408:00 +CPU= 40.017
Moving to projects directory: /home/ij2/slots/21/ICMGGh7zg+001632
Moving to projects directory: /home/ij2/slots/21/ICMSHh7zg+001632
Moving to projects directory: /home/ij2/slots/21/ICMUAh7zg+001632
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMGGh7zg+001536
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMSHh7zg+001536
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMUAh7zg+001536
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMGGh7zg+001548
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMSHh7zg+001548
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMUAh7zg+001548
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMGGh7zg+001560
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMSHh7zg+001560
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMUAh7zg+001560
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMGGh7zg+001572
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMSHh7zg+001572
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMUAh7zg+001572
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMGGh7zg+001584
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMSHh7zg+001584
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMUAh7zg+001584
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMGGh7zg+001596
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMSHh7zg+001596
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMUAh7zg+001596
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMGGh7zg+001608
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMSHh7zg+001608
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMUAh7zg+001608
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMGGh7zg+001620
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMSHh7zg+001620
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMUAh7zg+001620
Zipping up the intermediate file: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_a1fy_2016092300_20_1018_12290197_1_r509449324_16.zip
Uploading the intermediate file: upload_file_16.zip
  05:31:15 STEP 1633 H= 408:15 +CPU= 27.632
  05:31:43 STEP 1634 H= 408:30 +CPU= 27.909
  05:32:13 STEP 1635 H= 408:45 +CPU= 29.088
  05:32:41 STEP 1636 H= 409:00 +CPU= 28.330
  05:33:09 STEP 1637 H= 409:15 +CPU= 28.099
  05:33:37 STEP 1638 H= 409:30 +CPU= 27.691
  05:34:04 STEP 1639 H= 409:45 +CPU= 27.638
  05:34:34 STEP 1640 H= 410:00 +CPU= 29.613
  05:35:03 STEP 1641 H= 410:15 +CPU= 29.149
  05:35:31 STEP 1642 H= 410:30 +CPU= 27.579
  05:35:59 STEP 1643 H= 410:45 +CPU= 27.748
  05:36:35 STEP 1644 H= 411:00 +CPU= 36.335
Moving to projects directory: /home/ij2/slots/21/ICMGGh7zg+001644
Moving to projects directory: /home/ij2/slots/21/ICMSHh7zg+001644
Moving to projects directory: /home/ij2/slots/21/ICMUAh7zg+001644
  05:37:04 STEP 1645 H= 411:15 +CPU= 29.206
  05:37:34 STEP 1646 H= 411:30 +CPU= 29.420
  05:38:03 STEP 1647 H= 411:45 +CPU= 29.127
  05:38:32 STEP 1648 H= 412:00 +CPU= 29.314
  05:39:01 STEP 1649 H= 412:15 +CPU= 29.080
  05:39:30 STEP 1650 H= 412:30 +CPU= 28.899
  05:39:59 STEP 1651 H= 412:45 +CPU= 28.132
  05:40:29 STEP 1652 H= 413:00 +CPU= 29.977
  05:40:58 STEP 1653 H= 413:15 +CPU= 29.144
  05:41:27 STEP 1654 H= 413:30 +CPU= 29.300
  05:41:56 STEP 1655 H= 413:45 +CPU= 28.699
  05:42:33 STEP 1656 H= 414:00 +CPU= 37.228
Moving to projects directory: /home/ij2/slots/21/ICMGGh7zg+001656
Moving to projects directory: /home/ij2/slots/21/ICMSHh7zg+001656
Moving to projects directory: /home/ij2/slots/21/ICMUAh7zg+001656
  05:43:02 STEP 1657 H= 414:15 +CPU= 28.362
  05:43:31 STEP 1658 H= 414:30 +CPU= 28.931
  05:43:58 STEP 1659 H= 414:45 +CPU= 27.481
  05:44:28 STEP 1660 H= 415:00 +CPU= 29.842
  05:44:56 STEP 1661 H= 415:15 +CPU= 27.629
  05:45:25 STEP 1662 H= 415:30 +CPU= 29.026
  05:45:53 STEP 1663 H= 415:45 +CPU= 27.809
  05:46:22 STEP 1664 H= 416:00 +CPU= 29.049
  05:46:50 STEP 1665 H= 416:15 +CPU= 28.507
  05:47:19 STEP 1666 H= 416:30 +CPU= 28.984
  05:47:48 STEP 1667 H= 416:45 +CPU= 28.453
  05:48:25 STEP 1668 H= 417:00 +CPU= 37.367
Moving to projects directory: /home/ij2/slots/21/ICMGGh7zg+001668
Moving to projects directory: /home/ij2/slots/21/ICMSHh7zg+001668
Moving to projects directory: /home/ij2/slots/21/ICMUAh7zg+001668
  05:48:53 STEP 1669 H= 417:15 +CPU= 27.702
  05:49:22 STEP 1670 H= 417:30 +CPU= 29.010
  05:49:51 STEP 1671 H= 417:45 +CPU= 28.876
  05:50:17 STEP 1672 H= 418:00 +CPU= 25.913
  05:50:45 STEP 1673 H= 418:15 +CPU= 28.360
  05:51:13 STEP 1674 H= 418:30 +CPU= 27.783
  05:51:42 STEP 1675 H= 418:45 +CPU= 28.940
  05:52:12 STEP 1676 H= 419:00 +CPU= 30.049
  05:52:41 STEP 1677 H= 419:15 +CPU= 28.455
  05:53:09 STEP 1678 H= 419:30 +CPU= 27.718
  05:53:37 STEP 1679 H= 419:45 +CPU= 28.878
  05:54:18 STEP 1680 H= 420:00 +CPU= 39.826
Moving to projects directory: /home/ij2/slots/21/ICMGGh7zg+001680
Moving to projects directory: /home/ij2/slots/21/ICMSHh7zg+001680
Moving to projects directory: /home/ij2/slots/21/ICMUAh7zg+001680
  05:54:47 STEP 1681 H= 420:15 +CPU= 29.507
  05:55:17 STEP 1682 H= 420:30 +CPU= 29.092
  05:55:45 STEP 1683 H= 420:45 +CPU= 28.444
  05:56:10 STEP 1684 H= 421:00 +CPU= 24.680
  05:56:38 STEP 1685 H= 421:15 +CPU= 27.853
  05:57:08 STEP 1686 H= 421:30 +CPU= 29.956
  05:57:37 STEP 1687 H= 421:45 +CPU= 29.648
  05:58:07 STEP 1688 H= 422:00 +CPU= 29.755
  05:58:36 STEP 1689 H= 422:15 +CPU= 29.096
  05:59:05 STEP 1690 H= 422:30 +CPU= 29.040
  05:59:34 STEP 1691 H= 422:45 +CPU= 28.866
  06:00:09 STEP 1692 H= 423:00 +CPU= 34.486
Moving to projects directory: /home/ij2/slots/21/ICMGGh7zg+001692
Moving to projects directory: /home/ij2/slots/21/ICMSHh7zg+001692
Moving to projects directory: /home/ij2/slots/21/ICMUAh7zg+001692
  06:00:36 STEP 1693 H= 423:15 +CPU= 26.779
  06:01:03 STEP 1694 H= 423:30 +CPU= 26.897
  06:01:26 STEP 1695 H= 423:45 +CPU= 23.590
  06:01:54 STEP 1696 H= 424:00 +CPU= 27.422
  06:02:21 STEP 1697 H= 424:15 +CPU= 27.398
  06:02:49 STEP 1698 H= 424:30 +CPU= 27.505
  06:03:17 STEP 1699 H= 424:45 +CPU= 28.680
  06:03:44 STEP 1700 H= 425:00 +CPU= 26.850
  06:04:07 STEP 1701 H= 425:15 +CPU= 22.350
  06:04:34 STEP 1702 H= 425:30 +CPU= 26.744
  06:05:00 STEP 1703 H= 425:45 +CPU= 26.740
  06:05:37 STEP 1704 H= 426:00 +CPU= 36.041
Moving to projects directory: /home/ij2/slots/21/ICMGGh7zg+001704
Moving to projects directory: /home/ij2/slots/21/ICMSHh7zg+001704
Moving to projects directory: /home/ij2/slots/21/ICMUAh7zg+001704
  06:06:05 STEP 1705 H= 426:15 +CPU= 27.876
  06:06:34 STEP 1706 H= 426:30 +CPU= 28.881
  06:07:02 STEP 1707 H= 426:45 +CPU= 28.949
  06:07:32 STEP 1708 H= 427:00 +CPU= 29.765
  06:07:58 STEP 1709 H= 427:15 +CPU= 26.006
  06:08:27 STEP 1710 H= 427:30 +CPU= 28.573
  06:08:54 STEP 1711 H= 427:45 +CPU= 26.774
  06:09:24 STEP 1712 H= 428:00 +CPU= 29.664
  06:09:52 STEP 1713 H= 428:15 +CPU= 27.957
  06:10:18 STEP 1714 H= 428:30 +CPU= 26.765
  06:10:44 STEP 1715 H= 428:45 +CPU= 26.011
  06:11:21 STEP 1716 H= 429:00 +CPU= 36.260
Moving to projects directory: /home/ij2/slots/21/ICMGGh7zg+001716
Moving to projects directory: /home/ij2/slots/21/ICMSHh7zg+001716
Moving to projects directory: /home/ij2/slots/21/ICMUAh7zg+001716
  06:11:50 STEP 1717 H= 429:15 +CPU= 28.704
  06:12:19 STEP 1718 H= 429:30 +CPU= 29.251
  06:12:44 STEP 1719 H= 429:45 +CPU= 25.530
  06:13:12 STEP 1720 H= 430:00 +CPU= 27.954
  06:13:38 STEP 1721 H= 430:15 +CPU= 25.730
  06:14:07 STEP 1722 H= 430:30 +CPU= 29.118
  06:14:36 STEP 1723 H= 430:45 +CPU= 28.927
  06:15:06 STEP 1724 H= 431:00 +CPU= 29.461
  06:15:34 STEP 1725 H= 431:15 +CPU= 28.460
  06:16:02 STEP 1726 H= 431:30 +CPU= 27.972
  06:16:29 STEP 1727 H= 431:45 +CPU= 26.855
  06:17:10 STEP 1728 H= 432:00 +CPU= 40.505
Moving to projects directory: /home/ij2/slots/21/ICMGGh7zg+001728
Moving to projects directory: /home/ij2/slots/21/ICMSHh7zg+001728
Moving to projects directory: /home/ij2/slots/21/ICMUAh7zg+001728
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMGGh7zg+001632
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMSHh7zg+001632
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMUAh7zg+001632
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMGGh7zg+001644
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMSHh7zg+001644
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMUAh7zg+001644
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMGGh7zg+001656
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMSHh7zg+001656
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMUAh7zg+001656
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMGGh7zg+001668
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMSHh7zg+001668
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMUAh7zg+001668
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMGGh7zg+001680
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMSHh7zg+001680
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMUAh7zg+001680
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMGGh7zg+001692
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMSHh7zg+001692
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMUAh7zg+001692
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMGGh7zg+001704
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMSHh7zg+001704
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMUAh7zg+001704
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMGGh7zg+001716
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMSHh7zg+001716
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMUAh7zg+001716
Zipping up the intermediate file: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_a1fy_2016092300_20_1018_12290197_1_r509449324_17.zip
Uploading the intermediate file: upload_file_17.zip
  06:17:39 STEP 1729 H= 432:15 +CPU= 28.824
  06:18:09 STEP 1730 H= 432:30 +CPU= 29.552
  06:18:38 STEP 1731 H= 432:45 +CPU= 28.913
  06:19:08 STEP 1732 H= 433:00 +CPU= 29.802
  06:19:36 STEP 1733 H= 433:15 +CPU= 27.976
  06:20:03 STEP 1734 H= 433:30 +CPU= 27.598
  06:20:30 STEP 1735 H= 433:45 +CPU= 26.515
  06:20:58 STEP 1736 H= 434:00 +CPU= 28.091
  06:21:27 STEP 1737 H= 434:15 +CPU= 28.699
  06:21:54 STEP 1738 H= 434:30 +CPU= 27.288
  06:22:22 STEP 1739 H= 434:45 +CPU= 27.679
  06:22:58 STEP 1740 H= 435:00 +CPU= 35.933
Moving to projects directory: /home/ij2/slots/21/ICMGGh7zg+001740
Moving to projects directory: /home/ij2/slots/21/ICMSHh7zg+001740
Moving to projects directory: /home/ij2/slots/21/ICMUAh7zg+001740
  06:23:28 STEP 1741 H= 435:15 +CPU= 29.742
  06:23:56 STEP 1742 H= 435:30 +CPU= 27.956
  06:24:20 STEP 1743 H= 435:45 +CPU= 24.455
  06:24:48 STEP 1744 H= 436:00 +CPU= 27.989
  06:25:16 STEP 1745 H= 436:15 +CPU= 28.290
  06:25:45 STEP 1746 H= 436:30 +CPU= 28.978
  06:26:15 STEP 1747 H= 436:45 +CPU= 29.330
  06:26:44 STEP 1748 H= 437:00 +CPU= 29.436
  06:27:14 STEP 1749 H= 437:15 +CPU= 29.311
  06:27:42 STEP 1750 H= 437:30 +CPU= 28.238
  06:28:11 STEP 1751 H= 437:45 +CPU= 28.570
  06:28:45 STEP 1752 H= 438:00 +CPU= 34.455
Moving to projects directory: /home/ij2/slots/21/ICMGGh7zg+001752
Moving to projects directory: /home/ij2/slots/21/ICMSHh7zg+001752
Moving to projects directory: /home/ij2/slots/21/ICMUAh7zg+001752
  06:29:13 STEP 1753 H= 438:15 +CPU= 27.624
  06:29:42 STEP 1754 H= 438:30 +CPU= 28.964
  06:30:05 STEP 1755 H= 438:45 +CPU= 23.044
  06:30:34 STEP 1756 H= 439:00 +CPU= 28.521
  06:31:02 STEP 1757 H= 439:15 +CPU= 28.806
  06:31:31 STEP 1758 H= 439:30 +CPU= 28.334
  06:31:59 STEP 1759 H= 439:45 +CPU= 28.350
  06:32:28 STEP 1760 H= 440:00 +CPU= 28.635
  06:32:55 STEP 1761 H= 440:15 +CPU= 27.485
  06:33:24 STEP 1762 H= 440:30 +CPU= 28.349
  06:33:50 STEP 1763 H= 440:45 +CPU= 26.170
  06:34:26 STEP 1764 H= 441:00 +CPU= 35.436
Moving to projects directory: /home/ij2/slots/21/ICMGGh7zg+001764
Moving to projects directory: /home/ij2/slots/21/ICMSHh7zg+001764
Moving to projects directory: /home/ij2/slots/21/ICMUAh7zg+001764
  06:34:53 STEP 1765 H= 441:15 +CPU= 27.294
  06:35:19 STEP 1766 H= 441:30 +CPU= 26.113
  06:35:42 STEP 1767 H= 441:45 +CPU= 22.584
  06:36:07 STEP 1768 H= 442:00 +CPU= 25.445
  06:36:35 STEP 1769 H= 442:15 +CPU= 28.300
  06:37:05 STEP 1770 H= 442:30 +CPU= 28.953
  06:37:33 STEP 1771 H= 442:45 +CPU= 28.838
  06:38:03 STEP 1772 H= 443:00 +CPU= 29.783
  06:38:31 STEP 1773 H= 443:15 +CPU= 27.358
  06:38:59 STEP 1774 H= 443:30 +CPU= 28.570
  06:39:24 STEP 1775 H= 443:45 +CPU= 24.653
  06:40:04 STEP 1776 H= 444:00 +CPU= 39.666
Moving to projects directory: /home/ij2/slots/21/ICMGGh7zg+001776
Moving to projects directory: /home/ij2/slots/21/ICMSHh7zg+001776
Moving to projects directory: /home/ij2/slots/21/ICMUAh7zg+001776
  06:40:29 STEP 1777 H= 444:15 +CPU= 24.469
  06:40:57 STEP 1778 H= 444:30 +CPU= 28.536
  06:41:26 STEP 1779 H= 444:45 +CPU= 28.977
  06:41:53 STEP 1780 H= 445:00 +CPU= 26.913
  06:42:21 STEP 1781 H= 445:15 +CPU= 27.632
  06:42:50 STEP 1782 H= 445:30 +CPU= 28.421
  06:43:17 STEP 1783 H= 445:45 +CPU= 27.811
  06:43:46 STEP 1784 H= 446:00 +CPU= 28.422
  06:44:13 STEP 1785 H= 446:15 +CPU= 26.720
  06:44:41 STEP 1786 H= 446:30 +CPU= 28.449
  06:45:09 STEP 1787 H= 446:45 +CPU= 28.114
  06:45:42 STEP 1788 H= 447:00 +CPU= 32.318
Moving to projects directory: /home/ij2/slots/21/ICMGGh7zg+001788
Moving to projects directory: /home/ij2/slots/21/ICMSHh7zg+001788
Moving to projects directory: /home/ij2/slots/21/ICMUAh7zg+001788
  06:46:11 STEP 1789 H= 447:15 +CPU= 29.071
  06:46:40 STEP 1790 H= 447:30 +CPU= 28.569
  06:47:08 STEP 1791 H= 447:45 +CPU= 28.499
  06:47:38 STEP 1792 H= 448:00 +CPU= 29.782
  06:48:07 STEP 1793 H= 448:15 +CPU= 28.633
  06:48:36 STEP 1794 H= 448:30 +CPU= 28.920
  06:49:05 STEP 1795 H= 448:45 +CPU= 29.246
  06:49:31 STEP 1796 H= 449:00 +CPU= 26.223
  06:50:00 STEP 1797 H= 449:15 +CPU= 28.737
  06:50:28 STEP 1798 H= 449:30 +CPU= 28.399
  06:50:55 STEP 1799 H= 449:45 +CPU= 26.657
  06:51:32 STEP 1800 H= 450:00 +CPU= 36.864
Moving to projects directory: /home/ij2/slots/21/ICMGGh7zg+001800
Moving to projects directory: /home/ij2/slots/21/ICMSHh7zg+001800
Moving to projects directory: /home/ij2/slots/21/ICMUAh7zg+001800
  06:52:01 STEP 1801 H= 450:15 +CPU= 28.848
  06:52:29 STEP 1802 H= 450:30 +CPU= 28.266
  06:52:58 STEP 1803 H= 450:45 +CPU= 28.934
  06:53:28 STEP 1804 H= 451:00 +CPU= 29.411
  06:53:56 STEP 1805 H= 451:15 +CPU= 27.933
  06:54:21 STEP 1806 H= 451:30 +CPU= 25.470
  06:54:49 STEP 1807 H= 451:45 +CPU= 27.375
  06:55:15 STEP 1808 H= 452:00 +CPU= 26.500
  06:55:44 STEP 1809 H= 452:15 +CPU= 28.395
  06:56:13 STEP 1810 H= 452:30 +CPU= 29.158
  06:56:39 STEP 1811 H= 452:45 +CPU= 26.101
  06:57:15 STEP 1812 H= 453:00 +CPU= 35.507
Moving to projects directory: /home/ij2/slots/21/ICMGGh7zg+001812
Moving to projects directory: /home/ij2/slots/21/ICMSHh7zg+001812
Moving to projects directory: /home/ij2/slots/21/ICMUAh7zg+001812
  06:57:43 STEP 1813 H= 453:15 +CPU= 28.636
  06:58:12 STEP 1814 H= 453:30 +CPU= 28.634
  06:58:41 STEP 1815 H= 453:45 +CPU= 29.051
  06:59:10 STEP 1816 H= 454:00 +CPU= 28.660
  06:59:39 STEP 1817 H= 454:15 +CPU= 28.937
  07:00:08 STEP 1818 H= 454:30 +CPU= 29.153
  07:00:37 STEP 1819 H= 454:45 +CPU= 28.965
  07:01:07 STEP 1820 H= 455:00 +CPU= 29.867
  07:01:35 STEP 1821 H= 455:15 +CPU= 28.392
  07:02:02 STEP 1822 H= 455:30 +CPU= 26.791
  07:02:29 STEP 1823 H= 455:45 +CPU= 27.011
  07:03:09 STEP 1824 H= 456:00 +CPU= 39.689
Moving to projects directory: /home/ij2/slots/21/ICMGGh7zg+001824
Moving to projects directory: /home/ij2/slots/21/ICMSHh7zg+001824
Moving to projects directory: /home/ij2/slots/21/ICMUAh7zg+001824
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMGGh7zg+001728
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMSHh7zg+001728
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMUAh7zg+001728
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMGGh7zg+001740
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMSHh7zg+001740
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMUAh7zg+001740
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMGGh7zg+001752
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMSHh7zg+001752
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMUAh7zg+001752
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMGGh7zg+001764
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMSHh7zg+001764
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMUAh7zg+001764
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMGGh7zg+001776
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMSHh7zg+001776
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMUAh7zg+001776
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMGGh7zg+001788
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMSHh7zg+001788
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMUAh7zg+001788
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMGGh7zg+001800
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMSHh7zg+001800
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMUAh7zg+001800
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMGGh7zg+001812
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMSHh7zg+001812
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMUAh7zg+001812
Zipping up the intermediate file: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_a1fy_2016092300_20_1018_12290197_1_r509449324_18.zip
Uploading the intermediate file: upload_file_18.zip
  07:03:38 STEP 1825 H= 456:15 +CPU= 29.097
  07:04:07 STEP 1826 H= 456:30 +CPU= 28.933
  07:04:35 STEP 1827 H= 456:45 +CPU= 27.786
  07:05:05 STEP 1828 H= 457:00 +CPU= 29.415
  07:05:31 STEP 1829 H= 457:15 +CPU= 25.945
  07:05:57 STEP 1830 H= 457:30 +CPU= 26.794
  07:06:27 STEP 1831 H= 457:45 +CPU= 29.107
  07:06:57 STEP 1832 H= 458:00 +CPU= 29.933
  07:07:24 STEP 1833 H= 458:15 +CPU= 27.624
  07:07:54 STEP 1834 H= 458:30 +CPU= 29.848
  07:08:23 STEP 1835 H= 458:45 +CPU= 28.572
  07:09:00 STEP 1836 H= 459:00 +CPU= 36.837
Moving to projects directory: /home/ij2/slots/21/ICMGGh7zg+001836
Moving to projects directory: /home/ij2/slots/21/ICMSHh7zg+001836
Moving to projects directory: /home/ij2/slots/21/ICMUAh7zg+001836
  07:09:28 STEP 1837 H= 459:15 +CPU= 27.835
  07:09:57 STEP 1838 H= 459:30 +CPU= 28.691
  07:10:25 STEP 1839 H= 459:45 +CPU= 28.871
  07:10:56 STEP 1840 H= 460:00 +CPU= 30.097
  07:11:25 STEP 1841 H= 460:15 +CPU= 29.336
  07:11:53 STEP 1842 H= 460:30 +CPU= 28.286
  07:12:23 STEP 1843 H= 460:45 +CPU= 29.376
  07:12:53 STEP 1844 H= 461:00 +CPU= 29.708
  07:13:20 STEP 1845 H= 461:15 +CPU= 27.892
  07:13:48 STEP 1846 H= 461:30 +CPU= 27.209
  07:14:14 STEP 1847 H= 461:45 +CPU= 26.421
  07:14:50 STEP 1848 H= 462:00 +CPU= 36.282
Moving to projects directory: /home/ij2/slots/21/ICMGGh7zg+001848
Moving to projects directory: /home/ij2/slots/21/ICMSHh7zg+001848
Moving to projects directory: /home/ij2/slots/21/ICMUAh7zg+001848
  07:15:17 STEP 1849 H= 462:15 +CPU= 26.519
  07:15:46 STEP 1850 H= 462:30 +CPU= 28.712
  07:16:14 STEP 1851 H= 462:45 +CPU= 28.310
  07:16:43 STEP 1852 H= 463:00 +CPU= 28.554
  07:17:12 STEP 1853 H= 463:15 +CPU= 28.860
  07:17:39 STEP 1854 H= 463:30 +CPU= 27.243
  07:18:08 STEP 1855 H= 463:45 +CPU= 29.173
  07:18:34 STEP 1856 H= 464:00 +CPU= 25.458
  07:19:03 STEP 1857 H= 464:15 +CPU= 28.910
  07:19:31 STEP 1858 H= 464:30 +CPU= 27.960
  07:19:56 STEP 1859 H= 464:45 +CPU= 25.046
  07:20:29 STEP 1860 H= 465:00 +CPU= 32.972
Moving to projects directory: /home/ij2/slots/21/ICMGGh7zg+001860
Moving to projects directory: /home/ij2/slots/21/ICMSHh7zg+001860
Moving to projects directory: /home/ij2/slots/21/ICMUAh7zg+001860
  07:20:54 STEP 1861 H= 465:15 +CPU= 25.367
  07:21:23 STEP 1862 H= 465:30 +CPU= 28.958
  07:21:52 STEP 1863 H= 465:45 +CPU= 28.944
  07:22:22 STEP 1864 H= 466:00 +CPU= 29.710
  07:22:51 STEP 1865 H= 466:15 +CPU= 28.694
  07:23:20 STEP 1866 H= 466:30 +CPU= 29.232
  07:23:49 STEP 1867 H= 466:45 +CPU= 29.152
  07:24:20 STEP 1868 H= 467:00 +CPU= 30.237
  07:24:49 STEP 1869 H= 467:15 +CPU= 29.266
  07:25:14 STEP 1870 H= 467:30 +CPU= 24.939
  07:25:39 STEP 1871 H= 467:45 +CPU= 25.312
  07:26:19 STEP 1872 H= 468:00 +CPU= 39.316
Moving to projects directory: /home/ij2/slots/21/ICMGGh7zg+001872
Moving to projects directory: /home/ij2/slots/21/ICMSHh7zg+001872
Moving to projects directory: /home/ij2/slots/21/ICMUAh7zg+001872
  07:26:48 STEP 1873 H= 468:15 +CPU= 28.685
  07:27:16 STEP 1874 H= 468:30 +CPU= 28.551
  07:27:45 STEP 1875 H= 468:45 +CPU= 29.007
  07:28:15 STEP 1876 H= 469:00 +CPU= 30.010
  07:28:44 STEP 1877 H= 469:15 +CPU= 28.811
  07:29:11 STEP 1878 H= 469:30 +CPU= 27.104
  07:29:40 STEP 1879 H= 469:45 +CPU= 29.031
  07:30:10 STEP 1880 H= 470:00 +CPU= 29.713
  07:30:38 STEP 1881 H= 470:15 +CPU= 28.216
  07:31:06 STEP 1882 H= 470:30 +CPU= 28.096
  07:31:35 STEP 1883 H= 470:45 +CPU= 28.746
  07:32:12 STEP 1884 H= 471:00 +CPU= 36.254
Moving to projects directory: /home/ij2/slots/21/ICMGGh7zg+001884
Moving to projects directory: /home/ij2/slots/21/ICMSHh7zg+001884
Moving to projects directory: /home/ij2/slots/21/ICMUAh7zg+001884
  07:32:39 STEP 1885 H= 471:15 +CPU= 27.080
  07:33:06 STEP 1886 H= 471:30 +CPU= 27.389
  07:33:35 STEP 1887 H= 471:45 +CPU= 28.282
  07:34:04 STEP 1888 H= 472:00 +CPU= 28.883
  07:34:33 STEP 1889 H= 472:15 +CPU= 29.136
  07:35:01 STEP 1890 H= 472:30 +CPU= 28.488
  07:35:30 STEP 1891 H= 472:45 +CPU= 29.053
  07:36:00 STEP 1892 H= 473:00 +CPU= 29.862
  07:36:29 STEP 1893 H= 473:15 +CPU= 29.325
  07:36:58 STEP 1894 H= 473:30 +CPU= 28.438
  07:37:27 STEP 1895 H= 473:45 +CPU= 28.711
  07:38:03 STEP 1896 H= 474:00 +CPU= 35.952
Moving to projects directory: /home/ij2/slots/21/ICMGGh7zg+001896
Moving to projects directory: /home/ij2/slots/21/ICMSHh7zg+001896
Moving to projects directory: /home/ij2/slots/21/ICMUAh7zg+001896
  07:38:30 STEP 1897 H= 474:15 +CPU= 26.994
  07:39:00 STEP 1898 H= 474:30 +CPU= 29.948
  07:39:28 STEP 1899 H= 474:45 +CPU= 27.787
  07:39:58 STEP 1900 H= 475:00 +CPU= 30.255
  07:40:27 STEP 1901 H= 475:15 +CPU= 28.907
  07:40:53 STEP 1902 H= 475:30 +CPU= 26.102
  07:41:23 STEP 1903 H= 475:45 +CPU= 29.563
  07:41:53 STEP 1904 H= 476:00 +CPU= 29.952
  07:42:21 STEP 1905 H= 476:15 +CPU= 28.339
  07:42:45 STEP 1906 H= 476:30 +CPU= 24.264
  07:43:15 STEP 1907 H= 476:45 +CPU= 29.508
  07:43:52 STEP 1908 H= 477:00 +CPU= 36.584
Moving to projects directory: /home/ij2/slots/21/ICMGGh7zg+001908
Moving to projects directory: /home/ij2/slots/21/ICMSHh7zg+001908
Moving to projects directory: /home/ij2/slots/21/ICMUAh7zg+001908
  07:44:17 STEP 1909 H= 477:15 +CPU= 25.360
  07:44:46 STEP 1910 H= 477:30 +CPU= 28.629
  07:45:14 STEP 1911 H= 477:45 +CPU= 28.514
  07:45:44 STEP 1912 H= 478:00 +CPU= 29.719
  07:46:13 STEP 1913 H= 478:15 +CPU= 29.311
  07:46:41 STEP 1914 H= 478:30 +CPU= 27.285
  07:47:07 STEP 1915 H= 478:45 +CPU= 25.742
  07:47:36 STEP 1916 H= 479:00 +CPU= 29.082
  07:48:03 STEP 1917 H= 479:15 +CPU= 27.520
  07:48:29 STEP 1918 H= 479:30 +CPU= 26.174
  07:48:58 STEP 1919 H= 479:45 +CPU= 28.555
  07:49:35 STEP 1920 H= 480:00 +CPU= 36.918
..The child process terminated with status: 0
>>> Printing last 70 lines from file: NODE.001_01

I   VARIABLE   I      Initial value         I        Current value        I
I-------------------------------------------------------------------------I
I  OMEGA (P/S) I     -0.1959076066E-21      I       0.1003104193E-06      I
ICloud fractionI      0.0000000000E+00      I       0.6146476197E-03      I
I  Relat. Hum. I      0.3001386874E+00      I       0.3200358940E+00      I
I  PASS 01     I      0.0000000000E+00      I       0.8986201769E+01      I
DDH------------------------------------------------------------------------
  
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
 NSTEP =  1920 SCAN2M_HPOS  P
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ICMGGh7zg+001920 MODE=a
 IO-STREAM CLOSED - ICMGGh7zg+001920
 MPI-TASK:   1 -    301490116 BYTES IN      7 RECORDS TRANSFERRED IN    0.0004 SECONDS753725.3 Mbytes/s, TOTAL TIME=    0.0128( 3.1%)
 07:49:35 STEP 1920 H= 480:00 +CPU= 36.918
 END CNT3
 NSTEP =  1920 CNT0   000000000  27.496  27.496  27.547   0.998               0               0               0               0               0
IO-STREAM STATISTICS, TOTAL NO OF REC/BYTES READ            0/           0 WRITTEN        57546/************
 -TASK-OPENED-OPEN-RECS IN -KBYTE IN    -RECS OUT-KBYTE OUT-WALL   -WALL IN-WALL
  OU-TOT IN -TOT OUT
   1   1008    0        0            0    57546 ************   102.0     0.0   101.9     0.0   212.1
===-=== START OF TIMING STATISTICS ===-===
 
STATS FOR ALL TASKS
 NUM ROUTINE                                     CALLS  MEAN(ms)   MAX(ms)   FRAC(%)  UNBAL(%)
   0 CNT0     - COMPLETE EXECUTION                   1 ********* *********    100.00      0.00
   1 CNT4     - FORWARD INTEGRATION                  1 ********* *********     99.99      0.00
   8 SCAN2M - GRID-POINT DYNAMICS                 2082    4558.6    4558.6     17.08      0.00
   9 SPCM     - SPECTRAL COMP.                    1920     663.3     663.3      2.29      0.00
  10 SCAN2M - PHYSICS                             1921   13591.8   13591.8     46.99      0.00
  11 IOPACK   - OUTPUT P.P. RESULTS                161     864.8     864.8      0.25      0.00
  12 SPNORM   - SPECTRAL NORM COMP.                 83      42.7      42.7      0.01      0.00
  14 SUINIF                                          1    6369.5    6369.5      0.01      0.00
  17 GRIDFPOS IN CNT4                              161      29.5      29.5      0.01      0.00
  18 SUSPECG                                         1    3340.0    3340.0      0.01      0.00
  19 SUSPEC                                          1    3349.8    3349.8      0.01      0.00
  24 SUGRIDU                                         1    2550.0    2550.0      0.00      0.00
  25 SPECRT                                          1     282.7     282.7      0.00      0.00
  26 SUGRIDF                                         1     187.0     187.0      0.00      0.00
  27 RESTART FILES - WRITING                        40    3480.7    3480.7      0.25      0.00
  28 RESTART FILES - READING                         1       0.0       0.0      0.00      0.00
  29 SU4FPOS IN CNT4                               161       0.1       0.1      0.00      0.00
  30 DYNFPOS IN CNT4                               161    6787.8    6787.8      1.97      0.00
  31 POSDDH IN STEPO                                 9       4.7       4.7      0.00      0.00
  37 CPGLAG   - SL COMPUTATIONS                   1921    6316.1    6316.1     21.84      0.00
  39 SU0YOMB                                         1     766.5     766.5      0.00      0.00
  51 SCAN2M   - SL COMM. PART 1                   1921      22.4      22.4      0.08      0.00
  54 SPCM     - M TO S/S TO M TRANSP.             1920     178.2     178.2      0.62      0.00
  55 SPCIMPF  - S TO M/M TO S TRANSP.             1920      54.0      54.0      0.19      0.00
  56 SPNORM   - SPECTRAL NORM COMM.                 83       0.2       0.2      0.00      0.00
 102 LTINV_CTL   - INVERSE LEGENDRE TRANSFORM     4165     426.9     426.9      3.20      0.00
 103 LTDIR_CTL   - DIRECT LEGENDRE TRANSFORM      4006     265.4     265.4      1.91      0.00
 106 FTDIR_CTL   - DIRECT FOURIER TRANSFORM       4006     129.6     129.6      0.93      0.00
 107 FTINV_CTL   - INVERSE FOURIER TRANSFORM      4165     141.3     141.3      1.06      0.00
 140 SULEG       - COMP. OF LEGENDRE POL.            1      61.8      61.8      0.00      0.00
 152 LTINV_CTL   - M TO L TRANSPOSITION           4165      76.4      76.4      0.57      0.00
 153 LTDIR_CTL   - L TO M TRANSPOSITION           4006      35.9      35.9      0.26      0.00
 157 FTINV_CTL   - L TO G TRANSPOSITION           4165      78.3      78.3      0.59      0.00
 158 FTDIR_CTL   - G TO L TRANSPOSITION           4006      37.9      37.9      0.27      0.00
 400 GSTATS                                     223224       0.0       0.0      0.00      0.00
 401 GSTATS HOOK                                207696       0.0       0.0      0.00      0.00
TOTAL MEASURED IMBALANCE =       0.0 SECONDS,  0.0 PERCENT
TOTAL WALLCLOCK TIME    55566.0 CPU TIME   55452.8 VECTOR TIME    55452.8


===-=== END   OF TIMING STATISTICS ===-===
FORECAST DAYS PER DAY       31.1
  *** END CNT0 *** 
------------------------------------------------
Moving to projects directory: /home/ij2/slots/21/ICMGGh7zg+001920
Moving to projects directory: /home/ij2/slots/21/ICMSHh7zg+001920
Moving to projects directory: /home/ij2/slots/21/ICMUAh7zg+001920
Adding to the zip: /home/ij2/slots/21/NODE.001_01
Adding to the zip: /home/ij2/slots/21/ifs.stat
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMSHh7zg+001848
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMGGh7zg+001848
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMUAh7zg+001848
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMGGh7zg+001884
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMGGh7zg+001860
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMSHh7zg+001860
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMSHh7zg+001884
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMGGh7zg+001836
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMSHh7zg+001836
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMUAh7zg+001884
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMUAh7zg+001860
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMUAh7zg+001836
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMUAh7zg+001920
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMSHh7zg+001920
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMGGh7zg+001920
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMUAh7zg+001896
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMUAh7zg+001908
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMUAh7zg+001872
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMUAh7zg+001824
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMGGh7zg+001896
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMGGh7zg+001872
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMGGh7zg+001908
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMSHh7zg+001908
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMSHh7zg+001872
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMSHh7zg+001896
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMGGh7zg+001824
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12290197/ICMSHh7zg+001824
Zipping up the final file: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_a1fy_2016092300_20_1018_12290197_1_r509449324_19.zip
Uploading the final file: upload_file_19.zip
Uploading trickle at timestep: 1727100
07:53:08 (687): called boinc_finish(0)

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
17 Jun 2024 05:51:57 1549780 22449458 oifs_43r3_bl_a1fy_2016092300_20_1018_12290197_1 1,727,100 55,453 0.0321
16 Jun 2024 22:39:27 1549780 22449458 oifs_43r3_bl_a1fy_2016092300_20_1018_12290197_1 864,000 27,685 0.0320


©2024 cpdn.org