climateprediction.net home page
Task 22449462

Task 22449462

Name oifs_43r3_bl_a321_2016092300_20_1018_12292288_1
Workunit 12292288
Created 16 Jun 2024, 14:21:41 UTC
Sent 16 Jun 2024, 14:22:38 UTC
Report deadline 15 Aug 2024, 14:22:38 UTC
Received 17 Jun 2024, 6:51:52 UTC
Server state Over
Outcome Success
Client state Done
Exit status 0 (0x00000000)
Computer ID 1549780
Run time 15 hours 45 min 35 sec
CPU time 15 hours 39 min 56 sec
Validate state Valid
Credit 2,635.55
Device peak FLOPS 5.04 GFLOPS
Application version OpenIFS 43r3 Baroclinic Lifecycle v1.13
x86_64-pc-linux-gnu
Peak working set size 5,016.61 MB
Peak swap size 5,981.13 MB
Peak disk usage 1,358.84 MB
Stderr
<core_client_version>7.22.1</core_client_version>
<![CDATA[
<stderr_txt>
2.zip
  02:32:20 STEP 1249 H= 312:15 +CPU= 29.473
  02:32:49 STEP 1250 H= 312:30 +CPU= 29.663
  02:33:18 STEP 1251 H= 312:45 +CPU= 28.231
  02:33:48 STEP 1252 H= 313:00 +CPU= 30.483
  02:34:17 STEP 1253 H= 313:15 +CPU= 28.520
  02:34:46 STEP 1254 H= 313:30 +CPU= 29.322
  02:35:14 STEP 1255 H= 313:45 +CPU= 27.495
  02:35:43 STEP 1256 H= 314:00 +CPU= 28.836
  02:36:12 STEP 1257 H= 314:15 +CPU= 29.213
  02:36:39 STEP 1258 H= 314:30 +CPU= 27.578
  02:37:08 STEP 1259 H= 314:45 +CPU= 28.589
  02:37:45 STEP 1260 H= 315:00 +CPU= 36.726
Moving to projects directory: /home/ij2/slots/20/ICMGGh7zg+001260
Moving to projects directory: /home/ij2/slots/20/ICMSHh7zg+001260
Moving to projects directory: /home/ij2/slots/20/ICMUAh7zg+001260
  02:38:15 STEP 1261 H= 315:15 +CPU= 29.930
  02:38:45 STEP 1262 H= 315:30 +CPU= 29.571
  02:39:15 STEP 1263 H= 315:45 +CPU= 29.984
  02:39:45 STEP 1264 H= 316:00 +CPU= 30.198
  02:40:15 STEP 1265 H= 316:15 +CPU= 29.755
  02:40:43 STEP 1266 H= 316:30 +CPU= 28.584
  02:41:12 STEP 1267 H= 316:45 +CPU= 29.072
  02:41:43 STEP 1268 H= 317:00 +CPU= 30.769
  02:42:13 STEP 1269 H= 317:15 +CPU= 29.507
  02:42:43 STEP 1270 H= 317:30 +CPU= 29.706
  02:43:11 STEP 1271 H= 317:45 +CPU= 28.727
  02:43:47 STEP 1272 H= 318:00 +CPU= 35.165
Moving to projects directory: /home/ij2/slots/20/ICMGGh7zg+001272
Moving to projects directory: /home/ij2/slots/20/ICMSHh7zg+001272
Moving to projects directory: /home/ij2/slots/20/ICMUAh7zg+001272
  02:44:14 STEP 1273 H= 318:15 +CPU= 27.842
  02:44:43 STEP 1274 H= 318:30 +CPU= 28.736
  02:45:13 STEP 1275 H= 318:45 +CPU= 29.799
  02:45:42 STEP 1276 H= 319:00 +CPU= 29.342
  02:46:08 STEP 1277 H= 319:15 +CPU= 25.947
  02:46:36 STEP 1278 H= 319:30 +CPU= 27.031
  02:47:06 STEP 1279 H= 319:45 +CPU= 29.818
  02:47:36 STEP 1280 H= 320:00 +CPU= 30.779
  02:48:07 STEP 1281 H= 320:15 +CPU= 30.114
  02:48:36 STEP 1282 H= 320:30 +CPU= 29.094
  02:49:05 STEP 1283 H= 320:45 +CPU= 29.671
  02:49:43 STEP 1284 H= 321:00 +CPU= 37.932
Moving to projects directory: /home/ij2/slots/20/ICMGGh7zg+001284
Moving to projects directory: /home/ij2/slots/20/ICMSHh7zg+001284
Moving to projects directory: /home/ij2/slots/20/ICMUAh7zg+001284
  02:50:13 STEP 1285 H= 321:15 +CPU= 29.551
  02:50:43 STEP 1286 H= 321:30 +CPU= 29.525
  02:51:12 STEP 1287 H= 321:45 +CPU= 29.061
  02:51:38 STEP 1288 H= 322:00 +CPU= 25.839
  02:52:07 STEP 1289 H= 322:15 +CPU= 28.935
  02:52:30 STEP 1290 H= 322:30 +CPU= 23.658
  02:53:00 STEP 1291 H= 322:45 +CPU= 29.720
  02:53:31 STEP 1292 H= 323:00 +CPU= 30.837
  02:54:00 STEP 1293 H= 323:15 +CPU= 29.400
  02:54:27 STEP 1294 H= 323:30 +CPU= 26.581
  02:54:55 STEP 1295 H= 323:45 +CPU= 27.488
  02:55:36 STEP 1296 H= 324:00 +CPU= 41.176
Moving to projects directory: /home/ij2/slots/20/ICMGGh7zg+001296
Moving to projects directory: /home/ij2/slots/20/ICMSHh7zg+001296
Moving to projects directory: /home/ij2/slots/20/ICMUAh7zg+001296
  02:56:06 STEP 1297 H= 324:15 +CPU= 29.628
  02:56:36 STEP 1298 H= 324:30 +CPU= 30.523
  02:57:06 STEP 1299 H= 324:45 +CPU= 30.175
  02:57:37 STEP 1300 H= 325:00 +CPU= 30.684
  02:58:06 STEP 1301 H= 325:15 +CPU= 28.623
  02:58:33 STEP 1302 H= 325:30 +CPU= 26.998
  02:59:01 STEP 1303 H= 325:45 +CPU= 27.654
  02:59:29 STEP 1304 H= 326:00 +CPU= 28.113
  02:59:54 STEP 1305 H= 326:15 +CPU= 25.295
  03:00:22 STEP 1306 H= 326:30 +CPU= 27.746
  03:00:47 STEP 1307 H= 326:45 +CPU= 25.374
  03:01:24 STEP 1308 H= 327:00 +CPU= 37.141
Moving to projects directory: /home/ij2/slots/20/ICMGGh7zg+001308
Moving to projects directory: /home/ij2/slots/20/ICMSHh7zg+001308
Moving to projects directory: /home/ij2/slots/20/ICMUAh7zg+001308
  03:01:54 STEP 1309 H= 327:15 +CPU= 29.775
  03:02:23 STEP 1310 H= 327:30 +CPU= 29.151
  03:02:52 STEP 1311 H= 327:45 +CPU= 28.073
  03:03:20 STEP 1312 H= 328:00 +CPU= 28.214
  03:03:50 STEP 1313 H= 328:15 +CPU= 29.983
  03:04:20 STEP 1314 H= 328:30 +CPU= 29.991
  03:04:50 STEP 1315 H= 328:45 +CPU= 30.238
  03:05:21 STEP 1316 H= 329:00 +CPU= 30.780
  03:05:51 STEP 1317 H= 329:15 +CPU= 29.899
  03:06:18 STEP 1318 H= 329:30 +CPU= 27.176
  03:06:48 STEP 1319 H= 329:45 +CPU= 29.819
  03:07:23 STEP 1320 H= 330:00 +CPU= 35.022
Moving to projects directory: /home/ij2/slots/20/ICMGGh7zg+001320
Moving to projects directory: /home/ij2/slots/20/ICMSHh7zg+001320
Moving to projects directory: /home/ij2/slots/20/ICMUAh7zg+001320
  03:07:52 STEP 1321 H= 330:15 +CPU= 28.487
  03:08:21 STEP 1322 H= 330:30 +CPU= 29.879
  03:08:50 STEP 1323 H= 330:45 +CPU= 28.498
  03:09:13 STEP 1324 H= 331:00 +CPU= 23.427
  03:09:42 STEP 1325 H= 331:15 +CPU= 28.496
  03:10:12 STEP 1326 H= 331:30 +CPU= 29.861
  03:10:42 STEP 1327 H= 331:45 +CPU= 30.044
  03:11:13 STEP 1328 H= 332:00 +CPU= 30.502
  03:11:42 STEP 1329 H= 332:15 +CPU= 29.612
  03:12:12 STEP 1330 H= 332:30 +CPU= 29.793
  03:12:41 STEP 1331 H= 332:45 +CPU= 28.570
  03:13:17 STEP 1332 H= 333:00 +CPU= 35.906
Moving to projects directory: /home/ij2/slots/20/ICMGGh7zg+001332
Moving to projects directory: /home/ij2/slots/20/ICMSHh7zg+001332
Moving to projects directory: /home/ij2/slots/20/ICMUAh7zg+001332
  03:13:46 STEP 1333 H= 333:15 +CPU= 29.024
  03:14:15 STEP 1334 H= 333:30 +CPU= 28.681
  03:14:44 STEP 1335 H= 333:45 +CPU= 28.952
  03:15:13 STEP 1336 H= 334:00 +CPU= 29.555
  03:15:43 STEP 1337 H= 334:15 +CPU= 29.473
  03:16:13 STEP 1338 H= 334:30 +CPU= 30.432
  03:16:43 STEP 1339 H= 334:45 +CPU= 30.066
  03:17:13 STEP 1340 H= 335:00 +CPU= 29.422
  03:17:43 STEP 1341 H= 335:15 +CPU= 29.898
  03:18:12 STEP 1342 H= 335:30 +CPU= 29.572
  03:18:42 STEP 1343 H= 335:45 +CPU= 29.910
  03:19:24 STEP 1344 H= 336:00 +CPU= 41.024
Moving to projects directory: /home/ij2/slots/20/ICMGGh7zg+001344
Moving to projects directory: /home/ij2/slots/20/ICMSHh7zg+001344
Moving to projects directory: /home/ij2/slots/20/ICMUAh7zg+001344
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMGGh7zg+001248
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMSHh7zg+001248
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMUAh7zg+001248
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMGGh7zg+001260
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMSHh7zg+001260
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMUAh7zg+001260
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMGGh7zg+001272
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMSHh7zg+001272
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMUAh7zg+001272
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMGGh7zg+001284
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMSHh7zg+001284
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMUAh7zg+001284
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMGGh7zg+001296
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMSHh7zg+001296
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMUAh7zg+001296
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMGGh7zg+001308
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMSHh7zg+001308
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMUAh7zg+001308
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMGGh7zg+001320
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMSHh7zg+001320
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMUAh7zg+001320
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMGGh7zg+001332
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMSHh7zg+001332
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMUAh7zg+001332
Zipping up the intermediate file: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_a321_2016092300_20_1018_12292288_1_r1929080527_13.zip
Uploading the intermediate file: upload_file_13.zip
  03:19:54 STEP 1345 H= 336:15 +CPU= 29.740
  03:20:22 STEP 1346 H= 336:30 +CPU= 28.201
  03:20:49 STEP 1347 H= 336:45 +CPU= 26.913
  03:21:20 STEP 1348 H= 337:00 +CPU= 31.319
  03:21:50 STEP 1349 H= 337:15 +CPU= 29.636
  03:22:19 STEP 1350 H= 337:30 +CPU= 28.948
  03:22:47 STEP 1351 H= 337:45 +CPU= 28.126
  03:23:18 STEP 1352 H= 338:00 +CPU= 30.688
  03:23:47 STEP 1353 H= 338:15 +CPU= 29.613
  03:24:13 STEP 1354 H= 338:30 +CPU= 25.036
  03:24:38 STEP 1355 H= 338:45 +CPU= 25.429
  03:25:16 STEP 1356 H= 339:00 +CPU= 38.212
Moving to projects directory: /home/ij2/slots/20/ICMGGh7zg+001356
Moving to projects directory: /home/ij2/slots/20/ICMSHh7zg+001356
Moving to projects directory: /home/ij2/slots/20/ICMUAh7zg+001356
  03:25:47 STEP 1357 H= 339:15 +CPU= 30.213
  03:26:16 STEP 1358 H= 339:30 +CPU= 29.666
  03:26:45 STEP 1359 H= 339:45 +CPU= 28.936
  03:27:15 STEP 1360 H= 340:00 +CPU= 29.399
  03:27:44 STEP 1361 H= 340:15 +CPU= 29.312
  03:28:12 STEP 1362 H= 340:30 +CPU= 27.881
  03:28:41 STEP 1363 H= 340:45 +CPU= 29.480
  03:29:11 STEP 1364 H= 341:00 +CPU= 29.762
  03:29:41 STEP 1365 H= 341:15 +CPU= 30.104
  03:30:11 STEP 1366 H= 341:30 +CPU= 29.658
  03:30:38 STEP 1367 H= 341:45 +CPU= 26.770
  03:31:16 STEP 1368 H= 342:00 +CPU= 37.772
Moving to projects directory: /home/ij2/slots/20/ICMGGh7zg+001368
Moving to projects directory: /home/ij2/slots/20/ICMSHh7zg+001368
Moving to projects directory: /home/ij2/slots/20/ICMUAh7zg+001368
  03:31:44 STEP 1369 H= 342:15 +CPU= 28.051
  03:32:13 STEP 1370 H= 342:30 +CPU= 29.307
  03:32:41 STEP 1371 H= 342:45 +CPU= 27.904
  03:33:11 STEP 1372 H= 343:00 +CPU= 30.331
  03:33:42 STEP 1373 H= 343:15 +CPU= 30.226
  03:34:12 STEP 1374 H= 343:30 +CPU= 29.918
  03:34:41 STEP 1375 H= 343:45 +CPU= 29.647
  03:35:10 STEP 1376 H= 344:00 +CPU= 28.624
  03:35:40 STEP 1377 H= 344:15 +CPU= 29.966
  03:36:10 STEP 1378 H= 344:30 +CPU= 29.835
  03:36:40 STEP 1379 H= 344:45 +CPU= 29.858
  03:37:16 STEP 1380 H= 345:00 +CPU= 35.804
Moving to projects directory: /home/ij2/slots/20/ICMGGh7zg+001380
Moving to projects directory: /home/ij2/slots/20/ICMSHh7zg+001380
Moving to projects directory: /home/ij2/slots/20/ICMUAh7zg+001380
  03:37:44 STEP 1381 H= 345:15 +CPU= 28.071
  03:38:13 STEP 1382 H= 345:30 +CPU= 29.226
  03:38:43 STEP 1383 H= 345:45 +CPU= 30.048
  03:39:11 STEP 1384 H= 346:00 +CPU= 27.935
  03:39:41 STEP 1385 H= 346:15 +CPU= 29.515
  03:40:10 STEP 1386 H= 346:30 +CPU= 29.690
  03:40:40 STEP 1387 H= 346:45 +CPU= 29.933
  03:41:11 STEP 1388 H= 347:00 +CPU= 30.121
  03:41:41 STEP 1389 H= 347:15 +CPU= 29.980
  03:42:08 STEP 1390 H= 347:30 +CPU= 26.922
  03:42:36 STEP 1391 H= 347:45 +CPU= 28.187
  03:43:17 STEP 1392 H= 348:00 +CPU= 40.566
Moving to projects directory: /home/ij2/slots/20/ICMGGh7zg+001392
Moving to projects directory: /home/ij2/slots/20/ICMSHh7zg+001392
Moving to projects directory: /home/ij2/slots/20/ICMUAh7zg+001392
  03:43:42 STEP 1393 H= 348:15 +CPU= 25.489
  03:44:10 STEP 1394 H= 348:30 +CPU= 27.895
  03:44:37 STEP 1395 H= 348:45 +CPU= 27.200
  03:45:07 STEP 1396 H= 349:00 +CPU= 29.963
  03:45:38 STEP 1397 H= 349:15 +CPU= 30.432
  03:46:06 STEP 1398 H= 349:30 +CPU= 28.360
  03:46:37 STEP 1399 H= 349:45 +CPU= 30.207
  03:47:08 STEP 1400 H= 350:00 +CPU= 30.903
  03:47:34 STEP 1401 H= 350:15 +CPU= 26.410
  03:48:04 STEP 1402 H= 350:30 +CPU= 29.802
  03:48:34 STEP 1403 H= 350:45 +CPU= 30.186
  03:49:12 STEP 1404 H= 351:00 +CPU= 37.590
Moving to projects directory: /home/ij2/slots/20/ICMGGh7zg+001404
Moving to projects directory: /home/ij2/slots/20/ICMSHh7zg+001404
Moving to projects directory: /home/ij2/slots/20/ICMUAh7zg+001404
  03:49:42 STEP 1405 H= 351:15 +CPU= 29.655
  03:50:11 STEP 1406 H= 351:30 +CPU= 29.304
  03:50:38 STEP 1407 H= 351:45 +CPU= 27.130
  03:51:09 STEP 1408 H= 352:00 +CPU= 30.286
  03:51:38 STEP 1409 H= 352:15 +CPU= 29.797
  03:52:08 STEP 1410 H= 352:30 +CPU= 29.750
  03:52:38 STEP 1411 H= 352:45 +CPU= 29.539
  03:53:07 STEP 1412 H= 353:00 +CPU= 29.487
  03:53:37 STEP 1413 H= 353:15 +CPU= 29.158
  03:54:04 STEP 1414 H= 353:30 +CPU= 27.715
  03:54:32 STEP 1415 H= 353:45 +CPU= 27.417
  03:55:09 STEP 1416 H= 354:00 +CPU= 37.580
Moving to projects directory: /home/ij2/slots/20/ICMGGh7zg+001416
Moving to projects directory: /home/ij2/slots/20/ICMSHh7zg+001416
Moving to projects directory: /home/ij2/slots/20/ICMUAh7zg+001416
  03:55:37 STEP 1417 H= 354:15 +CPU= 28.003
  03:56:07 STEP 1418 H= 354:30 +CPU= 29.779
  03:56:35 STEP 1419 H= 354:45 +CPU= 27.651
  03:57:06 STEP 1420 H= 355:00 +CPU= 30.655
  03:57:34 STEP 1421 H= 355:15 +CPU= 28.016
  03:58:03 STEP 1422 H= 355:30 +CPU= 28.679
  03:58:33 STEP 1423 H= 355:45 +CPU= 29.938
  03:59:04 STEP 1424 H= 356:00 +CPU= 30.820
  03:59:33 STEP 1425 H= 356:15 +CPU= 29.313
  04:00:01 STEP 1426 H= 356:30 +CPU= 27.648
  04:00:30 STEP 1427 H= 356:45 +CPU= 29.717
  04:01:06 STEP 1428 H= 357:00 +CPU= 35.016
Moving to projects directory: /home/ij2/slots/20/ICMGGh7zg+001428
Moving to projects directory: /home/ij2/slots/20/ICMSHh7zg+001428
Moving to projects directory: /home/ij2/slots/20/ICMUAh7zg+001428
  04:01:35 STEP 1429 H= 357:15 +CPU= 28.850
  04:02:02 STEP 1430 H= 357:30 +CPU= 27.712
  04:02:31 STEP 1431 H= 357:45 +CPU= 28.323
  04:03:01 STEP 1432 H= 358:00 +CPU= 29.950
  04:03:29 STEP 1433 H= 358:15 +CPU= 28.545
  04:03:55 STEP 1434 H= 358:30 +CPU= 25.796
  04:04:25 STEP 1435 H= 358:45 +CPU= 29.412
  04:04:56 STEP 1436 H= 359:00 +CPU= 30.898
  04:05:25 STEP 1437 H= 359:15 +CPU= 29.551
  04:05:55 STEP 1438 H= 359:30 +CPU= 29.987
  04:06:25 STEP 1439 H= 359:45 +CPU= 29.621
  04:07:07 STEP 1440 H= 360:00 +CPU= 41.620
Moving to projects directory: /home/ij2/slots/20/ICMGGh7zg+001440
Moving to projects directory: /home/ij2/slots/20/ICMSHh7zg+001440
Moving to projects directory: /home/ij2/slots/20/ICMUAh7zg+001440
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMGGh7zg+001344
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMSHh7zg+001344
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMUAh7zg+001344
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMGGh7zg+001356
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMSHh7zg+001356
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMUAh7zg+001356
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMGGh7zg+001368
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMSHh7zg+001368
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMUAh7zg+001368
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMGGh7zg+001380
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMSHh7zg+001380
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMUAh7zg+001380
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMGGh7zg+001392
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMSHh7zg+001392
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMUAh7zg+001392
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMGGh7zg+001404
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMSHh7zg+001404
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMUAh7zg+001404
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMGGh7zg+001416
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMSHh7zg+001416
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMUAh7zg+001416
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMGGh7zg+001428
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMSHh7zg+001428
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMUAh7zg+001428
Zipping up the intermediate file: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_a321_2016092300_20_1018_12292288_1_r1929080527_14.zip
Uploading the intermediate file: upload_file_14.zip
  04:07:35 STEP 1441 H= 360:15 +CPU= 28.456
  04:08:01 STEP 1442 H= 360:30 +CPU= 25.913
  04:08:29 STEP 1443 H= 360:45 +CPU= 27.821
  04:08:58 STEP 1444 H= 361:00 +CPU= 28.981
  04:09:27 STEP 1445 H= 361:15 +CPU= 29.198
  04:09:57 STEP 1446 H= 361:30 +CPU= 29.395
  04:10:27 STEP 1447 H= 361:45 +CPU= 30.303
  04:10:58 STEP 1448 H= 362:00 +CPU= 30.537
  04:11:27 STEP 1449 H= 362:15 +CPU= 29.201
  04:11:57 STEP 1450 H= 362:30 +CPU= 30.267
  04:12:24 STEP 1451 H= 362:45 +CPU= 26.217
  04:13:02 STEP 1452 H= 363:00 +CPU= 38.150
Moving to projects directory: /home/ij2/slots/20/ICMGGh7zg+001452
Moving to projects directory: /home/ij2/slots/20/ICMSHh7zg+001452
Moving to projects directory: /home/ij2/slots/20/ICMUAh7zg+001452
  04:13:32 STEP 1453 H= 363:15 +CPU= 29.794
  04:14:01 STEP 1454 H= 363:30 +CPU= 29.667
  04:14:31 STEP 1455 H= 363:45 +CPU= 29.931
  04:15:02 STEP 1456 H= 364:00 +CPU= 30.426
  04:15:32 STEP 1457 H= 364:15 +CPU= 29.768
  04:16:01 STEP 1458 H= 364:30 +CPU= 28.984
  04:16:30 STEP 1459 H= 364:45 +CPU= 28.874
  04:17:01 STEP 1460 H= 365:00 +CPU= 31.402
  04:17:29 STEP 1461 H= 365:15 +CPU= 27.349
  04:17:58 STEP 1462 H= 365:30 +CPU= 29.760
  04:18:28 STEP 1463 H= 365:45 +CPU= 29.750
  04:19:06 STEP 1464 H= 366:00 +CPU= 37.906
Moving to projects directory: /home/ij2/slots/20/ICMGGh7zg+001464
Moving to projects directory: /home/ij2/slots/20/ICMSHh7zg+001464
Moving to projects directory: /home/ij2/slots/20/ICMUAh7zg+001464
  04:19:36 STEP 1465 H= 366:15 +CPU= 29.623
  04:20:04 STEP 1466 H= 366:30 +CPU= 28.209
  04:20:33 STEP 1467 H= 366:45 +CPU= 28.314
  04:21:03 STEP 1468 H= 367:00 +CPU= 30.491
  04:21:33 STEP 1469 H= 367:15 +CPU= 29.501
  04:22:03 STEP 1470 H= 367:30 +CPU= 29.899
  04:22:32 STEP 1471 H= 367:45 +CPU= 29.668
  04:23:03 STEP 1472 H= 368:00 +CPU= 30.422
  04:23:33 STEP 1473 H= 368:15 +CPU= 29.763
  04:24:00 STEP 1474 H= 368:30 +CPU= 27.194
  04:24:27 STEP 1475 H= 368:45 +CPU= 26.823
  04:25:03 STEP 1476 H= 369:00 +CPU= 36.467
Moving to projects directory: /home/ij2/slots/20/ICMGGh7zg+001476
Moving to projects directory: /home/ij2/slots/20/ICMSHh7zg+001476
Moving to projects directory: /home/ij2/slots/20/ICMUAh7zg+001476
  04:25:33 STEP 1477 H= 369:15 +CPU= 29.462
  04:26:02 STEP 1478 H= 369:30 +CPU= 29.452
  04:26:32 STEP 1479 H= 369:45 +CPU= 29.333
  04:27:03 STEP 1480 H= 370:00 +CPU= 30.816
  04:27:32 STEP 1481 H= 370:15 +CPU= 29.654
  04:28:01 STEP 1482 H= 370:30 +CPU= 29.027
  04:28:30 STEP 1483 H= 370:45 +CPU= 28.417
  04:28:58 STEP 1484 H= 371:00 +CPU= 28.394
  04:29:28 STEP 1485 H= 371:15 +CPU= 29.945
  04:29:56 STEP 1486 H= 371:30 +CPU= 28.112
  04:30:25 STEP 1487 H= 371:45 +CPU= 28.290
  04:31:07 STEP 1488 H= 372:00 +CPU= 42.030
Moving to projects directory: /home/ij2/slots/20/ICMGGh7zg+001488
Moving to projects directory: /home/ij2/slots/20/ICMSHh7zg+001488
Moving to projects directory: /home/ij2/slots/20/ICMUAh7zg+001488
  04:31:37 STEP 1489 H= 372:15 +CPU= 29.769
  04:32:07 STEP 1490 H= 372:30 +CPU= 29.941
  04:32:36 STEP 1491 H= 372:45 +CPU= 28.541
  04:33:04 STEP 1492 H= 373:00 +CPU= 28.383
  04:33:34 STEP 1493 H= 373:15 +CPU= 29.435
  04:34:03 STEP 1494 H= 373:30 +CPU= 29.526
  04:34:30 STEP 1495 H= 373:45 +CPU= 27.277
  04:35:00 STEP 1496 H= 374:00 +CPU= 29.811
  04:35:28 STEP 1497 H= 374:15 +CPU= 27.495
  04:35:57 STEP 1498 H= 374:30 +CPU= 29.309
  04:36:27 STEP 1499 H= 374:45 +CPU= 29.769
  04:37:05 STEP 1500 H= 375:00 +CPU= 37.521
Moving to projects directory: /home/ij2/slots/20/ICMGGh7zg+001500
Moving to projects directory: /home/ij2/slots/20/ICMSHh7zg+001500
Moving to projects directory: /home/ij2/slots/20/ICMUAh7zg+001500
  04:37:33 STEP 1501 H= 375:15 +CPU= 28.584
  04:38:00 STEP 1502 H= 375:30 +CPU= 26.674
  04:38:30 STEP 1503 H= 375:45 +CPU= 29.540
  04:38:58 STEP 1504 H= 376:00 +CPU= 28.007
  04:39:27 STEP 1505 H= 376:15 +CPU= 28.890
  04:39:56 STEP 1506 H= 376:30 +CPU= 29.716
  04:40:22 STEP 1507 H= 376:45 +CPU= 25.394
  04:40:49 STEP 1508 H= 377:00 +CPU= 27.430
  04:41:17 STEP 1509 H= 377:15 +CPU= 27.401
  04:41:47 STEP 1510 H= 377:30 +CPU= 30.447
  04:42:15 STEP 1511 H= 377:45 +CPU= 28.264
  04:42:49 STEP 1512 H= 378:00 +CPU= 33.610
Moving to projects directory: /home/ij2/slots/20/ICMGGh7zg+001512
Moving to projects directory: /home/ij2/slots/20/ICMSHh7zg+001512
Moving to projects directory: /home/ij2/slots/20/ICMUAh7zg+001512
  04:43:17 STEP 1513 H= 378:15 +CPU= 27.562
  04:43:46 STEP 1514 H= 378:30 +CPU= 29.618
  04:44:16 STEP 1515 H= 378:45 +CPU= 29.699
  04:44:47 STEP 1516 H= 379:00 +CPU= 30.724
  04:45:16 STEP 1517 H= 379:15 +CPU= 28.778
  04:45:45 STEP 1518 H= 379:30 +CPU= 29.617
  04:46:15 STEP 1519 H= 379:45 +CPU= 29.720
  04:46:44 STEP 1520 H= 380:00 +CPU= 28.516
  04:47:14 STEP 1521 H= 380:15 +CPU= 29.968
  04:47:43 STEP 1522 H= 380:30 +CPU= 29.164
  04:48:10 STEP 1523 H= 380:45 +CPU= 26.359
  04:48:41 STEP 1524 H= 381:00 +CPU= 31.066
Moving to projects directory: /home/ij2/slots/20/ICMGGh7zg+001524
Moving to projects directory: /home/ij2/slots/20/ICMSHh7zg+001524
Moving to projects directory: /home/ij2/slots/20/ICMUAh7zg+001524
  04:49:08 STEP 1525 H= 381:15 +CPU= 27.096
  04:49:32 STEP 1526 H= 381:30 +CPU= 24.052
  04:50:02 STEP 1527 H= 381:45 +CPU= 29.687
  04:50:32 STEP 1528 H= 382:00 +CPU= 30.686
  04:51:00 STEP 1529 H= 382:15 +CPU= 27.949
  04:51:30 STEP 1530 H= 382:30 +CPU= 29.086
  04:51:59 STEP 1531 H= 382:45 +CPU= 29.605
  04:52:31 STEP 1532 H= 383:00 +CPU= 31.318
  04:53:00 STEP 1533 H= 383:15 +CPU= 29.564
  04:53:30 STEP 1534 H= 383:30 +CPU= 29.562
  04:54:00 STEP 1535 H= 383:45 +CPU= 29.776
  04:54:41 STEP 1536 H= 384:00 +CPU= 40.798
Moving to projects directory: /home/ij2/slots/20/ICMGGh7zg+001536
Moving to projects directory: /home/ij2/slots/20/ICMSHh7zg+001536
Moving to projects directory: /home/ij2/slots/20/ICMUAh7zg+001536
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMGGh7zg+001440
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMSHh7zg+001440
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMUAh7zg+001440
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMGGh7zg+001452
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMSHh7zg+001452
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMUAh7zg+001452
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMGGh7zg+001464
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMSHh7zg+001464
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMUAh7zg+001464
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMGGh7zg+001476
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMSHh7zg+001476
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMUAh7zg+001476
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMGGh7zg+001488
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMSHh7zg+001488
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMUAh7zg+001488
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMGGh7zg+001500
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMSHh7zg+001500
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMUAh7zg+001500
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMGGh7zg+001512
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMSHh7zg+001512
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMUAh7zg+001512
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMGGh7zg+001524
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMSHh7zg+001524
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMUAh7zg+001524
Zipping up the intermediate file: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_a321_2016092300_20_1018_12292288_1_r1929080527_15.zip
Uploading the intermediate file: upload_file_15.zip
  04:55:11 STEP 1537 H= 384:15 +CPU= 30.309
  04:55:38 STEP 1538 H= 384:30 +CPU= 26.435
  04:56:07 STEP 1539 H= 384:45 +CPU= 29.042
  04:56:37 STEP 1540 H= 385:00 +CPU= 30.617
  04:57:06 STEP 1541 H= 385:15 +CPU= 28.959
  04:57:34 STEP 1542 H= 385:30 +CPU= 28.131
  04:58:03 STEP 1543 H= 385:45 +CPU= 28.794
  04:58:34 STEP 1544 H= 386:00 +CPU= 30.221
  04:59:01 STEP 1545 H= 386:15 +CPU= 27.765
  04:59:31 STEP 1546 H= 386:30 +CPU= 29.793
  05:00:00 STEP 1547 H= 386:45 +CPU= 28.698
  05:00:38 STEP 1548 H= 387:00 +CPU= 37.677
Moving to projects directory: /home/ij2/slots/20/ICMGGh7zg+001548
Moving to projects directory: /home/ij2/slots/20/ICMSHh7zg+001548
Moving to projects directory: /home/ij2/slots/20/ICMUAh7zg+001548
  05:01:07 STEP 1549 H= 387:15 +CPU= 29.372
  05:01:37 STEP 1550 H= 387:30 +CPU= 29.409
  05:02:05 STEP 1551 H= 387:45 +CPU= 28.590
  05:02:36 STEP 1552 H= 388:00 +CPU= 30.343
  05:03:03 STEP 1553 H= 388:15 +CPU= 27.564
  05:03:29 STEP 1554 H= 388:30 +CPU= 26.125
  05:03:59 STEP 1555 H= 388:45 +CPU= 29.418
  05:04:30 STEP 1556 H= 389:00 +CPU= 30.631
  05:04:59 STEP 1557 H= 389:15 +CPU= 29.188
  05:05:26 STEP 1558 H= 389:30 +CPU= 27.126
  05:05:56 STEP 1559 H= 389:45 +CPU= 29.765
  05:06:34 STEP 1560 H= 390:00 +CPU= 37.880
Moving to projects directory: /home/ij2/slots/20/ICMGGh7zg+001560
Moving to projects directory: /home/ij2/slots/20/ICMSHh7zg+001560
Moving to projects directory: /home/ij2/slots/20/ICMUAh7zg+001560
  05:07:01 STEP 1561 H= 390:15 +CPU= 27.427
  05:07:30 STEP 1562 H= 390:30 +CPU= 28.832
  05:07:59 STEP 1563 H= 390:45 +CPU= 28.690
  05:08:29 STEP 1564 H= 391:00 +CPU= 30.260
  05:08:59 STEP 1565 H= 391:15 +CPU= 29.234
  05:09:27 STEP 1566 H= 391:30 +CPU= 28.455
  05:09:56 STEP 1567 H= 391:45 +CPU= 28.816
  05:10:27 STEP 1568 H= 392:00 +CPU= 30.809
  05:10:57 STEP 1569 H= 392:15 +CPU= 29.748
  05:11:26 STEP 1570 H= 392:30 +CPU= 29.288
  05:11:56 STEP 1571 H= 392:45 +CPU= 29.694
  05:12:34 STEP 1572 H= 393:00 +CPU= 37.790
Moving to projects directory: /home/ij2/slots/20/ICMGGh7zg+001572
Moving to projects directory: /home/ij2/slots/20/ICMSHh7zg+001572
Moving to projects directory: /home/ij2/slots/20/ICMUAh7zg+001572
  05:13:04 STEP 1573 H= 393:15 +CPU= 30.216
  05:13:34 STEP 1574 H= 393:30 +CPU= 29.501
  05:14:04 STEP 1575 H= 393:45 +CPU= 29.911
  05:14:33 STEP 1576 H= 394:00 +CPU= 29.469
  05:15:02 STEP 1577 H= 394:15 +CPU= 28.900
  05:15:32 STEP 1578 H= 394:30 +CPU= 29.774
  05:16:02 STEP 1579 H= 394:45 +CPU= 29.612
  05:16:31 STEP 1580 H= 395:00 +CPU= 29.425
  05:17:01 STEP 1581 H= 395:15 +CPU= 29.697
  05:17:30 STEP 1582 H= 395:30 +CPU= 29.592
  05:17:59 STEP 1583 H= 395:45 +CPU= 28.606
  05:18:39 STEP 1584 H= 396:00 +CPU= 39.116
Moving to projects directory: /home/ij2/slots/20/ICMGGh7zg+001584
Moving to projects directory: /home/ij2/slots/20/ICMSHh7zg+001584
Moving to projects directory: /home/ij2/slots/20/ICMUAh7zg+001584
  05:19:08 STEP 1585 H= 396:15 +CPU= 29.157
  05:19:35 STEP 1586 H= 396:30 +CPU= 27.422
  05:20:01 STEP 1587 H= 396:45 +CPU= 25.985
  05:20:30 STEP 1588 H= 397:00 +CPU= 29.051
  05:21:01 STEP 1589 H= 397:15 +CPU= 30.071
  05:21:30 STEP 1590 H= 397:30 +CPU= 29.075
  05:21:58 STEP 1591 H= 397:45 +CPU= 28.751
  05:22:29 STEP 1592 H= 398:00 +CPU= 30.423
  05:22:58 STEP 1593 H= 398:15 +CPU= 29.399
  05:23:28 STEP 1594 H= 398:30 +CPU= 29.482
  05:23:57 STEP 1595 H= 398:45 +CPU= 29.277
  05:24:34 STEP 1596 H= 399:00 +CPU= 36.363
Moving to projects directory: /home/ij2/slots/20/ICMGGh7zg+001596
Moving to projects directory: /home/ij2/slots/20/ICMSHh7zg+001596
Moving to projects directory: /home/ij2/slots/20/ICMUAh7zg+001596
  05:25:01 STEP 1597 H= 399:15 +CPU= 27.463
  05:25:30 STEP 1598 H= 399:30 +CPU= 29.272
  05:26:00 STEP 1599 H= 399:45 +CPU= 29.359
  05:26:31 STEP 1600 H= 400:00 +CPU= 30.715
  05:27:01 STEP 1601 H= 400:15 +CPU= 30.246
  05:27:31 STEP 1602 H= 400:30 +CPU= 29.578
  05:27:58 STEP 1603 H= 400:45 +CPU= 27.150
  05:28:28 STEP 1604 H= 401:00 +CPU= 30.530
  05:28:58 STEP 1605 H= 401:15 +CPU= 29.604
  05:29:28 STEP 1606 H= 401:30 +CPU= 29.448
  05:29:57 STEP 1607 H= 401:45 +CPU= 29.656
  05:30:35 STEP 1608 H= 402:00 +CPU= 37.467
Moving to projects directory: /home/ij2/slots/20/ICMGGh7zg+001608
Moving to projects directory: /home/ij2/slots/20/ICMSHh7zg+001608
Moving to projects directory: /home/ij2/slots/20/ICMUAh7zg+001608
  05:31:05 STEP 1609 H= 402:15 +CPU= 30.006
  05:31:32 STEP 1610 H= 402:30 +CPU= 27.502
  05:32:02 STEP 1611 H= 402:45 +CPU= 29.746
  05:32:33 STEP 1612 H= 403:00 +CPU= 30.727
  05:33:01 STEP 1613 H= 403:15 +CPU= 27.637
  05:33:30 STEP 1614 H= 403:30 +CPU= 29.488
  05:33:59 STEP 1615 H= 403:45 +CPU= 28.403
  05:34:30 STEP 1616 H= 404:00 +CPU= 30.696
  05:34:59 STEP 1617 H= 404:15 +CPU= 29.676
  05:35:29 STEP 1618 H= 404:30 +CPU= 29.634
  05:35:59 STEP 1619 H= 404:45 +CPU= 29.945
  05:36:37 STEP 1620 H= 405:00 +CPU= 37.715
Moving to projects directory: /home/ij2/slots/20/ICMGGh7zg+001620
Moving to projects directory: /home/ij2/slots/20/ICMSHh7zg+001620
Moving to projects directory: /home/ij2/slots/20/ICMUAh7zg+001620
  05:37:07 STEP 1621 H= 405:15 +CPU= 29.952
  05:37:36 STEP 1622 H= 405:30 +CPU= 29.513
  05:38:06 STEP 1623 H= 405:45 +CPU= 29.570
  05:38:35 STEP 1624 H= 406:00 +CPU= 29.343
  05:39:04 STEP 1625 H= 406:15 +CPU= 28.819
  05:39:34 STEP 1626 H= 406:30 +CPU= 29.768
  05:40:04 STEP 1627 H= 406:45 +CPU= 29.827
  05:40:34 STEP 1628 H= 407:00 +CPU= 29.790
  05:41:02 STEP 1629 H= 407:15 +CPU= 28.752
  05:41:29 STEP 1630 H= 407:30 +CPU= 26.478
  05:41:54 STEP 1631 H= 407:45 +CPU= 25.157
  05:42:36 STEP 1632 H= 408:00 +CPU= 41.461
Moving to projects directory: /home/ij2/slots/20/ICMGGh7zg+001632
Moving to projects directory: /home/ij2/slots/20/ICMSHh7zg+001632
Moving to projects directory: /home/ij2/slots/20/ICMUAh7zg+001632
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMGGh7zg+001536
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMSHh7zg+001536
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMUAh7zg+001536
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMGGh7zg+001548
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMSHh7zg+001548
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMUAh7zg+001548
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMGGh7zg+001560
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMSHh7zg+001560
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMUAh7zg+001560
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMGGh7zg+001572
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMSHh7zg+001572
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMUAh7zg+001572
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMGGh7zg+001584
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMSHh7zg+001584
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMUAh7zg+001584
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMGGh7zg+001596
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMSHh7zg+001596
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMUAh7zg+001596
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMGGh7zg+001608
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMSHh7zg+001608
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMUAh7zg+001608
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMGGh7zg+001620
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMSHh7zg+001620
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMUAh7zg+001620
Zipping up the intermediate file: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_a321_2016092300_20_1018_12292288_1_r1929080527_16.zip
Uploading the intermediate file: upload_file_16.zip
  05:43:06 STEP 1633 H= 408:15 +CPU= 29.657
  05:43:34 STEP 1634 H= 408:30 +CPU= 28.190
  05:44:02 STEP 1635 H= 408:45 +CPU= 27.618
  05:44:33 STEP 1636 H= 409:00 +CPU= 30.867
  05:45:01 STEP 1637 H= 409:15 +CPU= 27.893
  05:45:29 STEP 1638 H= 409:30 +CPU= 28.589
  05:45:59 STEP 1639 H= 409:45 +CPU= 29.538
  05:46:28 STEP 1640 H= 410:00 +CPU= 29.102
  05:46:56 STEP 1641 H= 410:15 +CPU= 27.971
  05:47:26 STEP 1642 H= 410:30 +CPU= 29.645
  05:47:55 STEP 1643 H= 410:45 +CPU= 29.726
  05:48:34 STEP 1644 H= 411:00 +CPU= 38.281
Moving to projects directory: /home/ij2/slots/20/ICMGGh7zg+001644
Moving to projects directory: /home/ij2/slots/20/ICMSHh7zg+001644
Moving to projects directory: /home/ij2/slots/20/ICMUAh7zg+001644
  05:49:03 STEP 1645 H= 411:15 +CPU= 28.897
  05:49:32 STEP 1646 H= 411:30 +CPU= 29.226
  05:49:59 STEP 1647 H= 411:45 +CPU= 27.214
  05:50:26 STEP 1648 H= 412:00 +CPU= 26.588
  05:50:56 STEP 1649 H= 412:15 +CPU= 30.324
  05:51:23 STEP 1650 H= 412:30 +CPU= 26.307
  05:51:52 STEP 1651 H= 412:45 +CPU= 29.773
  05:52:22 STEP 1652 H= 413:00 +CPU= 29.822
  05:52:51 STEP 1653 H= 413:15 +CPU= 28.288
  05:53:18 STEP 1654 H= 413:30 +CPU= 27.577
  05:53:47 STEP 1655 H= 413:45 +CPU= 28.456
  05:54:23 STEP 1656 H= 414:00 +CPU= 36.258
Moving to projects directory: /home/ij2/slots/20/ICMGGh7zg+001656
Moving to projects directory: /home/ij2/slots/20/ICMSHh7zg+001656
Moving to projects directory: /home/ij2/slots/20/ICMUAh7zg+001656
  05:54:52 STEP 1657 H= 414:15 +CPU= 28.477
  05:55:21 STEP 1658 H= 414:30 +CPU= 29.649
  05:55:51 STEP 1659 H= 414:45 +CPU= 29.399
  05:56:21 STEP 1660 H= 415:00 +CPU= 30.101
  05:56:46 STEP 1661 H= 415:15 +CPU= 24.466
  05:57:13 STEP 1662 H= 415:30 +CPU= 27.300
  05:57:40 STEP 1663 H= 415:45 +CPU= 27.329
  05:58:11 STEP 1664 H= 416:00 +CPU= 30.383
  05:58:40 STEP 1665 H= 416:15 +CPU= 29.528
  05:59:10 STEP 1666 H= 416:30 +CPU= 29.777
  05:59:40 STEP 1667 H= 416:45 +CPU= 29.533
  06:00:17 STEP 1668 H= 417:00 +CPU= 37.471
Moving to projects directory: /home/ij2/slots/20/ICMGGh7zg+001668
Moving to projects directory: /home/ij2/slots/20/ICMSHh7zg+001668
Moving to projects directory: /home/ij2/slots/20/ICMUAh7zg+001668
  06:00:47 STEP 1669 H= 417:15 +CPU= 29.252
  06:01:17 STEP 1670 H= 417:30 +CPU= 29.817
  06:01:45 STEP 1671 H= 417:45 +CPU= 28.039
  06:02:11 STEP 1672 H= 418:00 +CPU= 26.706
  06:02:41 STEP 1673 H= 418:15 +CPU= 29.283
  06:03:08 STEP 1674 H= 418:30 +CPU= 27.272
  06:03:36 STEP 1675 H= 418:45 +CPU= 27.439
  06:04:04 STEP 1676 H= 419:00 +CPU= 28.473
  06:04:33 STEP 1677 H= 419:15 +CPU= 28.912
  06:05:02 STEP 1678 H= 419:30 +CPU= 28.843
  06:05:30 STEP 1679 H= 419:45 +CPU= 28.511
  06:06:09 STEP 1680 H= 420:00 +CPU= 37.685
Moving to projects directory: /home/ij2/slots/20/ICMGGh7zg+001680
Moving to projects directory: /home/ij2/slots/20/ICMSHh7zg+001680
Moving to projects directory: /home/ij2/slots/20/ICMUAh7zg+001680
  06:06:38 STEP 1681 H= 420:15 +CPU= 29.447
  06:07:07 STEP 1682 H= 420:30 +CPU= 28.914
  06:07:37 STEP 1683 H= 420:45 +CPU= 29.664
  06:08:07 STEP 1684 H= 421:00 +CPU= 30.195
  06:08:37 STEP 1685 H= 421:15 +CPU= 29.656
  06:09:07 STEP 1686 H= 421:30 +CPU= 29.919
  06:09:37 STEP 1687 H= 421:45 +CPU= 30.022
  06:10:05 STEP 1688 H= 422:00 +CPU= 27.941
  06:10:34 STEP 1689 H= 422:15 +CPU= 29.256
  06:11:03 STEP 1690 H= 422:30 +CPU= 29.271
  06:11:33 STEP 1691 H= 422:45 +CPU= 29.579
  06:12:10 STEP 1692 H= 423:00 +CPU= 37.207
Moving to projects directory: /home/ij2/slots/20/ICMGGh7zg+001692
Moving to projects directory: /home/ij2/slots/20/ICMSHh7zg+001692
Moving to projects directory: /home/ij2/slots/20/ICMUAh7zg+001692
  06:12:40 STEP 1693 H= 423:15 +CPU= 29.258
  06:13:08 STEP 1694 H= 423:30 +CPU= 28.490
  06:13:35 STEP 1695 H= 423:45 +CPU= 26.730
  06:14:06 STEP 1696 H= 424:00 +CPU= 30.583
  06:14:35 STEP 1697 H= 424:15 +CPU= 29.775
  06:15:06 STEP 1698 H= 424:30 +CPU= 30.227
  06:15:35 STEP 1699 H= 424:45 +CPU= 28.757
  06:16:05 STEP 1700 H= 425:00 +CPU= 30.834
  06:16:35 STEP 1701 H= 425:15 +CPU= 29.165
  06:17:05 STEP 1702 H= 425:30 +CPU= 30.373
  06:17:34 STEP 1703 H= 425:45 +CPU= 29.004
  06:18:12 STEP 1704 H= 426:00 +CPU= 37.913
Moving to projects directory: /home/ij2/slots/20/ICMGGh7zg+001704
Moving to projects directory: /home/ij2/slots/20/ICMSHh7zg+001704
Moving to projects directory: /home/ij2/slots/20/ICMUAh7zg+001704
  06:18:42 STEP 1705 H= 426:15 +CPU= 29.601
  06:19:10 STEP 1706 H= 426:30 +CPU= 27.964
  06:19:38 STEP 1707 H= 426:45 +CPU= 28.214
  06:20:08 STEP 1708 H= 427:00 +CPU= 29.807
  06:20:37 STEP 1709 H= 427:15 +CPU= 28.683
  06:21:06 STEP 1710 H= 427:30 +CPU= 29.164
  06:21:35 STEP 1711 H= 427:45 +CPU= 29.351
  06:22:06 STEP 1712 H= 428:00 +CPU= 30.676
  06:22:34 STEP 1713 H= 428:15 +CPU= 27.431
  06:23:04 STEP 1714 H= 428:30 +CPU= 29.892
  06:23:30 STEP 1715 H= 428:45 +CPU= 26.781
  06:24:06 STEP 1716 H= 429:00 +CPU= 35.701
Moving to projects directory: /home/ij2/slots/20/ICMGGh7zg+001716
Moving to projects directory: /home/ij2/slots/20/ICMSHh7zg+001716
Moving to projects directory: /home/ij2/slots/20/ICMUAh7zg+001716
  06:24:35 STEP 1717 H= 429:15 +CPU= 28.826
  06:24:59 STEP 1718 H= 429:30 +CPU= 24.183
  06:25:26 STEP 1719 H= 429:45 +CPU= 26.219
  06:25:55 STEP 1720 H= 430:00 +CPU= 29.697
  06:26:26 STEP 1721 H= 430:15 +CPU= 30.199
  06:26:55 STEP 1722 H= 430:30 +CPU= 29.847
  06:27:26 STEP 1723 H= 430:45 +CPU= 30.125
  06:27:56 STEP 1724 H= 431:00 +CPU= 30.617
  06:28:27 STEP 1725 H= 431:15 +CPU= 30.133
  06:28:54 STEP 1726 H= 431:30 +CPU= 27.096
  06:29:19 STEP 1727 H= 431:45 +CPU= 25.349
  06:29:55 STEP 1728 H= 432:00 +CPU= 35.918
Moving to projects directory: /home/ij2/slots/20/ICMGGh7zg+001728
Moving to projects directory: /home/ij2/slots/20/ICMSHh7zg+001728
Moving to projects directory: /home/ij2/slots/20/ICMUAh7zg+001728
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMGGh7zg+001632
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMSHh7zg+001632
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMUAh7zg+001632
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMGGh7zg+001644
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMSHh7zg+001644
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMUAh7zg+001644
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMGGh7zg+001656
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMSHh7zg+001656
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMUAh7zg+001656
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMGGh7zg+001668
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMSHh7zg+001668
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMUAh7zg+001668
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMGGh7zg+001680
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMSHh7zg+001680
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMUAh7zg+001680
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMGGh7zg+001692
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMSHh7zg+001692
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMUAh7zg+001692
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMGGh7zg+001704
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMSHh7zg+001704
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMUAh7zg+001704
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMGGh7zg+001716
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMSHh7zg+001716
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMUAh7zg+001716
Zipping up the intermediate file: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_a321_2016092300_20_1018_12292288_1_r1929080527_17.zip
Uploading the intermediate file: upload_file_17.zip
  06:30:25 STEP 1729 H= 432:15 +CPU= 29.158
  06:30:54 STEP 1730 H= 432:30 +CPU= 29.612
  06:31:24 STEP 1731 H= 432:45 +CPU= 29.782
  06:31:55 STEP 1732 H= 433:00 +CPU= 31.028
  06:32:25 STEP 1733 H= 433:15 +CPU= 29.966
  06:32:54 STEP 1734 H= 433:30 +CPU= 28.864
  06:33:24 STEP 1735 H= 433:45 +CPU= 30.142
  06:33:53 STEP 1736 H= 434:00 +CPU= 28.454
  06:34:22 STEP 1737 H= 434:15 +CPU= 28.760
  06:34:49 STEP 1738 H= 434:30 +CPU= 27.472
  06:35:15 STEP 1739 H= 434:45 +CPU= 25.618
  06:35:51 STEP 1740 H= 435:00 +CPU= 36.218
Moving to projects directory: /home/ij2/slots/20/ICMGGh7zg+001740
Moving to projects directory: /home/ij2/slots/20/ICMSHh7zg+001740
Moving to projects directory: /home/ij2/slots/20/ICMUAh7zg+001740
  06:36:20 STEP 1741 H= 435:15 +CPU= 29.067
  06:36:49 STEP 1742 H= 435:30 +CPU= 28.410
  06:37:18 STEP 1743 H= 435:45 +CPU= 29.707
  06:37:49 STEP 1744 H= 436:00 +CPU= 30.576
  06:38:17 STEP 1745 H= 436:15 +CPU= 28.338
  06:38:47 STEP 1746 H= 436:30 +CPU= 29.649
  06:39:15 STEP 1747 H= 436:45 +CPU= 27.489
  06:39:44 STEP 1748 H= 437:00 +CPU= 29.692
  06:40:13 STEP 1749 H= 437:15 +CPU= 28.198
  06:40:38 STEP 1750 H= 437:30 +CPU= 25.687
  06:41:07 STEP 1751 H= 437:45 +CPU= 28.430
  06:41:42 STEP 1752 H= 438:00 +CPU= 34.721
Moving to projects directory: /home/ij2/slots/20/ICMGGh7zg+001752
Moving to projects directory: /home/ij2/slots/20/ICMSHh7zg+001752
Moving to projects directory: /home/ij2/slots/20/ICMUAh7zg+001752
  06:42:10 STEP 1753 H= 438:15 +CPU= 28.033
  06:42:40 STEP 1754 H= 438:30 +CPU= 29.797
  06:43:07 STEP 1755 H= 438:45 +CPU= 27.749
  06:43:38 STEP 1756 H= 439:00 +CPU= 30.936
  06:44:06 STEP 1757 H= 439:15 +CPU= 27.677
  06:44:34 STEP 1758 H= 439:30 +CPU= 27.879
  06:45:03 STEP 1759 H= 439:45 +CPU= 28.642
  06:45:33 STEP 1760 H= 440:00 +CPU= 30.209
  06:46:03 STEP 1761 H= 440:15 +CPU= 29.307
  06:46:32 STEP 1762 H= 440:30 +CPU= 29.524
  06:47:01 STEP 1763 H= 440:45 +CPU= 28.857
  06:47:34 STEP 1764 H= 441:00 +CPU= 33.329
Moving to projects directory: /home/ij2/slots/20/ICMGGh7zg+001764
Moving to projects directory: /home/ij2/slots/20/ICMSHh7zg+001764
Moving to projects directory: /home/ij2/slots/20/ICMUAh7zg+001764
  06:48:04 STEP 1765 H= 441:15 +CPU= 29.666
  06:48:31 STEP 1766 H= 441:30 +CPU= 27.261
  06:49:01 STEP 1767 H= 441:45 +CPU= 30.055
  06:49:31 STEP 1768 H= 442:00 +CPU= 29.743
  06:49:58 STEP 1769 H= 442:15 +CPU= 27.006
  06:50:28 STEP 1770 H= 442:30 +CPU= 29.791
  06:50:56 STEP 1771 H= 442:45 +CPU= 27.302
  06:51:26 STEP 1772 H= 443:00 +CPU= 30.740
  06:51:56 STEP 1773 H= 443:15 +CPU= 29.850
  06:52:26 STEP 1774 H= 443:30 +CPU= 29.811
  06:52:56 STEP 1775 H= 443:45 +CPU= 29.647
  06:53:34 STEP 1776 H= 444:00 +CPU= 38.280
Moving to projects directory: /home/ij2/slots/20/ICMGGh7zg+001776
Moving to projects directory: /home/ij2/slots/20/ICMSHh7zg+001776
Moving to projects directory: /home/ij2/slots/20/ICMUAh7zg+001776
  06:54:04 STEP 1777 H= 444:15 +CPU= 29.593
  06:54:28 STEP 1778 H= 444:30 +CPU= 24.058
  06:54:57 STEP 1779 H= 444:45 +CPU= 28.492
  06:55:26 STEP 1780 H= 445:00 +CPU= 28.881
  06:55:56 STEP 1781 H= 445:15 +CPU= 29.826
  06:56:24 STEP 1782 H= 445:30 +CPU= 28.260
  06:56:53 STEP 1783 H= 445:45 +CPU= 29.316
  06:57:24 STEP 1784 H= 446:00 +CPU= 30.535
  06:57:54 STEP 1785 H= 446:15 +CPU= 29.681
  06:58:22 STEP 1786 H= 446:30 +CPU= 28.131
  06:58:50 STEP 1787 H= 446:45 +CPU= 28.035
  06:59:28 STEP 1788 H= 447:00 +CPU= 37.762
Moving to projects directory: /home/ij2/slots/20/ICMGGh7zg+001788
Moving to projects directory: /home/ij2/slots/20/ICMSHh7zg+001788
Moving to projects directory: /home/ij2/slots/20/ICMUAh7zg+001788
  06:59:57 STEP 1789 H= 447:15 +CPU= 29.526
  07:00:27 STEP 1790 H= 447:30 +CPU= 30.055
  07:00:56 STEP 1791 H= 447:45 +CPU= 28.434
  07:01:26 STEP 1792 H= 448:00 +CPU= 30.269
  07:01:55 STEP 1793 H= 448:15 +CPU= 29.170
  07:02:25 STEP 1794 H= 448:30 +CPU= 29.565
  07:02:55 STEP 1795 H= 448:45 +CPU= 29.827
  07:03:25 STEP 1796 H= 449:00 +CPU= 30.242
  07:03:56 STEP 1797 H= 449:15 +CPU= 30.278
  07:04:21 STEP 1798 H= 449:30 +CPU= 25.551
  07:04:48 STEP 1799 H= 449:45 +CPU= 26.813
  07:05:25 STEP 1800 H= 450:00 +CPU= 36.796
Moving to projects directory: /home/ij2/slots/20/ICMGGh7zg+001800
Moving to projects directory: /home/ij2/slots/20/ICMSHh7zg+001800
Moving to projects directory: /home/ij2/slots/20/ICMUAh7zg+001800
  07:05:54 STEP 1801 H= 450:15 +CPU= 28.810
  07:06:22 STEP 1802 H= 450:30 +CPU= 27.771
  07:06:48 STEP 1803 H= 450:45 +CPU= 26.318
  07:07:18 STEP 1804 H= 451:00 +CPU= 30.091
  07:07:48 STEP 1805 H= 451:15 +CPU= 29.850
  07:08:18 STEP 1806 H= 451:30 +CPU= 29.509
  07:08:45 STEP 1807 H= 451:45 +CPU= 27.511
  07:09:16 STEP 1808 H= 452:00 +CPU= 30.368
  07:09:44 STEP 1809 H= 452:15 +CPU= 28.767
  07:10:14 STEP 1810 H= 452:30 +CPU= 29.667
  07:10:42 STEP 1811 H= 452:45 +CPU= 28.109
  07:11:19 STEP 1812 H= 453:00 +CPU= 37.068
Moving to projects directory: /home/ij2/slots/20/ICMGGh7zg+001812
Moving to projects directory: /home/ij2/slots/20/ICMSHh7zg+001812
Moving to projects directory: /home/ij2/slots/20/ICMUAh7zg+001812
  07:11:46 STEP 1813 H= 453:15 +CPU= 26.686
  07:12:16 STEP 1814 H= 453:30 +CPU= 29.289
  07:12:46 STEP 1815 H= 453:45 +CPU= 30.110
  07:13:17 STEP 1816 H= 454:00 +CPU= 30.984
  07:13:46 STEP 1817 H= 454:15 +CPU= 29.495
  07:14:15 STEP 1818 H= 454:30 +CPU= 28.246
  07:14:43 STEP 1819 H= 454:45 +CPU= 28.008
  07:15:12 STEP 1820 H= 455:00 +CPU= 29.803
  07:15:42 STEP 1821 H= 455:15 +CPU= 29.236
  07:16:12 STEP 1822 H= 455:30 +CPU= 29.927
  07:16:42 STEP 1823 H= 455:45 +CPU= 29.865
  07:17:23 STEP 1824 H= 456:00 +CPU= 40.857
Moving to projects directory: /home/ij2/slots/20/ICMGGh7zg+001824
Moving to projects directory: /home/ij2/slots/20/ICMSHh7zg+001824
Moving to projects directory: /home/ij2/slots/20/ICMUAh7zg+001824
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMGGh7zg+001728
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMSHh7zg+001728
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMUAh7zg+001728
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMGGh7zg+001740
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMSHh7zg+001740
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMUAh7zg+001740
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMGGh7zg+001752
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMSHh7zg+001752
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMUAh7zg+001752
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMGGh7zg+001764
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMSHh7zg+001764
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMUAh7zg+001764
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMGGh7zg+001776
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMSHh7zg+001776
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMUAh7zg+001776
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMGGh7zg+001788
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMSHh7zg+001788
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMUAh7zg+001788
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMGGh7zg+001800
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMSHh7zg+001800
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMUAh7zg+001800
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMGGh7zg+001812
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMSHh7zg+001812
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMUAh7zg+001812
Zipping up the intermediate file: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_a321_2016092300_20_1018_12292288_1_r1929080527_18.zip
Uploading the intermediate file: upload_file_18.zip
  07:17:51 STEP 1825 H= 456:15 +CPU= 27.993
  07:18:21 STEP 1826 H= 456:30 +CPU= 30.355
  07:18:51 STEP 1827 H= 456:45 +CPU= 29.815
  07:19:19 STEP 1828 H= 457:00 +CPU= 27.914
  07:19:49 STEP 1829 H= 457:15 +CPU= 29.512
  07:20:15 STEP 1830 H= 457:30 +CPU= 26.830
  07:20:43 STEP 1831 H= 457:45 +CPU= 27.146
  07:21:13 STEP 1832 H= 458:00 +CPU= 30.375
  07:21:43 STEP 1833 H= 458:15 +CPU= 30.013
  07:22:13 STEP 1834 H= 458:30 +CPU= 29.816
  07:22:41 STEP 1835 H= 458:45 +CPU= 27.530
  07:23:19 STEP 1836 H= 459:00 +CPU= 38.520
Moving to projects directory: /home/ij2/slots/20/ICMGGh7zg+001836
Moving to projects directory: /home/ij2/slots/20/ICMSHh7zg+001836
Moving to projects directory: /home/ij2/slots/20/ICMUAh7zg+001836
  07:23:49 STEP 1837 H= 459:15 +CPU= 29.961
  07:24:19 STEP 1838 H= 459:30 +CPU= 29.632
  07:24:48 STEP 1839 H= 459:45 +CPU= 29.204
  07:25:19 STEP 1840 H= 460:00 +CPU= 30.443
  07:25:45 STEP 1841 H= 460:15 +CPU= 26.710
  07:26:13 STEP 1842 H= 460:30 +CPU= 27.231
  07:26:42 STEP 1843 H= 460:45 +CPU= 29.488
  07:27:12 STEP 1844 H= 461:00 +CPU= 30.274
  07:27:42 STEP 1845 H= 461:15 +CPU= 29.896
  07:28:12 STEP 1846 H= 461:30 +CPU= 30.003
  07:28:41 STEP 1847 H= 461:45 +CPU= 28.399
  07:29:17 STEP 1848 H= 462:00 +CPU= 36.274
Moving to projects directory: /home/ij2/slots/20/ICMGGh7zg+001848
Moving to projects directory: /home/ij2/slots/20/ICMSHh7zg+001848
Moving to projects directory: /home/ij2/slots/20/ICMUAh7zg+001848
  07:29:47 STEP 1849 H= 462:15 +CPU= 30.082
  07:30:17 STEP 1850 H= 462:30 +CPU= 29.141
  07:30:47 STEP 1851 H= 462:45 +CPU= 29.948
  07:31:17 STEP 1852 H= 463:00 +CPU= 30.394
  07:31:45 STEP 1853 H= 463:15 +CPU= 27.706
  07:32:11 STEP 1854 H= 463:30 +CPU= 26.569
  07:32:40 STEP 1855 H= 463:45 +CPU= 28.738
  07:33:10 STEP 1856 H= 464:00 +CPU= 29.888
  07:33:37 STEP 1857 H= 464:15 +CPU= 27.097
  07:34:06 STEP 1858 H= 464:30 +CPU= 28.882
  07:34:36 STEP 1859 H= 464:45 +CPU= 30.029
  07:35:08 STEP 1860 H= 465:00 +CPU= 31.634
Moving to projects directory: /home/ij2/slots/20/ICMGGh7zg+001860
Moving to projects directory: /home/ij2/slots/20/ICMSHh7zg+001860
Moving to projects directory: /home/ij2/slots/20/ICMUAh7zg+001860
  07:35:38 STEP 1861 H= 465:15 +CPU= 29.808
  07:36:07 STEP 1862 H= 465:30 +CPU= 29.315
  07:36:36 STEP 1863 H= 465:45 +CPU= 29.174
  07:37:07 STEP 1864 H= 466:00 +CPU= 30.565
  07:37:35 STEP 1865 H= 466:15 +CPU= 27.454
  07:38:04 STEP 1866 H= 466:30 +CPU= 29.924
  07:38:34 STEP 1867 H= 466:45 +CPU= 29.097
  07:38:59 STEP 1868 H= 467:00 +CPU= 25.407
  07:39:28 STEP 1869 H= 467:15 +CPU= 28.920
  07:39:58 STEP 1870 H= 467:30 +CPU= 30.319
  07:40:28 STEP 1871 H= 467:45 +CPU= 29.894
  07:41:07 STEP 1872 H= 468:00 +CPU= 38.114
Moving to projects directory: /home/ij2/slots/20/ICMGGh7zg+001872
Moving to projects directory: /home/ij2/slots/20/ICMSHh7zg+001872
Moving to projects directory: /home/ij2/slots/20/ICMUAh7zg+001872
  07:41:36 STEP 1873 H= 468:15 +CPU= 29.444
  07:42:02 STEP 1874 H= 468:30 +CPU= 25.453
  07:42:28 STEP 1875 H= 468:45 +CPU= 26.663
  07:42:58 STEP 1876 H= 469:00 +CPU= 29.352
  07:43:24 STEP 1877 H= 469:15 +CPU= 25.941
  07:43:52 STEP 1878 H= 469:30 +CPU= 28.470
  07:44:20 STEP 1879 H= 469:45 +CPU= 27.654
  07:44:49 STEP 1880 H= 470:00 +CPU= 28.629
  07:45:18 STEP 1881 H= 470:15 +CPU= 29.710
  07:45:46 STEP 1882 H= 470:30 +CPU= 27.819
  07:46:16 STEP 1883 H= 470:45 +CPU= 29.860
  07:46:51 STEP 1884 H= 471:00 +CPU= 35.130
Moving to projects directory: /home/ij2/slots/20/ICMGGh7zg+001884
Moving to projects directory: /home/ij2/slots/20/ICMSHh7zg+001884
Moving to projects directory: /home/ij2/slots/20/ICMUAh7zg+001884
  07:47:22 STEP 1885 H= 471:15 +CPU= 30.367
  07:47:51 STEP 1886 H= 471:30 +CPU= 29.702
  07:48:20 STEP 1887 H= 471:45 +CPU= 28.814
  07:48:51 STEP 1888 H= 472:00 +CPU= 30.366
  07:49:21 STEP 1889 H= 472:15 +CPU= 29.895
  07:49:51 STEP 1890 H= 472:30 +CPU= 29.960
  07:50:21 STEP 1891 H= 472:45 +CPU= 29.929
  07:50:51 STEP 1892 H= 473:00 +CPU= 30.092
  07:51:21 STEP 1893 H= 473:15 +CPU= 29.908
  07:51:51 STEP 1894 H= 473:30 +CPU= 29.950
  07:52:21 STEP 1895 H= 473:45 +CPU= 29.902
  07:52:59 STEP 1896 H= 474:00 +CPU= 37.930
Moving to projects directory: /home/ij2/slots/20/ICMGGh7zg+001896
Moving to projects directory: /home/ij2/slots/20/ICMSHh7zg+001896
Moving to projects directory: /home/ij2/slots/20/ICMUAh7zg+001896
  07:53:28 STEP 1897 H= 474:15 +CPU= 29.155
  07:53:58 STEP 1898 H= 474:30 +CPU= 29.821
  07:54:28 STEP 1899 H= 474:45 +CPU= 29.818
  07:54:58 STEP 1900 H= 475:00 +CPU= 29.869
  07:55:27 STEP 1901 H= 475:15 +CPU= 28.898
  07:55:57 STEP 1902 H= 475:30 +CPU= 30.094
  07:56:26 STEP 1903 H= 475:45 +CPU= 29.012
  07:56:56 STEP 1904 H= 476:00 +CPU= 29.487
  07:57:25 STEP 1905 H= 476:15 +CPU= 29.182
  07:57:55 STEP 1906 H= 476:30 +CPU= 29.682
  07:58:25 STEP 1907 H= 476:45 +CPU= 29.937
  07:59:03 STEP 1908 H= 477:00 +CPU= 38.015
Moving to projects directory: /home/ij2/slots/20/ICMGGh7zg+001908
Moving to projects directory: /home/ij2/slots/20/ICMSHh7zg+001908
Moving to projects directory: /home/ij2/slots/20/ICMUAh7zg+001908
  07:59:33 STEP 1909 H= 477:15 +CPU= 29.843
  08:00:02 STEP 1910 H= 477:30 +CPU= 29.834
  08:00:32 STEP 1911 H= 477:45 +CPU= 29.831
  08:01:03 STEP 1912 H= 478:00 +CPU= 30.676
  08:01:33 STEP 1913 H= 478:15 +CPU= 29.845
  08:02:03 STEP 1914 H= 478:30 +CPU= 29.805
  08:02:32 STEP 1915 H= 478:45 +CPU= 29.586
  08:03:03 STEP 1916 H= 479:00 +CPU= 30.686
  08:03:33 STEP 1917 H= 479:15 +CPU= 29.786
  08:04:03 STEP 1918 H= 479:30 +CPU= 29.714
  08:04:32 STEP 1919 H= 479:45 +CPU= 29.343
  08:05:11 STEP 1920 H= 480:00 +CPU= 38.719
..The child process terminated with status: 0
>>> Printing last 70 lines from file: NODE.001_01

I   VARIABLE   I      Initial value         I        Current value        I
I-------------------------------------------------------------------------I
I  OMEGA (P/S) I      0.1123745842E-21      I       0.9672388055E-05      I
ICloud fractionI      0.0000000000E+00      I       0.4009530603E-01      I
I  Relat. Hum. I      0.3641902370E+00      I       0.4322772475E+00      I
I  PASS 01     I      0.0000000000E+00      I       0.6335823670E+03      I
DDH------------------------------------------------------------------------
  
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
 NSTEP =  1920 SCAN2M_HPOS  P
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ICMGGh7zg+001920 MODE=a
 IO-STREAM CLOSED - ICMGGh7zg+001920
 MPI-TASK:   1 -    302452676 BYTES IN      7 RECORDS TRANSFERRED IN    0.0004 SECONDS756131.7 Mbytes/s, TOTAL TIME=    0.0128( 3.1%)
 08:05:11 STEP 1920 H= 480:00 +CPU= 38.719
 END CNT3
 NSTEP =  1920 CNT0   000000000  28.182  28.182  28.265   0.997               0               0               0               0               0
IO-STREAM STATISTICS, TOTAL NO OF REC/BYTES READ            0/           0 WRITTEN        57546/************
 -TASK-OPENED-OPEN-RECS IN -KBYTE IN    -RECS OUT-KBYTE OUT-WALL   -WALL IN-WALL
  OU-TOT IN -TOT OUT
   1   1008    0        0            0    57546 ************   103.8     0.0   103.7     0.0   225.8
===-=== START OF TIMING STATISTICS ===-===
 
STATS FOR ALL TASKS
 NUM ROUTINE                                     CALLS  MEAN(ms)   MAX(ms)   FRAC(%)  UNBAL(%)
   0 CNT0     - COMPLETE EXECUTION                   1 ********* *********    100.00      0.00
   1 CNT4     - FORWARD INTEGRATION                  1 ********* *********     99.99      0.00
   8 SCAN2M - GRID-POINT DYNAMICS                 2082    4579.7    4579.7     16.87      0.00
   9 SPCM     - SPECTRAL COMP.                    1920     664.4     664.4      2.26      0.00
  10 SCAN2M - PHYSICS                             1921   14045.1   14045.1     47.74      0.00
  11 IOPACK   - OUTPUT P.P. RESULTS                161     874.7     874.7      0.25      0.00
  12 SPNORM   - SPECTRAL NORM COMP.                 83      41.8      41.8      0.01      0.00
  14 SUINIF                                          1    6587.5    6587.5      0.01      0.00
  17 GRIDFPOS IN CNT4                              161      29.0      29.0      0.01      0.00
  18 SUSPECG                                         1    3387.4    3387.4      0.01      0.00
  19 SUSPEC                                          1    3397.4    3397.4      0.01      0.00
  24 SUGRIDU                                         1    2545.0    2545.0      0.00      0.00
  25 SPECRT                                          1     279.6     279.6      0.00      0.00
  26 SUGRIDF                                         1     365.3     365.3      0.00      0.00
  27 RESTART FILES - WRITING                        40    3520.7    3520.7      0.25      0.00
  28 RESTART FILES - READING                         1       0.0       0.0      0.00      0.00
  29 SU4FPOS IN CNT4                               161       0.1       0.1      0.00      0.00
  30 DYNFPOS IN CNT4                               161    6876.1    6876.1      1.96      0.00
  31 POSDDH IN STEPO                                 9       4.7       4.7      0.00      0.00
  37 CPGLAG   - SL COMPUTATIONS                   1921    6328.0    6328.0     21.51      0.00
  39 SU0YOMB                                         1     767.2     767.2      0.00      0.00
  51 SCAN2M   - SL COMM. PART 1                   1921      22.4      22.4      0.08      0.00
  54 SPCM     - M TO S/S TO M TRANSP.             1920     176.7     176.7      0.60      0.00
  55 SPCIMPF  - S TO M/M TO S TRANSP.             1920      53.9      53.9      0.18      0.00
  56 SPNORM   - SPECTRAL NORM COMM.                 83       0.2       0.2      0.00      0.00
 102 LTINV_CTL   - INVERSE LEGENDRE TRANSFORM     4165     426.0     426.0      3.14      0.00
 103 LTDIR_CTL   - DIRECT LEGENDRE TRANSFORM      4006     265.0     265.0      1.88      0.00
 106 FTDIR_CTL   - DIRECT FOURIER TRANSFORM       4006     129.3     129.3      0.92      0.00
 107 FTINV_CTL   - INVERSE FOURIER TRANSFORM      4165     141.5     141.5      1.04      0.00
 140 SULEG       - COMP. OF LEGENDRE POL.            1      61.1      61.1      0.00      0.00
 152 LTINV_CTL   - M TO L TRANSPOSITION           4165      76.3      76.3      0.56      0.00
 153 LTDIR_CTL   - L TO M TRANSPOSITION           4006      35.8      35.8      0.25      0.00
 157 FTINV_CTL   - L TO G TRANSPOSITION           4165      78.5      78.5      0.58      0.00
 158 FTDIR_CTL   - G TO L TRANSPOSITION           4006      37.9      37.9      0.27      0.00
 400 GSTATS                                     223224       0.0       0.0      0.00      0.00
 401 GSTATS HOOK                                207696       0.0       0.0      0.00      0.00
TOTAL MEASURED IMBALANCE =       0.0 SECONDS,  0.0 PERCENT
TOTAL WALLCLOCK TIME    56511.6 CPU TIME   56396.7 VECTOR TIME    56396.7


===-=== END   OF TIMING STATISTICS ===-===
FORECAST DAYS PER DAY       30.6
  *** END CNT0 *** 
------------------------------------------------
Moving to projects directory: /home/ij2/slots/20/ICMGGh7zg+001920
Moving to projects directory: /home/ij2/slots/20/ICMSHh7zg+001920
Moving to projects directory: /home/ij2/slots/20/ICMUAh7zg+001920
Adding to the zip: /home/ij2/slots/20/NODE.001_01
Adding to the zip: /home/ij2/slots/20/ifs.stat
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMUAh7zg+001920
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMSHh7zg+001920
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMGGh7zg+001920
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMUAh7zg+001872
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMUAh7zg+001908
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMUAh7zg+001896
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMUAh7zg+001824
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMGGh7zg+001908
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMGGh7zg+001872
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMGGh7zg+001896
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMSHh7zg+001896
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMSHh7zg+001872
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMSHh7zg+001908
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMGGh7zg+001824
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMSHh7zg+001824
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMSHh7zg+001848
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMGGh7zg+001848
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMUAh7zg+001848
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMGGh7zg+001860
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMGGh7zg+001884
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMSHh7zg+001884
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMSHh7zg+001860
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMGGh7zg+001836
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMSHh7zg+001836
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMUAh7zg+001860
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMUAh7zg+001884
Adding to the zip: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_12292288/ICMUAh7zg+001836
Zipping up the final file: /home/ij2/projects/climateprediction.net/oifs_43r3_bl_a321_2016092300_20_1018_12292288_1_r1929080527_19.zip
Uploading the final file: upload_file_19.zip
Uploading trickle at timestep: 1727100
08:08:48 (682): called boinc_finish(0)

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
17 Jun 2024 06:52:05 1549780 22449462 oifs_43r3_bl_a321_2016092300_20_1018_12292288_1 1,727,100 56,396 0.0327
16 Jun 2024 22:39:27 1549780 22449462 oifs_43r3_bl_a321_2016092300_20_1018_12292288_1 864,000 27,946 0.0323


©2024 cpdn.org