climateprediction.net home page
Task 22263367

Task 22263367

Name oifs_43r3_ps_0859_1992050100_123_961_12178503_0
Workunit 12178503
Created 22 Dec 2022, 1:36:12 UTC
Sent 26 Dec 2022, 10:50:29 UTC
Report deadline 25 Jan 2023, 10:50:29 UTC
Received 27 Dec 2022, 13:21:29 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 1 (0x00000001) Unknown error code
Computer ID 1504164
Run time 15 hours 48 min 42 sec
CPU time 13 hours 45 min 54 sec
Validate state Invalid
Credit 10,281.60
Device peak FLOPS 1.00 GFLOPS
Application version OpenIFS 43r3 Perturbed Surface v1.05
x86_64-pc-linux-gnu
Peak working set size 4,064.34 MB
Peak swap size 4,978.06 MB
Peak disk usage 4,176.72 MB
Stderr
<core_client_version>7.16.6</core_client_version>
<![CDATA[
<message>
process exited with code 1 (0x1, -255)</message>
<stderr_txt>
imateprediction.net/oifs_43r3_ps_12178503/ICMGGhq0f+001356
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMSHhq0f+001356
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMUAhq0f+001356
Zipping up the intermediate file: /data/projects/climateprediction.net/oifs_43r3_ps_0859_1992050100_123_961_12178503_0_r1860973431_56.zip
Uploading the intermediate file: upload_file_56.zip
  04:34:46 STEP 1369 H=1369:00 +CPU= 19.827
  04:35:09 STEP 1370 H=1370:00 +CPU= 21.340
  04:35:41 STEP 1371 H=1371:00 +CPU= 31.512
  04:36:03 STEP 1372 H=1372:00 +CPU= 20.420
  04:36:26 STEP 1373 H=1373:00 +CPU= 21.187
  04:36:58 STEP 1374 H=1374:00 +CPU= 30.386
  04:37:21 STEP 1375 H=1375:00 +CPU= 21.603
  04:37:42 STEP 1376 H=1376:00 +CPU= 19.968
  04:38:17 STEP 1377 H=1377:00 +CPU= 29.839
  04:38:38 STEP 1378 H=1378:00 +CPU= 20.203
  04:39:00 STEP 1379 H=1379:00 +CPU= 19.652
  04:39:40 STEP 1380 H=1380:00 +CPU= 35.579
Moving to projects directory: /data/slots/2/ICMGGhq0f+001380
Moving to projects directory: /data/slots/2/ICMSHhq0f+001380
Moving to projects directory: /data/slots/2/ICMUAhq0f+001380
  04:40:03 STEP 1381 H=1381:00 +CPU= 19.925
  04:40:28 STEP 1382 H=1382:00 +CPU= 19.834
  04:41:01 STEP 1383 H=1383:00 +CPU= 29.972
  04:41:23 STEP 1384 H=1384:00 +CPU= 19.568
  04:41:43 STEP 1385 H=1385:00 +CPU= 19.195
  04:42:14 STEP 1386 H=1386:00 +CPU= 30.100
  04:42:34 STEP 1387 H=1387:00 +CPU= 19.337
  04:42:55 STEP 1388 H=1388:00 +CPU= 20.268
  04:43:27 STEP 1389 H=1389:00 +CPU= 29.912
  04:43:47 STEP 1390 H=1390:00 +CPU= 19.768
  04:44:19 STEP 1391 H=1391:00 +CPU= 20.553
  04:45:54 STEP 1392 H=1392:00 +CPU= 36.914
Moving to projects directory: /data/slots/2/ICMGGhq0f+001392
Moving to projects directory: /data/slots/2/ICMSHhq0f+001392
Moving to projects directory: /data/slots/2/ICMUAhq0f+001392
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMGGhq0f+001368
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMSHhq0f+001368
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMUAhq0f+001368
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMGGhq0f+001380
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMSHhq0f+001380
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMUAhq0f+001380
Zipping up the intermediate file: /data/projects/climateprediction.net/oifs_43r3_ps_0859_1992050100_123_961_12178503_0_r1860973431_57.zip
Uploading the intermediate file: upload_file_57.zip
  04:46:15 STEP 1393 H=1393:00 +CPU= 18.645
  04:46:36 STEP 1394 H=1394:00 +CPU= 19.298
  04:47:15 STEP 1395 H=1395:00 +CPU= 31.663
  04:47:53 STEP 1396 H=1396:00 +CPU= 19.807
  04:48:12 STEP 1397 H=1397:00 +CPU= 19.425
  04:48:45 STEP 1398 H=1398:00 +CPU= 30.609
  04:49:04 STEP 1399 H=1399:00 +CPU= 18.936
  04:49:25 STEP 1400 H=1400:00 +CPU= 19.592
  04:49:58 STEP 1401 H=1401:00 +CPU= 30.289
  04:50:18 STEP 1402 H=1402:00 +CPU= 19.294
  04:50:39 STEP 1403 H=1403:00 +CPU= 19.154
  04:51:14 STEP 1404 H=1404:00 +CPU= 33.412
Moving to projects directory: /data/slots/2/ICMGGhq0f+001404
Moving to projects directory: /data/slots/2/ICMSHhq0f+001404
Moving to projects directory: /data/slots/2/ICMUAhq0f+001404
  04:51:34 STEP 1405 H=1405:00 +CPU= 19.008
  04:51:55 STEP 1406 H=1406:00 +CPU= 19.622
  04:52:26 STEP 1407 H=1407:00 +CPU= 30.384
  04:52:46 STEP 1408 H=1408:00 +CPU= 19.382
  04:53:07 STEP 1409 H=1409:00 +CPU= 19.612
  04:53:40 STEP 1410 H=1410:00 +CPU= 31.643
  04:54:01 STEP 1411 H=1411:00 +CPU= 19.660
  04:54:24 STEP 1412 H=1412:00 +CPU= 20.494
  04:54:56 STEP 1413 H=1413:00 +CPU= 29.915
  04:55:17 STEP 1414 H=1414:00 +CPU= 20.250
  04:55:39 STEP 1415 H=1415:00 +CPU= 20.894
  04:56:19 STEP 1416 H=1416:00 +CPU= 36.571
Moving to projects directory: /data/slots/2/ICMGGhq0f+001416
Moving to projects directory: /data/slots/2/ICMSHhq0f+001416
Moving to projects directory: /data/slots/2/ICMUAhq0f+001416
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMGGhq0f+001392
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMSHhq0f+001392
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMUAhq0f+001392
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMGGhq0f+001404
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMSHhq0f+001404
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMUAhq0f+001404
Zipping up the intermediate file: /data/projects/climateprediction.net/oifs_43r3_ps_0859_1992050100_123_961_12178503_0_r1860973431_58.zip
Uploading the intermediate file: upload_file_58.zip
  04:56:42 STEP 1417 H=1417:00 +CPU= 18.855
  04:57:02 STEP 1418 H=1418:00 +CPU= 19.504
  04:57:37 STEP 1419 H=1419:00 +CPU= 31.210
  04:58:00 STEP 1420 H=1420:00 +CPU= 21.813
  04:58:21 STEP 1421 H=1421:00 +CPU= 19.923
  04:58:54 STEP 1422 H=1422:00 +CPU= 31.438
  04:59:18 STEP 1423 H=1423:00 +CPU= 21.261
  04:59:40 STEP 1424 H=1424:00 +CPU= 21.847
  05:00:29 STEP 1425 H=1425:00 +CPU= 32.926
  05:01:11 STEP 1426 H=1426:00 +CPU= 22.163
  05:01:46 STEP 1427 H=1427:00 +CPU= 20.063
  05:02:52 STEP 1428 H=1428:00 +CPU= 35.986
Moving to projects directory: /data/slots/2/ICMGGhq0f+001428
Moving to projects directory: /data/slots/2/ICMSHhq0f+001428
Moving to projects directory: /data/slots/2/ICMUAhq0f+001428
  05:03:34 STEP 1429 H=1429:00 +CPU= 21.584
  05:04:17 STEP 1430 H=1430:00 +CPU= 22.565
  05:04:50 STEP 1431 H=1431:00 +CPU= 32.770
  05:05:12 STEP 1432 H=1432:00 +CPU= 20.859
  05:05:34 STEP 1433 H=1433:00 +CPU= 20.202
  05:06:09 STEP 1434 H=1434:00 +CPU= 31.954
  05:06:31 STEP 1435 H=1435:00 +CPU= 21.204
  05:06:52 STEP 1436 H=1436:00 +CPU= 20.575
  05:07:25 STEP 1437 H=1437:00 +CPU= 29.920
  05:07:45 STEP 1438 H=1438:00 +CPU= 19.479
  05:08:06 STEP 1439 H=1439:00 +CPU= 19.620
  05:08:52 STEP 1440 H=1440:00 +CPU= 35.653
Moving to projects directory: /data/slots/2/ICMGGhq0f+001440
Moving to projects directory: /data/slots/2/ICMSHhq0f+001440
Moving to projects directory: /data/slots/2/ICMUAhq0f+001440
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMGGhq0f+001416
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMSHhq0f+001416
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMUAhq0f+001416
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMGGhq0f+001428
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMSHhq0f+001428
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMUAhq0f+001428
Zipping up the intermediate file: /data/projects/climateprediction.net/oifs_43r3_ps_0859_1992050100_123_961_12178503_0_r1860973431_59.zip
Uploading the intermediate file: upload_file_59.zip
  05:09:13 STEP 1441 H=1441:00 +CPU= 18.553
  05:09:33 STEP 1442 H=1442:00 +CPU= 19.556
  05:10:05 STEP 1443 H=1443:00 +CPU= 29.808
  05:10:26 STEP 1444 H=1444:00 +CPU= 19.372
  05:10:47 STEP 1445 H=1445:00 +CPU= 18.905
  05:11:18 STEP 1446 H=1446:00 +CPU= 30.150
  05:11:38 STEP 1447 H=1447:00 +CPU= 18.873
  05:11:58 STEP 1448 H=1448:00 +CPU= 18.928
  05:12:28 STEP 1449 H=1449:00 +CPU= 28.845
  05:12:48 STEP 1450 H=1450:00 +CPU= 18.825
  05:13:06 STEP 1451 H=1451:00 +CPU= 18.474
  05:13:40 STEP 1452 H=1452:00 +CPU= 31.515
Moving to projects directory: /data/slots/2/ICMGGhq0f+001452
Moving to projects directory: /data/slots/2/ICMSHhq0f+001452
Moving to projects directory: /data/slots/2/ICMUAhq0f+001452
  05:13:59 STEP 1453 H=1453:00 +CPU= 18.476
  05:14:20 STEP 1454 H=1454:00 +CPU= 19.005
  05:14:51 STEP 1455 H=1455:00 +CPU= 28.813
  05:15:11 STEP 1456 H=1456:00 +CPU= 19.003
  05:15:31 STEP 1457 H=1457:00 +CPU= 18.270
  05:16:02 STEP 1458 H=1458:00 +CPU= 29.295
  05:16:22 STEP 1459 H=1459:00 +CPU= 18.408
  05:16:44 STEP 1460 H=1460:00 +CPU= 18.800
  05:17:17 STEP 1461 H=1461:00 +CPU= 30.102
  05:17:37 STEP 1462 H=1462:00 +CPU= 18.980
  05:17:57 STEP 1463 H=1463:00 +CPU= 18.682
  05:18:47 STEP 1464 H=1464:00 +CPU= 34.553
Moving to projects directory: /data/slots/2/ICMGGhq0f+001464
Moving to projects directory: /data/slots/2/ICMSHhq0f+001464
Moving to projects directory: /data/slots/2/ICMUAhq0f+001464
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMGGhq0f+001440
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMSHhq0f+001440
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMUAhq0f+001440
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMGGhq0f+001452
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMSHhq0f+001452
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMUAhq0f+001452
Zipping up the intermediate file: /data/projects/climateprediction.net/oifs_43r3_ps_0859_1992050100_123_961_12178503_0_r1860973431_60.zip
Uploading the intermediate file: upload_file_60.zip
  05:19:07 STEP 1465 H=1465:00 +CPU= 18.391
  05:19:27 STEP 1466 H=1466:00 +CPU= 18.856
  05:19:57 STEP 1467 H=1467:00 +CPU= 28.615
  05:20:17 STEP 1468 H=1468:00 +CPU= 19.071
  05:20:37 STEP 1469 H=1469:00 +CPU= 18.410
  05:21:09 STEP 1470 H=1470:00 +CPU= 29.357
  05:21:28 STEP 1471 H=1471:00 +CPU= 18.258
  05:21:47 STEP 1472 H=1472:00 +CPU= 18.770
  05:22:18 STEP 1473 H=1473:00 +CPU= 28.679
  05:22:38 STEP 1474 H=1474:00 +CPU= 18.796
  05:22:58 STEP 1475 H=1475:00 +CPU= 18.449
  05:23:31 STEP 1476 H=1476:00 +CPU= 31.657
Moving to projects directory: /data/slots/2/ICMGGhq0f+001476
Moving to projects directory: /data/slots/2/ICMSHhq0f+001476
Moving to projects directory: /data/slots/2/ICMUAhq0f+001476
  05:23:50 STEP 1477 H=1477:00 +CPU= 18.294
  05:24:09 STEP 1478 H=1478:00 +CPU= 18.841
  05:24:39 STEP 1479 H=1479:00 +CPU= 28.749
  05:24:59 STEP 1480 H=1480:00 +CPU= 18.897
  05:25:19 STEP 1481 H=1481:00 +CPU= 18.637
  05:25:50 STEP 1482 H=1482:00 +CPU= 29.291
  05:26:09 STEP 1483 H=1483:00 +CPU= 18.368
  05:26:29 STEP 1484 H=1484:00 +CPU= 18.685
  05:27:00 STEP 1485 H=1485:00 +CPU= 28.797
  05:27:21 STEP 1486 H=1486:00 +CPU= 19.094
  05:27:41 STEP 1487 H=1487:00 +CPU= 18.789
  05:28:18 STEP 1488 H=1488:00 +CPU= 34.390
Moving to projects directory: /data/slots/2/ICMGGhq0f+001488
Moving to projects directory: /data/slots/2/ICMSHhq0f+001488
Moving to projects directory: /data/slots/2/ICMUAhq0f+001488
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMGGhq0f+001464
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMSHhq0f+001464
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMUAhq0f+001464
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMGGhq0f+001476
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMSHhq0f+001476
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMUAhq0f+001476
Zipping up the intermediate file: /data/projects/climateprediction.net/oifs_43r3_ps_0859_1992050100_123_961_12178503_0_r1860973431_61.zip
Uploading the intermediate file: upload_file_61.zip
  05:28:37 STEP 1489 H=1489:00 +CPU= 18.243
  05:28:57 STEP 1490 H=1490:00 +CPU= 18.681
  05:29:28 STEP 1491 H=1491:00 +CPU= 28.835
  05:29:48 STEP 1492 H=1492:00 +CPU= 18.722
  05:30:12 STEP 1493 H=1493:00 +CPU= 18.429
  05:31:09 STEP 1494 H=1494:00 +CPU= 30.576
  05:31:45 STEP 1495 H=1495:00 +CPU= 19.232
  05:32:24 STEP 1496 H=1496:00 +CPU= 20.065
  05:33:22 STEP 1497 H=1497:00 +CPU= 30.550
  05:33:59 STEP 1498 H=1498:00 +CPU= 19.700
  05:34:18 STEP 1499 H=1499:00 +CPU= 18.460
  05:34:51 STEP 1500 H=1500:00 +CPU= 31.715
Moving to projects directory: /data/slots/2/ICMGGhq0f+001500
Moving to projects directory: /data/slots/2/ICMSHhq0f+001500
Moving to projects directory: /data/slots/2/ICMUAhq0f+001500
  05:35:12 STEP 1501 H=1501:00 +CPU= 18.495
  05:35:32 STEP 1502 H=1502:00 +CPU= 18.776
  05:36:04 STEP 1503 H=1503:00 +CPU= 28.752
  05:36:24 STEP 1504 H=1504:00 +CPU= 18.851
  05:36:43 STEP 1505 H=1505:00 +CPU= 18.343
  05:37:14 STEP 1506 H=1506:00 +CPU= 29.532
  05:37:33 STEP 1507 H=1507:00 +CPU= 18.412
  05:37:53 STEP 1508 H=1508:00 +CPU= 18.666
  05:38:25 STEP 1509 H=1509:00 +CPU= 28.977
  05:38:45 STEP 1510 H=1510:00 +CPU= 18.697
  05:39:05 STEP 1511 H=1511:00 +CPU= 18.899
  05:39:42 STEP 1512 H=1512:00 +CPU= 34.306
Moving to projects directory: /data/slots/2/ICMGGhq0f+001512
Moving to projects directory: /data/slots/2/ICMSHhq0f+001512
Moving to projects directory: /data/slots/2/ICMUAhq0f+001512
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMGGhq0f+001488
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMSHhq0f+001488
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMUAhq0f+001488
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMGGhq0f+001500
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMSHhq0f+001500
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMUAhq0f+001500
Zipping up the intermediate file: /data/projects/climateprediction.net/oifs_43r3_ps_0859_1992050100_123_961_12178503_0_r1860973431_62.zip
Uploading the intermediate file: upload_file_62.zip
  05:40:03 STEP 1513 H=1513:00 +CPU= 18.282
  05:40:24 STEP 1514 H=1514:00 +CPU= 18.829
  05:40:57 STEP 1515 H=1515:00 +CPU= 28.631
  05:41:17 STEP 1516 H=1516:00 +CPU= 18.969
  05:41:36 STEP 1517 H=1517:00 +CPU= 18.298
  05:42:07 STEP 1518 H=1518:00 +CPU= 29.302
  05:42:26 STEP 1519 H=1519:00 +CPU= 18.331
  05:42:46 STEP 1520 H=1520:00 +CPU= 18.832
  05:43:18 STEP 1521 H=1521:00 +CPU= 28.839
  05:43:38 STEP 1522 H=1522:00 +CPU= 18.902
  05:43:59 STEP 1523 H=1523:00 +CPU= 18.436
  05:44:34 STEP 1524 H=1524:00 +CPU= 31.639
Moving to projects directory: /data/slots/2/ICMGGhq0f+001524
Moving to projects directory: /data/slots/2/ICMSHhq0f+001524
Moving to projects directory: /data/slots/2/ICMUAhq0f+001524
  05:44:53 STEP 1525 H=1525:00 +CPU= 18.350
  05:45:13 STEP 1526 H=1526:00 +CPU= 18.996
  05:45:43 STEP 1527 H=1527:00 +CPU= 28.867
  05:46:03 STEP 1528 H=1528:00 +CPU= 18.805
  05:46:23 STEP 1529 H=1529:00 +CPU= 18.418
  05:46:53 STEP 1530 H=1530:00 +CPU= 29.128
  05:47:13 STEP 1531 H=1531:00 +CPU= 18.401
  05:47:33 STEP 1532 H=1532:00 +CPU= 18.752
  05:48:03 STEP 1533 H=1533:00 +CPU= 28.636
  05:48:23 STEP 1534 H=1534:00 +CPU= 18.802
  05:48:43 STEP 1535 H=1535:00 +CPU= 18.727
  05:49:23 STEP 1536 H=1536:00 +CPU= 34.647
Moving to projects directory: /data/slots/2/ICMGGhq0f+001536
Moving to projects directory: /data/slots/2/ICMSHhq0f+001536
Moving to projects directory: /data/slots/2/ICMUAhq0f+001536
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMGGhq0f+001512
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMSHhq0f+001512
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMUAhq0f+001512
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMGGhq0f+001524
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMSHhq0f+001524
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMUAhq0f+001524
Zipping up the intermediate file: /data/projects/climateprediction.net/oifs_43r3_ps_0859_1992050100_123_961_12178503_0_r1860973431_63.zip
Uploading the intermediate file: upload_file_63.zip
  05:49:43 STEP 1537 H=1537:00 +CPU= 18.372
  05:50:03 STEP 1538 H=1538:00 +CPU= 18.800
  05:50:34 STEP 1539 H=1539:00 +CPU= 28.776
  05:50:57 STEP 1540 H=1540:00 +CPU= 18.713
  05:51:16 STEP 1541 H=1541:00 +CPU= 18.437
  05:51:47 STEP 1542 H=1542:00 +CPU= 29.098
  05:52:06 STEP 1543 H=1543:00 +CPU= 18.379
  05:52:26 STEP 1544 H=1544:00 +CPU= 18.911
  05:52:56 STEP 1545 H=1545:00 +CPU= 28.634
  05:53:16 STEP 1546 H=1546:00 +CPU= 18.916
  05:53:35 STEP 1547 H=1547:00 +CPU= 18.425
  05:54:08 STEP 1548 H=1548:00 +CPU= 31.473
Moving to projects directory: /data/slots/2/ICMGGhq0f+001548
Moving to projects directory: /data/slots/2/ICMSHhq0f+001548
Moving to projects directory: /data/slots/2/ICMUAhq0f+001548
  05:54:27 STEP 1549 H=1549:00 +CPU= 18.350
  05:54:46 STEP 1550 H=1550:00 +CPU= 18.712
  05:55:17 STEP 1551 H=1551:00 +CPU= 28.702
  05:55:36 STEP 1552 H=1552:00 +CPU= 18.769
  05:55:55 STEP 1553 H=1553:00 +CPU= 18.225
  05:56:30 STEP 1554 H=1554:00 +CPU= 29.255
  05:56:50 STEP 1555 H=1555:00 +CPU= 18.196
  05:57:10 STEP 1556 H=1556:00 +CPU= 18.806
  05:57:40 STEP 1557 H=1557:00 +CPU= 28.886
  05:58:00 STEP 1558 H=1558:00 +CPU= 19.015
  05:58:21 STEP 1559 H=1559:00 +CPU= 19.316
  05:58:57 STEP 1560 H=1560:00 +CPU= 34.372
Moving to projects directory: /data/slots/2/ICMGGhq0f+001560
Moving to projects directory: /data/slots/2/ICMSHhq0f+001560
Moving to projects directory: /data/slots/2/ICMUAhq0f+001560
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMGGhq0f+001536
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMSHhq0f+001536
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMUAhq0f+001536
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMGGhq0f+001548
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMSHhq0f+001548
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMUAhq0f+001548
Zipping up the intermediate file: /data/projects/climateprediction.net/oifs_43r3_ps_0859_1992050100_123_961_12178503_0_r1860973431_64.zip
Uploading the intermediate file: upload_file_64.zip
  05:59:17 STEP 1561 H=1561:00 +CPU= 18.538
Uploading trickle at timestep: 5616000
  05:59:37 STEP 1562 H=1562:00 +CPU= 19.229
  06:00:14 STEP 1563 H=1563:00 +CPU= 29.127
  06:00:50 STEP 1564 H=1564:00 +CPU= 19.715
  06:01:27 STEP 1565 H=1565:00 +CPU= 19.235
  06:02:25 STEP 1566 H=1566:00 +CPU= 30.509
  06:03:00 STEP 1567 H=1567:00 +CPU= 19.135
  06:03:37 STEP 1568 H=1568:00 +CPU= 19.748
  06:04:16 STEP 1569 H=1569:00 +CPU= 29.384
  06:04:36 STEP 1570 H=1570:00 +CPU= 19.093
  06:04:55 STEP 1571 H=1571:00 +CPU= 18.321
  06:05:28 STEP 1572 H=1572:00 +CPU= 31.844
Moving to projects directory: /data/slots/2/ICMGGhq0f+001572
Moving to projects directory: /data/slots/2/ICMSHhq0f+001572
Moving to projects directory: /data/slots/2/ICMUAhq0f+001572
  06:05:46 STEP 1573 H=1573:00 +CPU= 18.379
  06:06:06 STEP 1574 H=1574:00 +CPU= 18.941
  06:06:36 STEP 1575 H=1575:00 +CPU= 28.842
  06:06:55 STEP 1576 H=1576:00 +CPU= 18.753
  06:07:14 STEP 1577 H=1577:00 +CPU= 18.396
  06:07:45 STEP 1578 H=1578:00 +CPU= 29.385
  06:08:04 STEP 1579 H=1579:00 +CPU= 18.291
  06:08:24 STEP 1580 H=1580:00 +CPU= 18.856
  06:08:55 STEP 1581 H=1581:00 +CPU= 28.776
  06:09:14 STEP 1582 H=1582:00 +CPU= 18.871
  06:09:35 STEP 1583 H=1583:00 +CPU= 18.941
  06:10:11 STEP 1584 H=1584:00 +CPU= 34.478
Moving to projects directory: /data/slots/2/ICMGGhq0f+001584
Moving to projects directory: /data/slots/2/ICMSHhq0f+001584
Moving to projects directory: /data/slots/2/ICMUAhq0f+001584
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMGGhq0f+001560
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMSHhq0f+001560
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMUAhq0f+001560
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMGGhq0f+001572
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMSHhq0f+001572
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMUAhq0f+001572
Zipping up the intermediate file: /data/projects/climateprediction.net/oifs_43r3_ps_0859_1992050100_123_961_12178503_0_r1860973431_65.zip
Uploading the intermediate file: upload_file_65.zip
  06:10:31 STEP 1585 H=1585:00 +CPU= 18.363
  06:10:51 STEP 1586 H=1586:00 +CPU= 18.686
  06:11:20 STEP 1587 H=1587:00 +CPU= 28.773
  06:11:40 STEP 1588 H=1588:00 +CPU= 18.888
  06:11:59 STEP 1589 H=1589:00 +CPU= 18.268
  06:12:31 STEP 1590 H=1590:00 +CPU= 29.500
  06:12:50 STEP 1591 H=1591:00 +CPU= 18.230
  06:13:09 STEP 1592 H=1592:00 +CPU= 18.859
  06:13:39 STEP 1593 H=1593:00 +CPU= 28.645
  06:14:00 STEP 1594 H=1594:00 +CPU= 18.779
  06:14:20 STEP 1595 H=1595:00 +CPU= 18.551
  06:14:57 STEP 1596 H=1596:00 +CPU= 31.705
Moving to projects directory: /data/slots/2/ICMGGhq0f+001596
Moving to projects directory: /data/slots/2/ICMSHhq0f+001596
Moving to projects directory: /data/slots/2/ICMUAhq0f+001596
  06:15:17 STEP 1597 H=1597:00 +CPU= 18.324
  06:15:39 STEP 1598 H=1598:00 +CPU= 18.960
  06:16:09 STEP 1599 H=1599:00 +CPU= 28.820
  06:16:28 STEP 1600 H=1600:00 +CPU= 18.759
  06:16:48 STEP 1601 H=1601:00 +CPU= 18.310
  06:17:21 STEP 1602 H=1602:00 +CPU= 31.463
  06:17:40 STEP 1603 H=1603:00 +CPU= 18.468
  06:18:00 STEP 1604 H=1604:00 +CPU= 18.679
  06:18:31 STEP 1605 H=1605:00 +CPU= 28.668
  06:18:52 STEP 1606 H=1606:00 +CPU= 19.075
  06:19:13 STEP 1607 H=1607:00 +CPU= 18.830
  06:19:49 STEP 1608 H=1608:00 +CPU= 34.371
Moving to projects directory: /data/slots/2/ICMGGhq0f+001608
Moving to projects directory: /data/slots/2/ICMSHhq0f+001608
Moving to projects directory: /data/slots/2/ICMUAhq0f+001608
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMGGhq0f+001584
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMSHhq0f+001584
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMUAhq0f+001584
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMGGhq0f+001596
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMSHhq0f+001596
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMUAhq0f+001596
Zipping up the intermediate file: /data/projects/climateprediction.net/oifs_43r3_ps_0859_1992050100_123_961_12178503_0_r1860973431_66.zip
Uploading the intermediate file: upload_file_66.zip
  06:20:08 STEP 1609 H=1609:00 +CPU= 18.289
  06:20:28 STEP 1610 H=1610:00 +CPU= 18.793
  06:20:58 STEP 1611 H=1611:00 +CPU= 28.778
  06:21:18 STEP 1612 H=1612:00 +CPU= 18.664
  06:21:37 STEP 1613 H=1613:00 +CPU= 18.197
  06:22:09 STEP 1614 H=1614:00 +CPU= 29.282
  06:22:28 STEP 1615 H=1615:00 +CPU= 18.210
  06:22:48 STEP 1616 H=1616:00 +CPU= 18.687
  06:23:18 STEP 1617 H=1617:00 +CPU= 28.687
  06:23:38 STEP 1618 H=1618:00 +CPU= 18.648
  06:23:58 STEP 1619 H=1619:00 +CPU= 18.629
  06:24:34 STEP 1620 H=1620:00 +CPU= 31.698
Moving to projects directory: /data/slots/2/ICMGGhq0f+001620
Moving to projects directory: /data/slots/2/ICMSHhq0f+001620
Moving to projects directory: /data/slots/2/ICMUAhq0f+001620
  06:24:54 STEP 1621 H=1621:00 +CPU= 18.287
  06:25:14 STEP 1622 H=1622:00 +CPU= 18.818
  06:25:47 STEP 1623 H=1623:00 +CPU= 28.566
  06:26:08 STEP 1624 H=1624:00 +CPU= 18.807
  06:26:28 STEP 1625 H=1625:00 +CPU= 18.241
  06:26:59 STEP 1626 H=1626:00 +CPU= 29.004
  06:27:20 STEP 1627 H=1627:00 +CPU= 18.337
  06:27:41 STEP 1628 H=1628:00 +CPU= 18.778
  06:28:13 STEP 1629 H=1629:00 +CPU= 28.673
  06:28:32 STEP 1630 H=1630:00 +CPU= 18.681
  06:28:53 STEP 1631 H=1631:00 +CPU= 18.818
  06:29:37 STEP 1632 H=1632:00 +CPU= 34.426
Moving to projects directory: /data/slots/2/ICMGGhq0f+001632
Moving to projects directory: /data/slots/2/ICMSHhq0f+001632
Moving to projects directory: /data/slots/2/ICMUAhq0f+001632
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMGGhq0f+001608
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMSHhq0f+001608
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMUAhq0f+001608
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMGGhq0f+001620
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMSHhq0f+001620
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMUAhq0f+001620
Zipping up the intermediate file: /data/projects/climateprediction.net/oifs_43r3_ps_0859_1992050100_123_961_12178503_0_r1860973431_67.zip
Uploading the intermediate file: upload_file_67.zip
  06:29:57 STEP 1633 H=1633:00 +CPU= 18.199
  06:30:30 STEP 1634 H=1634:00 +CPU= 19.459
  06:31:25 STEP 1635 H=1635:00 +CPU= 29.819
  06:32:01 STEP 1636 H=1636:00 +CPU= 19.601
  06:32:38 STEP 1637 H=1637:00 +CPU= 19.260
  06:33:34 STEP 1638 H=1638:00 +CPU= 30.570
  06:34:05 STEP 1639 H=1639:00 +CPU= 19.080
  06:34:24 STEP 1640 H=1640:00 +CPU= 18.784
  06:34:54 STEP 1641 H=1641:00 +CPU= 28.625
  06:35:13 STEP 1642 H=1642:00 +CPU= 18.925
  06:35:32 STEP 1643 H=1643:00 +CPU= 18.304
  06:36:05 STEP 1644 H=1644:00 +CPU= 31.608
Moving to projects directory: /data/slots/2/ICMGGhq0f+001644
Moving to projects directory: /data/slots/2/ICMSHhq0f+001644
Moving to projects directory: /data/slots/2/ICMUAhq0f+001644
  06:36:24 STEP 1645 H=1645:00 +CPU= 18.442
  06:36:43 STEP 1646 H=1646:00 +CPU= 18.733
  06:37:14 STEP 1647 H=1647:00 +CPU= 28.749
  06:37:33 STEP 1648 H=1648:00 +CPU= 18.855
  06:37:52 STEP 1649 H=1649:00 +CPU= 18.149
  06:38:24 STEP 1650 H=1650:00 +CPU= 29.486
  06:38:45 STEP 1651 H=1651:00 +CPU= 18.374
  06:39:07 STEP 1652 H=1652:00 +CPU= 18.858
  06:39:38 STEP 1653 H=1653:00 +CPU= 28.765
  06:39:58 STEP 1654 H=1654:00 +CPU= 18.614
  06:40:19 STEP 1655 H=1655:00 +CPU= 18.994
  06:40:55 STEP 1656 H=1656:00 +CPU= 34.315
Moving to projects directory: /data/slots/2/ICMGGhq0f+001656
Moving to projects directory: /data/slots/2/ICMSHhq0f+001656
Moving to projects directory: /data/slots/2/ICMUAhq0f+001656
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMGGhq0f+001632
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMSHhq0f+001632
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMUAhq0f+001632
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMGGhq0f+001644
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMSHhq0f+001644
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMUAhq0f+001644
Zipping up the intermediate file: /data/projects/climateprediction.net/oifs_43r3_ps_0859_1992050100_123_961_12178503_0_r1860973431_68.zip
Uploading the intermediate file: upload_file_68.zip
  06:41:15 STEP 1657 H=1657:00 +CPU= 18.537
  06:41:34 STEP 1658 H=1658:00 +CPU= 18.668
  06:42:04 STEP 1659 H=1659:00 +CPU= 28.800
  06:42:24 STEP 1660 H=1660:00 +CPU= 18.994
  06:42:42 STEP 1661 H=1661:00 +CPU= 18.281
  06:43:14 STEP 1662 H=1662:00 +CPU= 29.520
  06:43:33 STEP 1663 H=1663:00 +CPU= 18.251
  06:43:52 STEP 1664 H=1664:00 +CPU= 18.725
  06:44:22 STEP 1665 H=1665:00 +CPU= 28.692
  06:44:44 STEP 1666 H=1666:00 +CPU= 18.774
  06:45:06 STEP 1667 H=1667:00 +CPU= 18.448
  06:45:40 STEP 1668 H=1668:00 +CPU= 31.573
Moving to projects directory: /data/slots/2/ICMGGhq0f+001668
Moving to projects directory: /data/slots/2/ICMSHhq0f+001668
Moving to projects directory: /data/slots/2/ICMUAhq0f+001668
  06:45:59 STEP 1669 H=1669:00 +CPU= 18.188
  06:46:19 STEP 1670 H=1670:00 +CPU= 19.015
  06:46:49 STEP 1671 H=1671:00 +CPU= 28.457
  06:47:10 STEP 1672 H=1672:00 +CPU= 18.924
  06:47:29 STEP 1673 H=1673:00 +CPU= 18.197
  06:48:00 STEP 1674 H=1674:00 +CPU= 28.969
  06:48:20 STEP 1675 H=1675:00 +CPU= 18.737
  06:48:41 STEP 1676 H=1676:00 +CPU= 18.912
  06:49:12 STEP 1677 H=1677:00 +CPU= 28.938
  06:49:32 STEP 1678 H=1678:00 +CPU= 18.806
  06:49:51 STEP 1679 H=1679:00 +CPU= 18.687
  06:50:28 STEP 1680 H=1680:00 +CPU= 34.615
Moving to projects directory: /data/slots/2/ICMGGhq0f+001680
Moving to projects directory: /data/slots/2/ICMSHhq0f+001680
Moving to projects directory: /data/slots/2/ICMUAhq0f+001680
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMGGhq0f+001656
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMSHhq0f+001656
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMUAhq0f+001656
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMGGhq0f+001668
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMSHhq0f+001668
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMUAhq0f+001668
Zipping up the intermediate file: /data/projects/climateprediction.net/oifs_43r3_ps_0859_1992050100_123_961_12178503_0_r1860973431_69.zip
Uploading the intermediate file: upload_file_69.zip
  06:50:47 STEP 1681 H=1681:00 +CPU= 18.152
  06:51:07 STEP 1682 H=1682:00 +CPU= 18.775
  06:51:37 STEP 1683 H=1683:00 +CPU= 28.683
  06:51:57 STEP 1684 H=1684:00 +CPU= 18.571
  06:52:16 STEP 1685 H=1685:00 +CPU= 18.305
  06:52:47 STEP 1686 H=1686:00 +CPU= 29.286
  06:53:06 STEP 1687 H=1687:00 +CPU= 18.275
  06:53:26 STEP 1688 H=1688:00 +CPU= 18.729
  06:53:56 STEP 1689 H=1689:00 +CPU= 28.633
  06:54:16 STEP 1690 H=1690:00 +CPU= 18.904
  06:54:36 STEP 1691 H=1691:00 +CPU= 18.222
  06:55:09 STEP 1692 H=1692:00 +CPU= 31.557
Moving to projects directory: /data/slots/2/ICMGGhq0f+001692
Moving to projects directory: /data/slots/2/ICMSHhq0f+001692
Moving to projects directory: /data/slots/2/ICMUAhq0f+001692
  06:55:29 STEP 1693 H=1693:00 +CPU= 18.263
  06:55:48 STEP 1694 H=1694:00 +CPU= 18.699
  06:56:19 STEP 1695 H=1695:00 +CPU= 28.870
  06:56:39 STEP 1696 H=1696:00 +CPU= 18.778
  06:56:58 STEP 1697 H=1697:00 +CPU= 18.148
  06:57:30 STEP 1698 H=1698:00 +CPU= 29.649
  06:57:49 STEP 1699 H=1699:00 +CPU= 18.459
  06:58:09 STEP 1700 H=1700:00 +CPU= 19.334
  06:58:40 STEP 1701 H=1701:00 +CPU= 28.562
  06:59:00 STEP 1702 H=1702:00 +CPU= 18.671
  06:59:21 STEP 1703 H=1703:00 +CPU= 18.955
  07:00:07 STEP 1704 H=1704:00 +CPU= 34.408
Moving to projects directory: /data/slots/2/ICMGGhq0f+001704
Moving to projects directory: /data/slots/2/ICMSHhq0f+001704
Moving to projects directory: /data/slots/2/ICMUAhq0f+001704
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMGGhq0f+001680
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMSHhq0f+001680
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMUAhq0f+001680
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMGGhq0f+001692
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMSHhq0f+001692
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMUAhq0f+001692
Zipping up the intermediate file: /data/projects/climateprediction.net/oifs_43r3_ps_0859_1992050100_123_961_12178503_0_r1860973431_70.zip
Uploading the intermediate file: upload_file_70.zip
  07:00:45 STEP 1705 H=1705:00 +CPU= 19.426
  07:01:21 STEP 1706 H=1706:00 +CPU= 19.516
  07:02:16 STEP 1707 H=1707:00 +CPU= 29.752
  07:02:53 STEP 1708 H=1708:00 +CPU= 19.591
  07:03:29 STEP 1709 H=1709:00 +CPU= 19.359
  07:04:13 STEP 1710 H=1710:00 +CPU= 29.888
  07:04:32 STEP 1711 H=1711:00 +CPU= 18.274
  07:04:56 STEP 1712 H=1712:00 +CPU= 18.705
  07:05:27 STEP 1713 H=1713:00 +CPU= 28.772
  07:05:47 STEP 1714 H=1714:00 +CPU= 18.778
  07:06:06 STEP 1715 H=1715:00 +CPU= 18.354
  07:06:42 STEP 1716 H=1716:00 +CPU= 31.641
Moving to projects directory: /data/slots/2/ICMGGhq0f+001716
Moving to projects directory: /data/slots/2/ICMSHhq0f+001716
Moving to projects directory: /data/slots/2/ICMUAhq0f+001716
  07:07:02 STEP 1717 H=1717:00 +CPU= 18.481
  07:07:21 STEP 1718 H=1718:00 +CPU= 18.837
  07:07:52 STEP 1719 H=1719:00 +CPU= 28.782
  07:08:11 STEP 1720 H=1720:00 +CPU= 18.798
  07:08:31 STEP 1721 H=1721:00 +CPU= 18.275
  07:09:02 STEP 1722 H=1722:00 +CPU= 29.130
  07:09:22 STEP 1723 H=1723:00 +CPU= 18.259
  07:09:42 STEP 1724 H=1724:00 +CPU= 18.864
  07:10:13 STEP 1725 H=1725:00 +CPU= 28.816
  07:10:34 STEP 1726 H=1726:00 +CPU= 18.751
  07:10:53 STEP 1727 H=1727:00 +CPU= 18.866
  07:11:44 STEP 1728 H=1728:00 +CPU= 34.489
Moving to projects directory: /data/slots/2/ICMGGhq0f+001728
Moving to projects directory: /data/slots/2/ICMSHhq0f+001728
Moving to projects directory: /data/slots/2/ICMUAhq0f+001728
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMGGhq0f+001704
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMSHhq0f+001704
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMUAhq0f+001704
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMGGhq0f+001716
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMSHhq0f+001716
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMUAhq0f+001716
Zipping up the intermediate file: /data/projects/climateprediction.net/oifs_43r3_ps_0859_1992050100_123_961_12178503_0_r1860973431_71.zip
Uploading the intermediate file: upload_file_71.zip
  07:12:06 STEP 1729 H=1729:00 +CPU= 18.398
  07:12:25 STEP 1730 H=1730:00 +CPU= 18.807
  07:12:55 STEP 1731 H=1731:00 +CPU= 28.620
  07:13:17 STEP 1732 H=1732:00 +CPU= 18.891
  07:13:40 STEP 1733 H=1733:00 +CPU= 18.402
  07:14:12 STEP 1734 H=1734:00 +CPU= 29.228
  07:14:33 STEP 1735 H=1735:00 +CPU= 18.294
  07:14:52 STEP 1736 H=1736:00 +CPU= 18.791
  07:15:26 STEP 1737 H=1737:00 +CPU= 28.597
  07:15:46 STEP 1738 H=1738:00 +CPU= 19.077
  07:16:07 STEP 1739 H=1739:00 +CPU= 18.348
  07:16:41 STEP 1740 H=1740:00 +CPU= 31.445
Moving to projects directory: /data/slots/2/ICMGGhq0f+001740
Moving to projects directory: /data/slots/2/ICMSHhq0f+001740
Moving to projects directory: /data/slots/2/ICMUAhq0f+001740
  07:17:00 STEP 1741 H=1741:00 +CPU= 18.345
  07:17:23 STEP 1742 H=1742:00 +CPU= 20.446
  07:17:55 STEP 1743 H=1743:00 +CPU= 28.790
  07:18:16 STEP 1744 H=1744:00 +CPU= 18.906
  07:18:38 STEP 1745 H=1745:00 +CPU= 18.291
  07:19:11 STEP 1746 H=1746:00 +CPU= 29.317
  07:19:31 STEP 1747 H=1747:00 +CPU= 18.207
  07:19:51 STEP 1748 H=1748:00 +CPU= 18.868
  07:20:26 STEP 1749 H=1749:00 +CPU= 28.861
  07:20:46 STEP 1750 H=1750:00 +CPU= 18.667
  07:21:07 STEP 1751 H=1751:00 +CPU= 18.796
  07:21:43 STEP 1752 H=1752:00 +CPU= 34.374
Moving to projects directory: /data/slots/2/ICMGGhq0f+001752
Moving to projects directory: /data/slots/2/ICMSHhq0f+001752
Moving to projects directory: /data/slots/2/ICMUAhq0f+001752
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMGGhq0f+001728
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMSHhq0f+001728
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMUAhq0f+001728
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMGGhq0f+001740
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMSHhq0f+001740
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMUAhq0f+001740
Zipping up the intermediate file: /data/projects/climateprediction.net/oifs_43r3_ps_0859_1992050100_123_961_12178503_0_r1860973431_72.zip
Uploading the intermediate file: upload_file_72.zip
  07:22:04 STEP 1753 H=1753:00 +CPU= 18.184
  07:22:24 STEP 1754 H=1754:00 +CPU= 18.817
  07:22:57 STEP 1755 H=1755:00 +CPU= 28.537
  07:23:17 STEP 1756 H=1756:00 +CPU= 18.761
  07:23:36 STEP 1757 H=1757:00 +CPU= 18.239
  07:24:08 STEP 1758 H=1758:00 +CPU= 29.300
  07:24:30 STEP 1759 H=1759:00 +CPU= 18.430
  07:24:51 STEP 1760 H=1760:00 +CPU= 18.708
  07:25:21 STEP 1761 H=1761:00 +CPU= 28.707
  07:25:42 STEP 1762 H=1762:00 +CPU= 18.823
  07:26:02 STEP 1763 H=1763:00 +CPU= 18.353
  07:26:34 STEP 1764 H=1764:00 +CPU= 31.516
Moving to projects directory: /data/slots/2/ICMGGhq0f+001764
Moving to projects directory: /data/slots/2/ICMSHhq0f+001764
Moving to projects directory: /data/slots/2/ICMUAhq0f+001764
  07:26:54 STEP 1765 H=1765:00 +CPU= 18.384
  07:27:13 STEP 1766 H=1766:00 +CPU= 18.774
  07:27:45 STEP 1767 H=1767:00 +CPU= 28.659
  07:28:06 STEP 1768 H=1768:00 +CPU= 18.883
  07:28:25 STEP 1769 H=1769:00 +CPU= 18.232
  07:28:56 STEP 1770 H=1770:00 +CPU= 29.134
  07:29:15 STEP 1771 H=1771:00 +CPU= 18.411
  07:29:35 STEP 1772 H=1772:00 +CPU= 18.700
  07:30:10 STEP 1773 H=1773:00 +CPU= 28.913
  07:30:46 STEP 1774 H=1774:00 +CPU= 19.720
  07:31:22 STEP 1775 H=1775:00 +CPU= 19.709
  07:32:28 STEP 1776 H=1776:00 +CPU= 35.913
Moving to projects directory: /data/slots/2/ICMGGhq0f+001776
Moving to projects directory: /data/slots/2/ICMSHhq0f+001776
Moving to projects directory: /data/slots/2/ICMUAhq0f+001776
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMGGhq0f+001752
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMSHhq0f+001752
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMUAhq0f+001752
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMGGhq0f+001764
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMSHhq0f+001764
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMUAhq0f+001764
Zipping up the intermediate file: /data/projects/climateprediction.net/oifs_43r3_ps_0859_1992050100_123_961_12178503_0_r1860973431_73.zip
Uploading the intermediate file: upload_file_73.zip
  07:33:05 STEP 1777 H=1777:00 +CPU= 19.355
  07:33:40 STEP 1778 H=1778:00 +CPU= 19.717
  07:34:20 STEP 1779 H=1779:00 +CPU= 29.210
  07:34:39 STEP 1780 H=1780:00 +CPU= 18.666
  07:34:59 STEP 1781 H=1781:00 +CPU= 18.308
  07:35:30 STEP 1782 H=1782:00 +CPU= 29.393
  07:35:49 STEP 1783 H=1783:00 +CPU= 18.286
  07:36:10 STEP 1784 H=1784:00 +CPU= 18.791
  07:36:40 STEP 1785 H=1785:00 +CPU= 28.616
  07:37:00 STEP 1786 H=1786:00 +CPU= 18.754
  07:37:20 STEP 1787 H=1787:00 +CPU= 18.336
  07:37:53 STEP 1788 H=1788:00 +CPU= 31.587
Moving to projects directory: /data/slots/2/ICMGGhq0f+001788
Moving to projects directory: /data/slots/2/ICMSHhq0f+001788
Moving to projects directory: /data/slots/2/ICMUAhq0f+001788
  07:38:13 STEP 1789 H=1789:00 +CPU= 18.644
  07:38:34 STEP 1790 H=1790:00 +CPU= 18.740
  07:39:07 STEP 1791 H=1791:00 +CPU= 28.762
  07:39:27 STEP 1792 H=1792:00 +CPU= 18.769
  07:39:45 STEP 1793 H=1793:00 +CPU= 18.186
  07:40:17 STEP 1794 H=1794:00 +CPU= 29.385
  07:40:36 STEP 1795 H=1795:00 +CPU= 18.177
  07:40:55 STEP 1796 H=1796:00 +CPU= 18.724
  07:41:26 STEP 1797 H=1797:00 +CPU= 28.939
  07:41:45 STEP 1798 H=1798:00 +CPU= 18.619
  07:42:06 STEP 1799 H=1799:00 +CPU= 18.975
  07:42:42 STEP 1800 H=1800:00 +CPU= 34.412
Moving to projects directory: /data/slots/2/ICMGGhq0f+001800
Moving to projects directory: /data/slots/2/ICMSHhq0f+001800
Moving to projects directory: /data/slots/2/ICMUAhq0f+001800
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMGGhq0f+001776
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMSHhq0f+001776
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMUAhq0f+001776
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMGGhq0f+001788
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMSHhq0f+001788
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMUAhq0f+001788
Zipping up the intermediate file: /data/projects/climateprediction.net/oifs_43r3_ps_0859_1992050100_123_961_12178503_0_r1860973431_74.zip
Uploading the intermediate file: upload_file_74.zip
  07:43:02 STEP 1801 H=1801:00 +CPU= 18.443
Uploading trickle at timestep: 6480000
  07:43:22 STEP 1802 H=1802:00 +CPU= 18.862
  07:43:51 STEP 1803 H=1803:00 +CPU= 28.703
  07:44:12 STEP 1804 H=1804:00 +CPU= 18.796
  07:44:30 STEP 1805 H=1805:00 +CPU= 18.307
  07:45:00 STEP 1806 H=1806:00 +CPU= 29.140
  07:45:20 STEP 1807 H=1807:00 +CPU= 18.506
  07:45:39 STEP 1808 H=1808:00 +CPU= 18.735
  07:46:08 STEP 1809 H=1809:00 +CPU= 28.786
  07:46:28 STEP 1810 H=1810:00 +CPU= 18.845
  07:46:46 STEP 1811 H=1811:00 +CPU= 18.283
  07:47:20 STEP 1812 H=1812:00 +CPU= 31.671
Moving to projects directory: /data/slots/2/ICMGGhq0f+001812
Moving to projects directory: /data/slots/2/ICMSHhq0f+001812
Moving to projects directory: /data/slots/2/ICMUAhq0f+001812
  07:47:39 STEP 1813 H=1813:00 +CPU= 18.452
  07:47:58 STEP 1814 H=1814:00 +CPU= 18.884
  07:48:28 STEP 1815 H=1815:00 +CPU= 28.786
  07:48:47 STEP 1816 H=1816:00 +CPU= 18.800
  07:49:07 STEP 1817 H=1817:00 +CPU= 18.390
  07:49:42 STEP 1818 H=1818:00 +CPU= 29.191
  07:50:01 STEP 1819 H=1819:00 +CPU= 18.222
  07:50:25 STEP 1820 H=1820:00 +CPU= 18.994
  07:50:57 STEP 1821 H=1821:00 +CPU= 28.755
  07:51:19 STEP 1822 H=1822:00 +CPU= 18.869
  07:51:39 STEP 1823 H=1823:00 +CPU= 18.886
  07:52:24 STEP 1824 H=1824:00 +CPU= 35.175
Moving to projects directory: /data/slots/2/ICMGGhq0f+001824
Moving to projects directory: /data/slots/2/ICMSHhq0f+001824
Moving to projects directory: /data/slots/2/ICMUAhq0f+001824
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMGGhq0f+001800
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMSHhq0f+001800
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMUAhq0f+001800
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMGGhq0f+001812
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMSHhq0f+001812
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMUAhq0f+001812
Zipping up the intermediate file: /data/projects/climateprediction.net/oifs_43r3_ps_0859_1992050100_123_961_12178503_0_r1860973431_75.zip
Uploading the intermediate file: upload_file_75.zip
  07:52:44 STEP 1825 H=1825:00 +CPU= 18.899
  07:53:04 STEP 1826 H=1826:00 +CPU= 18.997
  07:53:34 STEP 1827 H=1827:00 +CPU= 28.700
  07:53:53 STEP 1828 H=1828:00 +CPU= 18.683
  07:54:13 STEP 1829 H=1829:00 +CPU= 18.460
  07:54:43 STEP 1830 H=1830:00 +CPU= 29.095
  07:55:02 STEP 1831 H=1831:00 +CPU= 18.298
  07:55:22 STEP 1832 H=1832:00 +CPU= 18.969
  07:55:52 STEP 1833 H=1833:00 +CPU= 28.678
  07:56:12 STEP 1834 H=1834:00 +CPU= 18.912
  07:56:31 STEP 1835 H=1835:00 +CPU= 18.363
  07:57:03 STEP 1836 H=1836:00 +CPU= 31.625
Moving to projects directory: /data/slots/2/ICMGGhq0f+001836
Moving to projects directory: /data/slots/2/ICMSHhq0f+001836
Moving to projects directory: /data/slots/2/ICMUAhq0f+001836
  07:57:23 STEP 1837 H=1837:00 +CPU= 18.415
  07:57:43 STEP 1838 H=1838:00 +CPU= 19.128
  07:58:12 STEP 1839 H=1839:00 +CPU= 28.738
  07:58:32 STEP 1840 H=1840:00 +CPU= 18.912
  07:58:50 STEP 1841 H=1841:00 +CPU= 18.148
  07:59:22 STEP 1842 H=1842:00 +CPU= 29.609
  07:59:42 STEP 1843 H=1843:00 +CPU= 18.281
  08:00:02 STEP 1844 H=1844:00 +CPU= 19.163
  08:00:58 STEP 1845 H=1845:00 +CPU= 29.995
  08:01:35 STEP 1846 H=1846:00 +CPU= 19.752
  08:02:14 STEP 1847 H=1847:00 +CPU= 20.949
  08:03:25 STEP 1848 H=1848:00 +CPU= 36.503
Moving to projects directory: /data/slots/2/ICMGGhq0f+001848
Moving to projects directory: /data/slots/2/ICMSHhq0f+001848
Moving to projects directory: /data/slots/2/ICMUAhq0f+001848
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMGGhq0f+001824
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMSHhq0f+001824
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMUAhq0f+001824
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMGGhq0f+001836
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMSHhq0f+001836
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMUAhq0f+001836
Zipping up the intermediate file: /data/projects/climateprediction.net/oifs_43r3_ps_0859_1992050100_123_961_12178503_0_r1860973431_76.zip
Uploading the intermediate file: upload_file_76.zip
  08:03:59 STEP 1849 H=1849:00 +CPU= 19.619
  08:04:21 STEP 1850 H=1850:00 +CPU= 19.604
  08:04:54 STEP 1851 H=1851:00 +CPU= 29.604
  08:05:16 STEP 1852 H=1852:00 +CPU= 19.563
  08:05:37 STEP 1853 H=1853:00 +CPU= 19.067
  08:06:10 STEP 1854 H=1854:00 +CPU= 30.287
  08:06:34 STEP 1855 H=1855:00 +CPU= 19.391
  08:06:55 STEP 1856 H=1856:00 +CPU= 19.464
  08:07:29 STEP 1857 H=1857:00 +CPU= 29.821
  08:07:50 STEP 1858 H=1858:00 +CPU= 19.544
  08:08:12 STEP 1859 H=1859:00 +CPU= 19.186
  08:08:57 STEP 1860 H=1860:00 +CPU= 32.534
Moving to projects directory: /data/slots/2/ICMGGhq0f+001860
Moving to projects directory: /data/slots/2/ICMSHhq0f+001860
Moving to projects directory: /data/slots/2/ICMUAhq0f+001860
  08:09:19 STEP 1861 H=1861:00 +CPU= 19.283
  08:09:41 STEP 1862 H=1862:00 +CPU= 19.750
  08:10:14 STEP 1863 H=1863:00 +CPU= 29.743
  08:10:38 STEP 1864 H=1864:00 +CPU= 19.881
  08:11:02 STEP 1865 H=1865:00 +CPU= 19.349
  08:11:37 STEP 1866 H=1866:00 +CPU= 30.927
  08:12:11 STEP 1867 H=1867:00 +CPU= 19.354
  08:12:34 STEP 1868 H=1868:00 +CPU= 19.833
  08:13:07 STEP 1869 H=1869:00 +CPU= 29.973
  08:13:31 STEP 1870 H=1870:00 +CPU= 19.841
  08:13:59 STEP 1871 H=1871:00 +CPU= 20.101
  08:14:53 STEP 1872 H=1872:00 +CPU= 35.521
Moving to projects directory: /data/slots/2/ICMGGhq0f+001872
Moving to projects directory: /data/slots/2/ICMSHhq0f+001872
Moving to projects directory: /data/slots/2/ICMUAhq0f+001872
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMGGhq0f+001848
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMSHhq0f+001848
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMUAhq0f+001848
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMGGhq0f+001860
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMSHhq0f+001860
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMUAhq0f+001860
Zipping up the intermediate file: /data/projects/climateprediction.net/oifs_43r3_ps_0859_1992050100_123_961_12178503_0_r1860973431_77.zip
Uploading the intermediate file: upload_file_77.zip
  08:15:18 STEP 1873 H=1873:00 +CPU= 19.340
  08:15:41 STEP 1874 H=1874:00 +CPU= 20.002
  08:16:13 STEP 1875 H=1875:00 +CPU= 29.705
  08:16:34 STEP 1876 H=1876:00 +CPU= 19.279
  08:16:53 STEP 1877 H=1877:00 +CPU= 18.729
  08:17:26 STEP 1878 H=1878:00 +CPU= 31.811
  08:17:46 STEP 1879 H=1879:00 +CPU= 18.801
  08:18:07 STEP 1880 H=1880:00 +CPU= 19.569
  08:18:38 STEP 1881 H=1881:00 +CPU= 29.327
  08:18:58 STEP 1882 H=1882:00 +CPU= 19.238
  08:19:18 STEP 1883 H=1883:00 +CPU= 18.902
  08:19:53 STEP 1884 H=1884:00 +CPU= 32.205
Moving to projects directory: /data/slots/2/ICMGGhq0f+001884
Moving to projects directory: /data/slots/2/ICMSHhq0f+001884
Moving to projects directory: /data/slots/2/ICMUAhq0f+001884
  08:20:13 STEP 1885 H=1885:00 +CPU= 18.881
  08:20:34 STEP 1886 H=1886:00 +CPU= 19.266
  08:21:05 STEP 1887 H=1887:00 +CPU= 29.112
  08:21:29 STEP 1888 H=1888:00 +CPU= 19.615
  08:21:49 STEP 1889 H=1889:00 +CPU= 18.668
  08:22:20 STEP 1890 H=1890:00 +CPU= 29.609
  08:22:41 STEP 1891 H=1891:00 +CPU= 18.840
  08:23:01 STEP 1892 H=1892:00 +CPU= 19.257
  08:23:35 STEP 1893 H=1893:00 +CPU= 29.273
  08:23:57 STEP 1894 H=1894:00 +CPU= 19.776
  08:24:18 STEP 1895 H=1895:00 +CPU= 19.992
  08:25:10 STEP 1896 H=1896:00 +CPU= 35.968
Moving to projects directory: /data/slots/2/ICMGGhq0f+001896
Moving to projects directory: /data/slots/2/ICMSHhq0f+001896
Moving to projects directory: /data/slots/2/ICMUAhq0f+001896
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMGGhq0f+001872
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMSHhq0f+001872
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMUAhq0f+001872
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMGGhq0f+001884
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMSHhq0f+001884
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMUAhq0f+001884
Zipping up the intermediate file: /data/projects/climateprediction.net/oifs_43r3_ps_0859_1992050100_123_961_12178503_0_r1860973431_78.zip
Uploading the intermediate file: upload_file_78.zip
  08:25:35 STEP 1897 H=1897:00 +CPU= 19.575
  08:26:00 STEP 1898 H=1898:00 +CPU= 20.077
  08:26:32 STEP 1899 H=1899:00 +CPU= 29.743
  08:26:55 STEP 1900 H=1900:00 +CPU= 19.661
  08:27:18 STEP 1901 H=1901:00 +CPU= 19.435
  08:27:56 STEP 1902 H=1902:00 +CPU= 30.083
  08:28:16 STEP 1903 H=1903:00 +CPU= 18.908
  08:28:36 STEP 1904 H=1904:00 +CPU= 19.100
  08:29:06 STEP 1905 H=1905:00 +CPU= 29.154
  08:29:26 STEP 1906 H=1906:00 +CPU= 18.910
  08:29:45 STEP 1907 H=1907:00 +CPU= 18.354
  08:30:35 STEP 1908 H=1908:00 +CPU= 32.878
Moving to projects directory: /data/slots/2/ICMGGhq0f+001908
Moving to projects directory: /data/slots/2/ICMSHhq0f+001908
Moving to projects directory: /data/slots/2/ICMUAhq0f+001908
  08:31:12 STEP 1909 H=1909:00 +CPU= 19.541
  08:31:50 STEP 1910 H=1910:00 +CPU= 20.078
  08:32:47 STEP 1911 H=1911:00 +CPU= 30.570
  08:33:25 STEP 1912 H=1912:00 +CPU= 20.153
  08:34:01 STEP 1913 H=1913:00 +CPU= 19.830
  08:34:31 STEP 1914 H=1914:00 +CPU= 30.031
  08:34:51 STEP 1915 H=1915:00 +CPU= 18.630
  08:35:12 STEP 1916 H=1916:00 +CPU= 19.746
  08:35:41 STEP 1917 H=1917:00 +CPU= 28.986
  08:36:03 STEP 1918 H=1918:00 +CPU= 19.306
  08:36:30 STEP 1919 H=1919:00 +CPU= 19.472
  08:37:12 STEP 1920 H=1920:00 +CPU= 36.013
Moving to projects directory: /data/slots/2/ICMGGhq0f+001920
Moving to projects directory: /data/slots/2/ICMSHhq0f+001920
Moving to projects directory: /data/slots/2/ICMUAhq0f+001920
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMGGhq0f+001896
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMSHhq0f+001896
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMUAhq0f+001896
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMGGhq0f+001908
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMSHhq0f+001908
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMUAhq0f+001908
Zipping up the intermediate file: /data/projects/climateprediction.net/oifs_43r3_ps_0859_1992050100_123_961_12178503_0_r1860973431_79.zip
Uploading the intermediate file: upload_file_79.zip
  08:37:34 STEP 1921 H=1921:00 +CPU= 19.153
  08:37:57 STEP 1922 H=1922:00 +CPU= 19.478
  08:38:29 STEP 1923 H=1923:00 +CPU= 29.320
  08:38:50 STEP 1924 H=1924:00 +CPU= 19.216
  08:39:12 STEP 1925 H=1925:00 +CPU= 19.384
  08:39:44 STEP 1926 H=1926:00 +CPU= 29.659
  08:40:06 STEP 1927 H=1927:00 +CPU= 19.453
  08:40:28 STEP 1928 H=1928:00 +CPU= 19.625
  08:40:59 STEP 1929 H=1929:00 +CPU= 29.235
  08:41:20 STEP 1930 H=1930:00 +CPU= 19.824
  08:41:41 STEP 1931 H=1931:00 +CPU= 18.854
  08:42:17 STEP 1932 H=1932:00 +CPU= 32.710
Moving to projects directory: /data/slots/2/ICMGGhq0f+001932
Moving to projects directory: /data/slots/2/ICMSHhq0f+001932
Moving to projects directory: /data/slots/2/ICMUAhq0f+001932
  08:42:39 STEP 1933 H=1933:00 +CPU= 19.256
  08:43:00 STEP 1934 H=1934:00 +CPU= 19.308
  08:43:33 STEP 1935 H=1935:00 +CPU= 29.490
  08:43:54 STEP 1936 H=1936:00 +CPU= 19.274
  08:44:14 STEP 1937 H=1937:00 +CPU= 18.805
  08:44:45 STEP 1938 H=1938:00 +CPU= 29.802
  08:45:06 STEP 1939 H=1939:00 +CPU= 18.934
  08:45:25 STEP 1940 H=1940:00 +CPU= 19.196
  08:45:55 STEP 1941 H=1941:00 +CPU= 29.215
  08:46:16 STEP 1942 H=1942:00 +CPU= 19.527
  08:46:36 STEP 1943 H=1943:00 +CPU= 19.097
  08:47:24 STEP 1944 H=1944:00 +CPU= 35.226
Moving to projects directory: /data/slots/2/ICMGGhq0f+001944
Moving to projects directory: /data/slots/2/ICMSHhq0f+001944
Moving to projects directory: /data/slots/2/ICMUAhq0f+001944
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMGGhq0f+001920
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMSHhq0f+001920
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMUAhq0f+001920
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMGGhq0f+001932
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMSHhq0f+001932
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMUAhq0f+001932
Zipping up the intermediate file: /data/projects/climateprediction.net/oifs_43r3_ps_0859_1992050100_123_961_12178503_0_r1860973431_80.zip
Uploading the intermediate file: upload_file_80.zip
  08:47:44 STEP 1945 H=1945:00 +CPU= 18.671
  08:48:04 STEP 1946 H=1946:00 +CPU= 19.353
  08:48:35 STEP 1947 H=1947:00 +CPU= 29.132
  08:48:54 STEP 1948 H=1948:00 +CPU= 19.143
  08:49:14 STEP 1949 H=1949:00 +CPU= 18.806
  08:49:44 STEP 1950 H=1950:00 +CPU= 29.543
  08:50:03 STEP 1951 H=1951:00 +CPU= 18.679
  08:50:24 STEP 1952 H=1952:00 +CPU= 19.346
  08:50:55 STEP 1953 H=1953:00 +CPU= 29.054
  08:51:15 STEP 1954 H=1954:00 +CPU= 19.218
  08:51:35 STEP 1955 H=1955:00 +CPU= 18.676
  08:52:07 STEP 1956 H=1956:00 +CPU= 31.919
Moving to projects directory: /data/slots/2/ICMGGhq0f+001956
Moving to projects directory: /data/slots/2/ICMSHhq0f+001956
Moving to projects directory: /data/slots/2/ICMUAhq0f+001956
  08:52:28 STEP 1957 H=1957:00 +CPU= 18.674
  08:52:47 STEP 1958 H=1958:00 +CPU= 19.032
  08:53:19 STEP 1959 H=1959:00 +CPU= 29.451
  08:53:38 STEP 1960 H=1960:00 +CPU= 19.020
  08:53:58 STEP 1961 H=1961:00 +CPU= 18.436
  08:54:30 STEP 1962 H=1962:00 +CPU= 30.019
  08:54:51 STEP 1963 H=1963:00 +CPU= 18.462
  08:55:11 STEP 1964 H=1964:00 +CPU= 18.967
  08:55:41 STEP 1965 H=1965:00 +CPU= 28.988
  08:56:01 STEP 1966 H=1966:00 +CPU= 19.008
  08:56:22 STEP 1967 H=1967:00 +CPU= 19.444
  08:56:59 STEP 1968 H=1968:00 +CPU= 35.022
Moving to projects directory: /data/slots/2/ICMGGhq0f+001968
Moving to projects directory: /data/slots/2/ICMSHhq0f+001968
Moving to projects directory: /data/slots/2/ICMUAhq0f+001968
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMGGhq0f+001944
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMSHhq0f+001944
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMUAhq0f+001944
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMGGhq0f+001956
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMSHhq0f+001956
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMUAhq0f+001956
Zipping up the intermediate file: /data/projects/climateprediction.net/oifs_43r3_ps_0859_1992050100_123_961_12178503_0_r1860973431_81.zip
Uploading the intermediate file: upload_file_81.zip
  08:57:19 STEP 1969 H=1969:00 +CPU= 18.534
  08:57:39 STEP 1970 H=1970:00 +CPU= 18.967
  08:58:10 STEP 1971 H=1971:00 +CPU= 28.964
  08:58:30 STEP 1972 H=1972:00 +CPU= 19.235
  08:58:49 STEP 1973 H=1973:00 +CPU= 18.509
  08:59:20 STEP 1974 H=1974:00 +CPU= 29.672
  08:59:39 STEP 1975 H=1975:00 +CPU= 18.740
  08:59:59 STEP 1976 H=1976:00 +CPU= 19.060
  09:00:52 STEP 1977 H=1977:00 +CPU= 30.066
  09:01:30 STEP 1978 H=1978:00 +CPU= 19.852
  09:02:05 STEP 1979 H=1979:00 +CPU= 19.364
  09:03:08 STEP 1980 H=1980:00 +CPU= 33.294
Moving to projects directory: /data/slots/2/ICMGGhq0f+001980
Moving to projects directory: /data/slots/2/ICMSHhq0f+001980
Moving to projects directory: /data/slots/2/ICMUAhq0f+001980
  09:03:45 STEP 1981 H=1981:00 +CPU= 19.337
  09:04:12 STEP 1982 H=1982:00 +CPU= 19.505
  09:04:46 STEP 1983 H=1983:00 +CPU= 29.134
  09:05:09 STEP 1984 H=1984:00 +CPU= 19.044
  09:05:30 STEP 1985 H=1985:00 +CPU= 18.454
  09:06:00 STEP 1986 H=1986:00 +CPU= 29.439
  09:06:20 STEP 1987 H=1987:00 +CPU= 18.745
  09:06:41 STEP 1988 H=1988:00 +CPU= 19.240
  09:07:11 STEP 1989 H=1989:00 +CPU= 29.180
  09:07:32 STEP 1990 H=1990:00 +CPU= 18.867
  09:07:51 STEP 1991 H=1991:00 +CPU= 18.843
  09:08:30 STEP 1992 H=1992:00 +CPU= 34.771
Moving to projects directory: /data/slots/2/ICMGGhq0f+001992
Moving to projects directory: /data/slots/2/ICMSHhq0f+001992
Moving to projects directory: /data/slots/2/ICMUAhq0f+001992
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMGGhq0f+001968
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMSHhq0f+001968
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMUAhq0f+001968
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMGGhq0f+001980
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMSHhq0f+001980
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMUAhq0f+001980
Zipping up the intermediate file: /data/projects/climateprediction.net/oifs_43r3_ps_0859_1992050100_123_961_12178503_0_r1860973431_82.zip
Uploading the intermediate file: upload_file_82.zip
  09:08:50 STEP 1993 H=1993:00 +CPU= 18.436
  09:09:10 STEP 1994 H=1994:00 +CPU= 19.009
  09:09:41 STEP 1995 H=1995:00 +CPU= 29.440
  09:10:01 STEP 1996 H=1996:00 +CPU= 18.914
  09:10:20 STEP 1997 H=1997:00 +CPU= 18.442
  09:10:52 STEP 1998 H=1998:00 +CPU= 29.414
  09:11:12 STEP 1999 H=1999:00 +CPU= 18.440
  09:11:32 STEP 2000 H=2000:00 +CPU= 18.931
  09:12:04 STEP 2001 H=2001:00 +CPU= 29.264
  09:12:24 STEP 2002 H=2002:00 +CPU= 19.351
  09:12:45 STEP 2003 H=2003:00 +CPU= 18.748
  09:13:18 STEP 2004 H=2004:00 +CPU= 31.962
Moving to projects directory: /data/slots/2/ICMGGhq0f+002004
Moving to projects directory: /data/slots/2/ICMSHhq0f+002004
Moving to projects directory: /data/slots/2/ICMUAhq0f+002004
  09:13:37 STEP 2005 H=2005:00 +CPU= 18.688
  09:13:58 STEP 2006 H=2006:00 +CPU= 18.889
  09:14:28 STEP 2007 H=2007:00 +CPU= 29.003
  09:14:49 STEP 2008 H=2008:00 +CPU= 19.293
  09:15:08 STEP 2009 H=2009:00 +CPU= 18.469
  09:15:39 STEP 2010 H=2010:00 +CPU= 29.454
  09:15:59 STEP 2011 H=2011:00 +CPU= 18.900
  09:16:19 STEP 2012 H=2012:00 +CPU= 18.952
  09:16:52 STEP 2013 H=2013:00 +CPU= 30.503
  09:17:13 STEP 2014 H=2014:00 +CPU= 20.400
  09:17:37 STEP 2015 H=2015:00 +CPU= 19.446
  09:18:14 STEP 2016 H=2016:00 +CPU= 34.938
Moving to projects directory: /data/slots/2/ICMGGhq0f+002016
Moving to projects directory: /data/slots/2/ICMSHhq0f+002016
Moving to projects directory: /data/slots/2/ICMUAhq0f+002016
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMGGhq0f+001992
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMSHhq0f+001992
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMUAhq0f+001992
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMGGhq0f+002004
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMSHhq0f+002004
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMUAhq0f+002004
Zipping up the intermediate file: /data/projects/climateprediction.net/oifs_43r3_ps_0859_1992050100_123_961_12178503_0_r1860973431_83.zip
Uploading the intermediate file: upload_file_83.zip
  09:18:33 STEP 2017 H=2017:00 +CPU= 18.471
  09:18:54 STEP 2018 H=2018:00 +CPU= 18.901
  09:19:23 STEP 2019 H=2019:00 +CPU= 28.909
  09:19:44 STEP 2020 H=2020:00 +CPU= 19.040
  09:20:03 STEP 2021 H=2021:00 +CPU= 18.376
  09:20:33 STEP 2022 H=2022:00 +CPU= 29.509
  09:20:54 STEP 2023 H=2023:00 +CPU= 18.822
  09:21:14 STEP 2024 H=2024:00 +CPU= 19.314
  09:21:45 STEP 2025 H=2025:00 +CPU= 29.193
  09:22:04 STEP 2026 H=2026:00 +CPU= 19.003
  09:22:24 STEP 2027 H=2027:00 +CPU= 18.457
  09:22:59 STEP 2028 H=2028:00 +CPU= 32.285
Moving to projects directory: /data/slots/2/ICMGGhq0f+002028
Moving to projects directory: /data/slots/2/ICMSHhq0f+002028
Moving to projects directory: /data/slots/2/ICMUAhq0f+002028
  09:23:19 STEP 2029 H=2029:00 +CPU= 19.100
  09:23:39 STEP 2030 H=2030:00 +CPU= 19.296
  09:24:10 STEP 2031 H=2031:00 +CPU= 29.445
  09:24:29 STEP 2032 H=2032:00 +CPU= 18.910
  09:24:49 STEP 2033 H=2033:00 +CPU= 18.521
  09:25:20 STEP 2034 H=2034:00 +CPU= 29.717
  09:25:39 STEP 2035 H=2035:00 +CPU= 18.454
  09:25:59 STEP 2036 H=2036:00 +CPU= 19.271
  09:26:29 STEP 2037 H=2037:00 +CPU= 29.019
  09:26:49 STEP 2038 H=2038:00 +CPU= 18.888
  09:27:09 STEP 2039 H=2039:00 +CPU= 19.057
  09:27:46 STEP 2040 H=2040:00 +CPU= 34.869
Moving to projects directory: /data/slots/2/ICMGGhq0f+002040
Moving to projects directory: /data/slots/2/ICMSHhq0f+002040
Moving to projects directory: /data/slots/2/ICMUAhq0f+002040
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMGGhq0f+002016
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMSHhq0f+002016
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMUAhq0f+002016
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMGGhq0f+002028
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMSHhq0f+002028
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMUAhq0f+002028
Zipping up the intermediate file: /data/projects/climateprediction.net/oifs_43r3_ps_0859_1992050100_123_961_12178503_0_r1860973431_84.zip
Uploading the intermediate file: upload_file_84.zip
  09:28:06 STEP 2041 H=2041:00 +CPU= 18.537
Uploading trickle at timestep: 7344000
  09:28:26 STEP 2042 H=2042:00 +CPU= 18.938
  09:28:56 STEP 2043 H=2043:00 +CPU= 28.762
  09:29:16 STEP 2044 H=2044:00 +CPU= 18.895
  09:29:35 STEP 2045 H=2045:00 +CPU= 18.454
  09:30:13 STEP 2046 H=2046:00 +CPU= 29.775
  09:30:49 STEP 2047 H=2047:00 +CPU= 19.293
  09:31:24 STEP 2048 H=2048:00 +CPU= 19.705
  09:32:20 STEP 2049 H=2049:00 +CPU= 30.174
  09:32:58 STEP 2050 H=2050:00 +CPU= 20.309
  09:33:35 STEP 2051 H=2051:00 +CPU= 19.839
  09:34:20 STEP 2052 H=2052:00 +CPU= 32.602
Moving to projects directory: /data/slots/2/ICMGGhq0f+002052
Moving to projects directory: /data/slots/2/ICMSHhq0f+002052
Moving to projects directory: /data/slots/2/ICMUAhq0f+002052
  09:34:40 STEP 2053 H=2053:00 +CPU= 19.589
  09:35:01 STEP 2054 H=2054:00 +CPU= 19.562
  09:35:31 STEP 2055 H=2055:00 +CPU= 29.105
  09:35:52 STEP 2056 H=2056:00 +CPU= 18.891
  09:36:14 STEP 2057 H=2057:00 +CPU= 18.645
  09:36:46 STEP 2058 H=2058:00 +CPU= 29.511
  09:37:07 STEP 2059 H=2059:00 +CPU= 18.734
  09:37:28 STEP 2060 H=2060:00 +CPU= 18.913
  09:38:00 STEP 2061 H=2061:00 +CPU= 29.120
  09:38:20 STEP 2062 H=2062:00 +CPU= 19.012
  09:38:39 STEP 2063 H=2063:00 +CPU= 18.761
  09:39:23 STEP 2064 H=2064:00 +CPU= 34.935
Moving to projects directory: /data/slots/2/ICMGGhq0f+002064
Moving to projects directory: /data/slots/2/ICMSHhq0f+002064
Moving to projects directory: /data/slots/2/ICMUAhq0f+002064
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMGGhq0f+002040
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMSHhq0f+002040
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMUAhq0f+002040
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMGGhq0f+002052
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMSHhq0f+002052
Adding to the zip: /data/projects/climateprediction.net/oifs_43r3_ps_12178503/ICMUAhq0f+002052
Zipping up the intermediate file: /data/projects/climateprediction.net/oifs_43r3_ps_0859_1992050100_123_961_12178503_0_r1860973431_85.zip
Uploading the intermediate file: upload_file_85.zip
  09:39:43 STEP 2065 H=2065:00 +CPU= 18.436
  09:40:04 STEP 2066 H=2066:00 +CPU= 19.038
  09:40:35 STEP 2067 H=2067:00 +CPU= 28.954
  09:40:56 STEP 2068 H=2068:00 +CPU= 19.308
  09:41:16 STEP 2069 H=2069:00 +CPU= 18.613
  09:41:47 STEP 2070 H=2070:00 +CPU= 29.469
  09:42:06 STEP 2071 H=2071:00 +CPU= 18.570
  09:42:25 STEP 2072 H=2072:00 +CPU= 18.832
  09:42:56 STEP 2073 H=2073:00 +CPU= 28.898
  09:43:17 STEP 2074 H=2074:00 +CPU= 18.973
  09:43:37 STEP 2075 H=2075:00 +CPU= 18.411
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
The child process terminated with status: 0
..Failed, model did not complete successfully

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 May 2023 15:58:28 1504164 22263367 oifs_43r3_ps_0859_1992050100_123_961_12178503_0 7,344,000 48,750 0.0066
09 May 2023 15:58:21 1504164 22263367 oifs_43r3_ps_0859_1992050100_123_961_12178503_0 6,480,000 43,256 0.0067
09 May 2023 15:58:14 1504164 22263367 oifs_43r3_ps_0859_1992050100_123_961_12178503_0 5,616,000 37,855 0.0067
09 May 2023 15:58:07 1504164 22263367 oifs_43r3_ps_0859_1992050100_123_961_12178503_0 4,752,000 32,319 0.0068
09 May 2023 15:58:01 1504164 22263367 oifs_43r3_ps_0859_1992050100_123_961_12178503_0 3,888,000 26,880 0.0069
09 May 2023 15:57:53 1504164 22263367 oifs_43r3_ps_0859_1992050100_123_961_12178503_0 3,024,000 21,174 0.0070
09 May 2023 15:57:44 1504164 22263367 oifs_43r3_ps_0859_1992050100_123_961_12178503_0 2,160,000 14,672 0.0068
09 May 2023 15:57:36 1504164 22263367 oifs_43r3_ps_0859_1992050100_123_961_12178503_0 1,296,000 9,017 0.0070


©2024 cpdn.org