climateprediction.net home page
Task 22447805

Task 22447805

Name oifs_43r3_bl_a4it_2016092300_20_1018_12294188_0
Workunit 12294188
Created 13 Jun 2024, 4:29:59 UTC
Sent 14 Jun 2024, 0:00:43 UTC
Report deadline 13 Aug 2024, 0:00:43 UTC
Received 18 Jun 2024, 15:24:44 UTC
Server state Over
Outcome Success
Client state Done
Exit status 0 (0x00000000)
Computer ID 1542649
Run time 20 hours 4 min 39 sec
CPU time 11 hours 29 min 45 sec
Validate state Valid
Credit 2,635.55
Device peak FLOPS 4.95 GFLOPS
Application version OpenIFS 43r3 Baroclinic Lifecycle v1.13
x86_64-pc-linux-gnu
Peak working set size 5,024.82 MB
Peak swap size 5,981.07 MB
Peak disk usage 2,771.20 MB
Stderr
<core_client_version>7.18.1</core_client_version>
<![CDATA[
<stderr_txt>
 /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMUAh7zg+001380
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMGGh7zg+001392
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMSHh7zg+001392
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMUAh7zg+001392
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMGGh7zg+001404
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMSHh7zg+001404
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMUAh7zg+001404
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMGGh7zg+001416
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMSHh7zg+001416
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMUAh7zg+001416
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMGGh7zg+001428
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMSHh7zg+001428
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMUAh7zg+001428
Zipping up the intermediate file: /home/amar/projects/climateprediction.net/oifs_43r3_bl_a4it_2016092300_20_1018_12294188_0_r48016068_14.zip
Uploading the intermediate file: upload_file_14.zip
  09:16:44 STEP 1441 H= 360:15 +CPU= 20.794
  09:17:19 STEP 1442 H= 360:30 +CPU= 20.127
  09:17:53 STEP 1443 H= 360:45 +CPU= 19.985
  09:18:28 STEP 1444 H= 361:00 +CPU= 20.510
  09:19:01 STEP 1445 H= 361:15 +CPU= 20.041
  09:19:35 STEP 1446 H= 361:30 +CPU= 19.986
  09:20:10 STEP 1447 H= 361:45 +CPU= 20.272
  09:20:43 STEP 1448 H= 362:00 +CPU= 20.294
  09:21:15 STEP 1449 H= 362:15 +CPU= 20.060
  09:21:48 STEP 1450 H= 362:30 +CPU= 19.917
  09:22:23 STEP 1451 H= 362:45 +CPU= 20.059
  09:23:07 STEP 1452 H= 363:00 +CPU= 25.306
Moving to projects directory: /home/amar/slots/0/ICMGGh7zg+001452
Moving to projects directory: /home/amar/slots/0/ICMSHh7zg+001452
Moving to projects directory: /home/amar/slots/0/ICMUAh7zg+001452
  09:23:41 STEP 1453 H= 363:15 +CPU= 19.971
  09:24:14 STEP 1454 H= 363:30 +CPU= 20.142
  09:24:48 STEP 1455 H= 363:45 +CPU= 20.026
  09:25:23 STEP 1456 H= 364:00 +CPU= 20.534
  09:25:56 STEP 1457 H= 364:15 +CPU= 19.966
  09:26:31 STEP 1458 H= 364:30 +CPU= 20.122
  09:27:03 STEP 1459 H= 364:45 +CPU= 20.009
  09:27:36 STEP 1460 H= 365:00 +CPU= 20.275
  09:28:09 STEP 1461 H= 365:15 +CPU= 19.995
  09:28:42 STEP 1462 H= 365:30 +CPU= 19.962
  09:29:17 STEP 1463 H= 365:45 +CPU= 20.078
  09:29:59 STEP 1464 H= 366:00 +CPU= 25.027
Moving to projects directory: /home/amar/slots/0/ICMGGh7zg+001464
Moving to projects directory: /home/amar/slots/0/ICMSHh7zg+001464
Moving to projects directory: /home/amar/slots/0/ICMUAh7zg+001464
  09:30:33 STEP 1465 H= 366:15 +CPU= 20.170
  09:31:06 STEP 1466 H= 366:30 +CPU= 20.084
  09:31:40 STEP 1467 H= 366:45 +CPU= 19.935
  09:32:15 STEP 1468 H= 367:00 +CPU= 20.477
  09:32:48 STEP 1469 H= 367:15 +CPU= 19.951
  09:33:22 STEP 1470 H= 367:30 +CPU= 20.103
  09:33:55 STEP 1471 H= 367:45 +CPU= 19.919
  09:34:28 STEP 1472 H= 368:00 +CPU= 20.347
  09:35:01 STEP 1473 H= 368:15 +CPU= 20.059
  09:35:37 STEP 1474 H= 368:30 +CPU= 20.163
  09:36:09 STEP 1475 H= 368:45 +CPU= 20.031
  09:36:52 STEP 1476 H= 369:00 +CPU= 24.992
Moving to projects directory: /home/amar/slots/0/ICMGGh7zg+001476
Moving to projects directory: /home/amar/slots/0/ICMSHh7zg+001476
Moving to projects directory: /home/amar/slots/0/ICMUAh7zg+001476
  09:37:27 STEP 1477 H= 369:15 +CPU= 20.176
  09:38:01 STEP 1478 H= 369:30 +CPU= 19.967
  09:38:33 STEP 1479 H= 369:45 +CPU= 20.083
  09:39:09 STEP 1480 H= 370:00 +CPU= 20.413
  09:39:43 STEP 1481 H= 370:15 +CPU= 19.958
  09:40:16 STEP 1482 H= 370:30 +CPU= 20.158
  09:40:49 STEP 1483 H= 370:45 +CPU= 19.906
  09:41:24 STEP 1484 H= 371:00 +CPU= 20.451
  09:41:58 STEP 1485 H= 371:15 +CPU= 19.898
  09:42:30 STEP 1486 H= 371:30 +CPU= 20.043
  09:43:04 STEP 1487 H= 371:45 +CPU= 20.001
  09:43:52 STEP 1488 H= 372:00 +CPU= 27.715
Moving to projects directory: /home/amar/slots/0/ICMGGh7zg+001488
Moving to projects directory: /home/amar/slots/0/ICMSHh7zg+001488
Moving to projects directory: /home/amar/slots/0/ICMUAh7zg+001488
  09:44:25 STEP 1489 H= 372:15 +CPU= 19.992
  09:44:57 STEP 1490 H= 372:30 +CPU= 19.998
  09:45:30 STEP 1491 H= 372:45 +CPU= 20.064
  09:46:05 STEP 1492 H= 373:00 +CPU= 20.395
  09:46:37 STEP 1493 H= 373:15 +CPU= 19.905
  09:47:10 STEP 1494 H= 373:30 +CPU= 20.030
  09:47:43 STEP 1495 H= 373:45 +CPU= 19.930
  09:48:16 STEP 1496 H= 374:00 +CPU= 20.396
  09:48:50 STEP 1497 H= 374:15 +CPU= 20.054
  09:49:24 STEP 1498 H= 374:30 +CPU= 20.108
  09:49:58 STEP 1499 H= 374:45 +CPU= 19.968
  09:50:41 STEP 1500 H= 375:00 +CPU= 25.183
Moving to projects directory: /home/amar/slots/0/ICMGGh7zg+001500
Moving to projects directory: /home/amar/slots/0/ICMSHh7zg+001500
Moving to projects directory: /home/amar/slots/0/ICMUAh7zg+001500
  09:51:15 STEP 1501 H= 375:15 +CPU= 20.036
  09:51:47 STEP 1502 H= 375:30 +CPU= 19.856
  09:52:21 STEP 1503 H= 375:45 +CPU= 20.001
  09:52:55 STEP 1504 H= 376:00 +CPU= 20.362
  09:53:30 STEP 1505 H= 376:15 +CPU= 20.134
  09:54:04 STEP 1506 H= 376:30 +CPU= 20.214
  09:54:37 STEP 1507 H= 376:45 +CPU= 19.975
  09:55:12 STEP 1508 H= 377:00 +CPU= 20.637
  09:55:47 STEP 1509 H= 377:15 +CPU= 20.100
  09:56:22 STEP 1510 H= 377:30 +CPU= 20.066
  09:56:54 STEP 1511 H= 377:45 +CPU= 19.977
  09:57:37 STEP 1512 H= 378:00 +CPU= 25.058
Moving to projects directory: /home/amar/slots/0/ICMGGh7zg+001512
Moving to projects directory: /home/amar/slots/0/ICMSHh7zg+001512
Moving to projects directory: /home/amar/slots/0/ICMUAh7zg+001512
  09:58:11 STEP 1513 H= 378:15 +CPU= 20.050
  09:58:44 STEP 1514 H= 378:30 +CPU= 19.912
  09:59:19 STEP 1515 H= 378:45 +CPU= 20.163
  09:59:52 STEP 1516 H= 379:00 +CPU= 20.213
  10:00:26 STEP 1517 H= 379:15 +CPU= 20.206
  10:01:01 STEP 1518 H= 379:30 +CPU= 20.207
  10:01:34 STEP 1519 H= 379:45 +CPU= 19.961
  10:02:07 STEP 1520 H= 380:00 +CPU= 20.372
  10:02:41 STEP 1521 H= 380:15 +CPU= 19.923
  10:03:15 STEP 1522 H= 380:30 +CPU= 20.107
  10:03:48 STEP 1523 H= 380:45 +CPU= 19.954
  10:04:28 STEP 1524 H= 381:00 +CPU= 24.974
Moving to projects directory: /home/amar/slots/0/ICMGGh7zg+001524
Moving to projects directory: /home/amar/slots/0/ICMSHh7zg+001524
Moving to projects directory: /home/amar/slots/0/ICMUAh7zg+001524
  10:05:02 STEP 1525 H= 381:15 +CPU= 20.003
  10:05:37 STEP 1526 H= 381:30 +CPU= 20.081
  10:06:10 STEP 1527 H= 381:45 +CPU= 19.954
  10:06:44 STEP 1528 H= 382:00 +CPU= 20.329
  10:07:19 STEP 1529 H= 382:15 +CPU= 20.188
  10:07:52 STEP 1530 H= 382:30 +CPU= 19.988
  10:08:26 STEP 1531 H= 382:45 +CPU= 20.028
  10:09:00 STEP 1532 H= 383:00 +CPU= 20.306
  10:09:33 STEP 1533 H= 383:15 +CPU= 19.978
  10:10:08 STEP 1534 H= 383:30 +CPU= 20.130
  10:10:42 STEP 1535 H= 383:45 +CPU= 19.989
  10:11:28 STEP 1536 H= 384:00 +CPU= 27.766
Moving to projects directory: /home/amar/slots/0/ICMGGh7zg+001536
Moving to projects directory: /home/amar/slots/0/ICMSHh7zg+001536
Moving to projects directory: /home/amar/slots/0/ICMUAh7zg+001536
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMGGh7zg+001440
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMSHh7zg+001440
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMUAh7zg+001440
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMGGh7zg+001452
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMSHh7zg+001452
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMUAh7zg+001452
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMGGh7zg+001464
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMSHh7zg+001464
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMUAh7zg+001464
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMGGh7zg+001476
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMSHh7zg+001476
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMUAh7zg+001476
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMGGh7zg+001488
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMSHh7zg+001488
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMUAh7zg+001488
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMGGh7zg+001500
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMSHh7zg+001500
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMUAh7zg+001500
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMGGh7zg+001512
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMSHh7zg+001512
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMUAh7zg+001512
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMGGh7zg+001524
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMSHh7zg+001524
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMUAh7zg+001524
Zipping up the intermediate file: /home/amar/projects/climateprediction.net/oifs_43r3_bl_a4it_2016092300_20_1018_12294188_0_r48016068_15.zip
Uploading the intermediate file: upload_file_15.zip
  10:11:59 STEP 1537 H= 384:15 +CPU= 20.885
  10:12:37 STEP 1538 H= 384:30 +CPU= 20.265
  10:13:11 STEP 1539 H= 384:45 +CPU= 20.069
  10:13:45 STEP 1540 H= 385:00 +CPU= 20.264
  10:14:19 STEP 1541 H= 385:15 +CPU= 20.001
  10:14:51 STEP 1542 H= 385:30 +CPU= 19.927
  10:15:25 STEP 1543 H= 385:45 +CPU= 20.150
  10:16:00 STEP 1544 H= 386:00 +CPU= 20.331
  10:16:33 STEP 1545 H= 386:15 +CPU= 20.152
  10:17:07 STEP 1546 H= 386:30 +CPU= 20.106
  10:17:41 STEP 1547 H= 386:45 +CPU= 19.972
  10:18:24 STEP 1548 H= 387:00 +CPU= 25.090
Moving to projects directory: /home/amar/slots/0/ICMGGh7zg+001548
Moving to projects directory: /home/amar/slots/0/ICMSHh7zg+001548
Moving to projects directory: /home/amar/slots/0/ICMUAh7zg+001548
  10:18:57 STEP 1549 H= 387:15 +CPU= 20.005
  10:19:32 STEP 1550 H= 387:30 +CPU= 20.056
  10:20:05 STEP 1551 H= 387:45 +CPU= 19.984
  10:20:40 STEP 1552 H= 388:00 +CPU= 20.451
  10:21:14 STEP 1553 H= 388:15 +CPU= 20.171
  10:21:47 STEP 1554 H= 388:30 +CPU= 20.033
  10:22:20 STEP 1555 H= 388:45 +CPU= 19.974
  10:22:54 STEP 1556 H= 389:00 +CPU= 20.304
  10:23:28 STEP 1557 H= 389:15 +CPU= 20.096
  10:24:01 STEP 1558 H= 389:30 +CPU= 19.919
  10:24:34 STEP 1559 H= 389:45 +CPU= 19.968
  10:25:19 STEP 1560 H= 390:00 +CPU= 25.431
Moving to projects directory: /home/amar/slots/0/ICMGGh7zg+001560
Moving to projects directory: /home/amar/slots/0/ICMSHh7zg+001560
Moving to projects directory: /home/amar/slots/0/ICMUAh7zg+001560
  10:25:53 STEP 1561 H= 390:15 +CPU= 19.981
  10:26:27 STEP 1562 H= 390:30 +CPU= 20.014
  10:27:01 STEP 1563 H= 390:45 +CPU= 20.136
  10:27:37 STEP 1564 H= 391:00 +CPU= 20.285
  10:28:10 STEP 1565 H= 391:15 +CPU= 19.968
  10:28:43 STEP 1566 H= 391:30 +CPU= 19.948
  10:29:16 STEP 1567 H= 391:45 +CPU= 20.015
  10:29:50 STEP 1568 H= 392:00 +CPU= 20.220
  10:30:25 STEP 1569 H= 392:15 +CPU= 20.145
  10:30:58 STEP 1570 H= 392:30 +CPU= 20.020
  10:31:33 STEP 1571 H= 392:45 +CPU= 20.193
  10:32:15 STEP 1572 H= 393:00 +CPU= 25.160
Moving to projects directory: /home/amar/slots/0/ICMGGh7zg+001572
Moving to projects directory: /home/amar/slots/0/ICMSHh7zg+001572
Moving to projects directory: /home/amar/slots/0/ICMUAh7zg+001572
  10:32:47 STEP 1573 H= 393:15 +CPU= 19.897
  10:33:22 STEP 1574 H= 393:30 +CPU= 20.156
  10:33:55 STEP 1575 H= 393:45 +CPU= 19.915
  10:34:30 STEP 1576 H= 394:00 +CPU= 20.377
  10:35:03 STEP 1577 H= 394:15 +CPU= 20.082
  10:35:36 STEP 1578 H= 394:30 +CPU= 19.965
  10:36:08 STEP 1579 H= 394:45 +CPU= 19.999
  10:36:43 STEP 1580 H= 395:00 +CPU= 20.346
  10:37:16 STEP 1581 H= 395:15 +CPU= 20.134
  10:37:49 STEP 1582 H= 395:30 +CPU= 19.930
  10:38:23 STEP 1583 H= 395:45 +CPU= 20.036
  10:39:10 STEP 1584 H= 396:00 +CPU= 27.781
Moving to projects directory: /home/amar/slots/0/ICMGGh7zg+001584
Moving to projects directory: /home/amar/slots/0/ICMSHh7zg+001584
Moving to projects directory: /home/amar/slots/0/ICMUAh7zg+001584
  10:39:43 STEP 1585 H= 396:15 +CPU= 19.922
  10:40:16 STEP 1586 H= 396:30 +CPU= 20.064
  10:40:50 STEP 1587 H= 396:45 +CPU= 19.964
  10:41:24 STEP 1588 H= 397:00 +CPU= 20.437
  10:41:58 STEP 1589 H= 397:15 +CPU= 19.998
  10:42:31 STEP 1590 H= 397:30 +CPU= 20.029
  10:43:04 STEP 1591 H= 397:45 +CPU= 20.131
  10:43:42 STEP 1592 H= 398:00 +CPU= 20.854
  10:44:16 STEP 1593 H= 398:15 +CPU= 19.960
  10:44:49 STEP 1594 H= 398:30 +CPU= 20.035
  10:45:23 STEP 1595 H= 398:45 +CPU= 20.120
  10:46:05 STEP 1596 H= 399:00 +CPU= 24.997
Moving to projects directory: /home/amar/slots/0/ICMGGh7zg+001596
Moving to projects directory: /home/amar/slots/0/ICMSHh7zg+001596
Moving to projects directory: /home/amar/slots/0/ICMUAh7zg+001596
  10:46:38 STEP 1597 H= 399:15 +CPU= 19.982
  10:47:12 STEP 1598 H= 399:30 +CPU= 20.020
  10:47:45 STEP 1599 H= 399:45 +CPU= 20.006
  10:48:19 STEP 1600 H= 400:00 +CPU= 20.385
  10:48:53 STEP 1601 H= 400:15 +CPU= 20.010
  10:49:27 STEP 1602 H= 400:30 +CPU= 20.121
  10:50:00 STEP 1603 H= 400:45 +CPU= 20.000
  10:50:36 STEP 1604 H= 401:00 +CPU= 20.616
  10:51:08 STEP 1605 H= 401:15 +CPU= 20.078
  10:51:41 STEP 1606 H= 401:30 +CPU= 19.924
  10:52:15 STEP 1607 H= 401:45 +CPU= 20.085
  10:52:57 STEP 1608 H= 402:00 +CPU= 24.989
Moving to projects directory: /home/amar/slots/0/ICMGGh7zg+001608
Moving to projects directory: /home/amar/slots/0/ICMSHh7zg+001608
Moving to projects directory: /home/amar/slots/0/ICMUAh7zg+001608
  10:53:31 STEP 1609 H= 402:15 +CPU= 20.111
  10:54:04 STEP 1610 H= 402:30 +CPU= 20.086
  10:54:38 STEP 1611 H= 402:45 +CPU= 19.974
  10:55:07 STEP 1612 H= 403:00 +CPU= 20.258
  10:55:42 STEP 1613 H= 403:15 +CPU= 20.109
  10:56:16 STEP 1614 H= 403:30 +CPU= 20.108
  10:56:49 STEP 1615 H= 403:45 +CPU= 20.065
  10:57:23 STEP 1616 H= 404:00 +CPU= 20.346
  10:57:56 STEP 1617 H= 404:15 +CPU= 19.985
  10:58:30 STEP 1618 H= 404:30 +CPU= 20.197
  10:59:03 STEP 1619 H= 404:45 +CPU= 20.060
  10:59:45 STEP 1620 H= 405:00 +CPU= 24.940
Moving to projects directory: /home/amar/slots/0/ICMGGh7zg+001620
Moving to projects directory: /home/amar/slots/0/ICMSHh7zg+001620
Moving to projects directory: /home/amar/slots/0/ICMUAh7zg+001620
  11:00:19 STEP 1621 H= 405:15 +CPU= 20.086
  11:00:53 STEP 1622 H= 405:30 +CPU= 20.138
  11:01:26 STEP 1623 H= 405:45 +CPU= 20.119
  11:02:00 STEP 1624 H= 406:00 +CPU= 20.352
  11:02:34 STEP 1625 H= 406:15 +CPU= 20.037
  11:03:07 STEP 1626 H= 406:30 +CPU= 20.105
  11:03:41 STEP 1627 H= 406:45 +CPU= 19.985
  11:04:16 STEP 1628 H= 407:00 +CPU= 20.465
  11:04:50 STEP 1629 H= 407:15 +CPU= 19.919
  11:05:22 STEP 1630 H= 407:30 +CPU= 20.075
  11:05:56 STEP 1631 H= 407:45 +CPU= 20.045
  11:06:43 STEP 1632 H= 408:00 +CPU= 27.685
Moving to projects directory: /home/amar/slots/0/ICMGGh7zg+001632
Moving to projects directory: /home/amar/slots/0/ICMSHh7zg+001632
Moving to projects directory: /home/amar/slots/0/ICMUAh7zg+001632
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMGGh7zg+001536
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMSHh7zg+001536
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMUAh7zg+001536
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMGGh7zg+001548
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMSHh7zg+001548
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMUAh7zg+001548
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMGGh7zg+001560
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMSHh7zg+001560
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMUAh7zg+001560
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMGGh7zg+001572
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMSHh7zg+001572
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMUAh7zg+001572
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMGGh7zg+001584
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMSHh7zg+001584
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMUAh7zg+001584
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMGGh7zg+001596
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMSHh7zg+001596
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMUAh7zg+001596
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMGGh7zg+001608
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMSHh7zg+001608
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMUAh7zg+001608
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMGGh7zg+001620
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMSHh7zg+001620
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMUAh7zg+001620
Zipping up the intermediate file: /home/amar/projects/climateprediction.net/oifs_43r3_bl_a4it_2016092300_20_1018_12294188_0_r48016068_16.zip
Uploading the intermediate file: upload_file_16.zip
  11:07:14 STEP 1633 H= 408:15 +CPU= 21.183
  11:07:50 STEP 1634 H= 408:30 +CPU= 20.066
  11:08:25 STEP 1635 H= 408:45 +CPU= 20.152
  11:09:00 STEP 1636 H= 409:00 +CPU= 20.434
  11:09:33 STEP 1637 H= 409:15 +CPU= 20.209
  11:10:08 STEP 1638 H= 409:30 +CPU= 20.080
  11:10:42 STEP 1639 H= 409:45 +CPU= 20.177
  11:11:16 STEP 1640 H= 410:00 +CPU= 20.398
  11:11:48 STEP 1641 H= 410:15 +CPU= 19.870
  11:12:22 STEP 1642 H= 410:30 +CPU= 20.064
  11:12:55 STEP 1643 H= 410:45 +CPU= 19.989
  11:13:40 STEP 1644 H= 411:00 +CPU= 25.221
Moving to projects directory: /home/amar/slots/0/ICMGGh7zg+001644
Moving to projects directory: /home/amar/slots/0/ICMSHh7zg+001644
Moving to projects directory: /home/amar/slots/0/ICMUAh7zg+001644
  11:14:13 STEP 1645 H= 411:15 +CPU= 20.017
  11:14:47 STEP 1646 H= 411:30 +CPU= 20.077
  11:15:22 STEP 1647 H= 411:45 +CPU= 20.014
  11:15:57 STEP 1648 H= 412:00 +CPU= 20.505
  11:16:32 STEP 1649 H= 412:15 +CPU= 20.136
  11:17:04 STEP 1650 H= 412:30 +CPU= 20.009
  11:17:38 STEP 1651 H= 412:45 +CPU= 20.026
  11:18:13 STEP 1652 H= 413:00 +CPU= 20.364
  11:18:47 STEP 1653 H= 413:15 +CPU= 20.047
  11:19:22 STEP 1654 H= 413:30 +CPU= 20.219
  11:19:55 STEP 1655 H= 413:45 +CPU= 20.050
  11:20:36 STEP 1656 H= 414:00 +CPU= 25.134
Moving to projects directory: /home/amar/slots/0/ICMGGh7zg+001656
Moving to projects directory: /home/amar/slots/0/ICMSHh7zg+001656
Moving to projects directory: /home/amar/slots/0/ICMUAh7zg+001656
  11:21:10 STEP 1657 H= 414:15 +CPU= 20.088
  11:21:42 STEP 1658 H= 414:30 +CPU= 19.873
  11:22:15 STEP 1659 H= 414:45 +CPU= 19.991
  11:22:50 STEP 1660 H= 415:00 +CPU= 20.354
  11:23:23 STEP 1661 H= 415:15 +CPU= 20.001
  11:23:57 STEP 1662 H= 415:30 +CPU= 20.099
  11:24:31 STEP 1663 H= 415:45 +CPU= 20.022
  11:25:05 STEP 1664 H= 416:00 +CPU= 20.542
  11:25:38 STEP 1665 H= 416:15 +CPU= 20.031
  11:26:12 STEP 1666 H= 416:30 +CPU= 20.122
  11:26:46 STEP 1667 H= 416:45 +CPU= 20.032
  11:27:28 STEP 1668 H= 417:00 +CPU= 25.023
Moving to projects directory: /home/amar/slots/0/ICMGGh7zg+001668
Moving to projects directory: /home/amar/slots/0/ICMSHh7zg+001668
Moving to projects directory: /home/amar/slots/0/ICMUAh7zg+001668
  11:28:00 STEP 1669 H= 417:15 +CPU= 19.909
  11:28:34 STEP 1670 H= 417:30 +CPU= 19.977
  11:29:09 STEP 1671 H= 417:45 +CPU= 20.087
  11:29:43 STEP 1672 H= 418:00 +CPU= 20.287
  11:30:17 STEP 1673 H= 418:15 +CPU= 20.056
  11:30:50 STEP 1674 H= 418:30 +CPU= 20.055
  11:31:23 STEP 1675 H= 418:45 +CPU= 20.132
  11:31:58 STEP 1676 H= 419:00 +CPU= 20.303
  11:32:32 STEP 1677 H= 419:15 +CPU= 20.048
  11:33:05 STEP 1678 H= 419:30 +CPU= 20.034
  11:33:39 STEP 1679 H= 419:45 +CPU= 19.956
  11:34:26 STEP 1680 H= 420:00 +CPU= 27.843
Moving to projects directory: /home/amar/slots/0/ICMGGh7zg+001680
Moving to projects directory: /home/amar/slots/0/ICMSHh7zg+001680
Moving to projects directory: /home/amar/slots/0/ICMUAh7zg+001680
  11:35:00 STEP 1681 H= 420:15 +CPU= 19.949
  11:35:36 STEP 1682 H= 420:30 +CPU= 20.205
  11:36:11 STEP 1683 H= 420:45 +CPU= 20.240
  11:36:43 STEP 1684 H= 421:00 +CPU= 20.344
  11:37:18 STEP 1685 H= 421:15 +CPU= 20.346
  11:37:53 STEP 1686 H= 421:30 +CPU= 20.021
  11:38:26 STEP 1687 H= 421:45 +CPU= 20.097
  11:39:01 STEP 1688 H= 422:00 +CPU= 20.384
  11:39:35 STEP 1689 H= 422:15 +CPU= 20.193
  11:40:11 STEP 1690 H= 422:30 +CPU= 20.409
  11:40:46 STEP 1691 H= 422:45 +CPU= 20.134
  11:41:28 STEP 1692 H= 423:00 +CPU= 25.082
Moving to projects directory: /home/amar/slots/0/ICMGGh7zg+001692
Moving to projects directory: /home/amar/slots/0/ICMSHh7zg+001692
Moving to projects directory: /home/amar/slots/0/ICMUAh7zg+001692
  11:42:02 STEP 1693 H= 423:15 +CPU= 19.979
  11:42:35 STEP 1694 H= 423:30 +CPU= 19.951
  11:43:07 STEP 1695 H= 423:45 +CPU= 20.036
  11:43:42 STEP 1696 H= 424:00 +CPU= 20.371
  11:44:15 STEP 1697 H= 424:15 +CPU= 20.131
  11:44:50 STEP 1698 H= 424:30 +CPU= 20.024
  11:45:23 STEP 1699 H= 424:45 +CPU= 20.040
  11:45:57 STEP 1700 H= 425:00 +CPU= 20.359
  11:46:31 STEP 1701 H= 425:15 +CPU= 20.169
  11:47:05 STEP 1702 H= 425:30 +CPU= 20.021
  11:47:37 STEP 1703 H= 425:45 +CPU= 19.874
  11:48:20 STEP 1704 H= 426:00 +CPU= 25.102
Moving to projects directory: /home/amar/slots/0/ICMGGh7zg+001704
Moving to projects directory: /home/amar/slots/0/ICMSHh7zg+001704
Moving to projects directory: /home/amar/slots/0/ICMUAh7zg+001704
  11:48:53 STEP 1705 H= 426:15 +CPU= 19.839
  11:49:27 STEP 1706 H= 426:30 +CPU= 20.091
  11:50:02 STEP 1707 H= 426:45 +CPU= 20.075
  11:50:35 STEP 1708 H= 427:00 +CPU= 20.417
  11:51:08 STEP 1709 H= 427:15 +CPU= 20.067
  11:51:42 STEP 1710 H= 427:30 +CPU= 19.938
  11:52:16 STEP 1711 H= 427:45 +CPU= 20.063
  11:52:50 STEP 1712 H= 428:00 +CPU= 20.320
  11:53:23 STEP 1713 H= 428:15 +CPU= 20.114
  11:53:56 STEP 1714 H= 428:30 +CPU= 19.955
  11:54:30 STEP 1715 H= 428:45 +CPU= 20.042
  11:55:12 STEP 1716 H= 429:00 +CPU= 25.248
Moving to projects directory: /home/amar/slots/0/ICMGGh7zg+001716
Moving to projects directory: /home/amar/slots/0/ICMSHh7zg+001716
Moving to projects directory: /home/amar/slots/0/ICMUAh7zg+001716
  11:55:47 STEP 1717 H= 429:15 +CPU= 20.336
  11:56:23 STEP 1718 H= 429:30 +CPU= 20.266
  11:56:56 STEP 1719 H= 429:45 +CPU= 19.967
  11:57:31 STEP 1720 H= 430:00 +CPU= 20.554
  11:58:06 STEP 1721 H= 430:15 +CPU= 20.191
  11:58:39 STEP 1722 H= 430:30 +CPU= 19.848
  11:59:13 STEP 1723 H= 430:45 +CPU= 20.179
  11:59:47 STEP 1724 H= 431:00 +CPU= 20.284
  12:00:21 STEP 1725 H= 431:15 +CPU= 20.199
  12:00:56 STEP 1726 H= 431:30 +CPU= 20.140
  12:01:30 STEP 1727 H= 431:45 +CPU= 20.151
  12:02:16 STEP 1728 H= 432:00 +CPU= 27.644
Moving to projects directory: /home/amar/slots/0/ICMGGh7zg+001728
Moving to projects directory: /home/amar/slots/0/ICMSHh7zg+001728
Moving to projects directory: /home/amar/slots/0/ICMUAh7zg+001728
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMGGh7zg+001632
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMSHh7zg+001632
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMUAh7zg+001632
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMGGh7zg+001644
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMSHh7zg+001644
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMUAh7zg+001644
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMGGh7zg+001656
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMSHh7zg+001656
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMUAh7zg+001656
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMGGh7zg+001668
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMSHh7zg+001668
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMUAh7zg+001668
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMGGh7zg+001680
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMSHh7zg+001680
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMUAh7zg+001680
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMGGh7zg+001692
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMSHh7zg+001692
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMUAh7zg+001692
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMGGh7zg+001704
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMSHh7zg+001704
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMUAh7zg+001704
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMGGh7zg+001716
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMSHh7zg+001716
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMUAh7zg+001716
Zipping up the intermediate file: /home/amar/projects/climateprediction.net/oifs_43r3_bl_a4it_2016092300_20_1018_12294188_0_r48016068_17.zip
Uploading the intermediate file: upload_file_17.zip
  12:02:49 STEP 1729 H= 432:15 +CPU= 20.997
  12:03:25 STEP 1730 H= 432:30 +CPU= 20.227
  12:03:59 STEP 1731 H= 432:45 +CPU= 19.978
  12:04:33 STEP 1732 H= 433:00 +CPU= 20.447
  12:05:07 STEP 1733 H= 433:15 +CPU= 20.087
  12:05:40 STEP 1734 H= 433:30 +CPU= 20.130
  12:06:16 STEP 1735 H= 433:45 +CPU= 20.235
  12:06:50 STEP 1736 H= 434:00 +CPU= 20.327
  12:07:24 STEP 1737 H= 434:15 +CPU= 20.254
  12:07:58 STEP 1738 H= 434:30 +CPU= 20.068
  12:08:34 STEP 1739 H= 434:45 +CPU= 20.277
  12:09:17 STEP 1740 H= 435:00 +CPU= 25.265
Moving to projects directory: /home/amar/slots/0/ICMGGh7zg+001740
Moving to projects directory: /home/amar/slots/0/ICMSHh7zg+001740
Moving to projects directory: /home/amar/slots/0/ICMUAh7zg+001740
  12:09:50 STEP 1741 H= 435:15 +CPU= 19.992
  12:10:26 STEP 1742 H= 435:30 +CPU= 20.135
  12:11:00 STEP 1743 H= 435:45 +CPU= 20.050
  12:11:34 STEP 1744 H= 436:00 +CPU= 20.388
  12:12:07 STEP 1745 H= 436:15 +CPU= 20.019
  12:12:40 STEP 1746 H= 436:30 +CPU= 19.881
  12:13:14 STEP 1747 H= 436:45 +CPU= 20.228
  12:13:48 STEP 1748 H= 437:00 +CPU= 20.239
  12:14:22 STEP 1749 H= 437:15 +CPU= 20.038
  12:14:55 STEP 1750 H= 437:30 +CPU= 19.974
  12:15:29 STEP 1751 H= 437:45 +CPU= 20.043
  12:16:13 STEP 1752 H= 438:00 +CPU= 25.481
Moving to projects directory: /home/amar/slots/0/ICMGGh7zg+001752
Moving to projects directory: /home/amar/slots/0/ICMSHh7zg+001752
Moving to projects directory: /home/amar/slots/0/ICMUAh7zg+001752
  12:16:46 STEP 1753 H= 438:15 +CPU= 20.053
  12:17:20 STEP 1754 H= 438:30 +CPU= 20.121
  12:17:53 STEP 1755 H= 438:45 +CPU= 19.991
  12:18:28 STEP 1756 H= 439:00 +CPU= 20.355
  12:19:02 STEP 1757 H= 439:15 +CPU= 20.155
  12:19:36 STEP 1758 H= 439:30 +CPU= 19.997
  12:20:09 STEP 1759 H= 439:45 +CPU= 20.053
  12:20:44 STEP 1760 H= 440:00 +CPU= 20.444
  12:21:19 STEP 1761 H= 440:15 +CPU= 20.189
  12:21:52 STEP 1762 H= 440:30 +CPU= 19.956
  12:22:26 STEP 1763 H= 440:45 +CPU= 20.024
  12:23:09 STEP 1764 H= 441:00 +CPU= 25.074
Moving to projects directory: /home/amar/slots/0/ICMGGh7zg+001764
Moving to projects directory: /home/amar/slots/0/ICMSHh7zg+001764
Moving to projects directory: /home/amar/slots/0/ICMUAh7zg+001764
  12:23:42 STEP 1765 H= 441:15 +CPU= 20.040
  12:24:15 STEP 1766 H= 441:30 +CPU= 19.979
  12:24:48 STEP 1767 H= 441:45 +CPU= 20.002
  12:25:24 STEP 1768 H= 442:00 +CPU= 20.518
  12:25:57 STEP 1769 H= 442:15 +CPU= 20.057
  12:26:31 STEP 1770 H= 442:30 +CPU= 20.035
  12:27:04 STEP 1771 H= 442:45 +CPU= 20.024
  12:27:38 STEP 1772 H= 443:00 +CPU= 20.306
  12:28:12 STEP 1773 H= 443:15 +CPU= 20.005
  12:28:44 STEP 1774 H= 443:30 +CPU= 19.949
  12:29:17 STEP 1775 H= 443:45 +CPU= 20.061
  12:30:04 STEP 1776 H= 444:00 +CPU= 27.765
Moving to projects directory: /home/amar/slots/0/ICMGGh7zg+001776
Moving to projects directory: /home/amar/slots/0/ICMSHh7zg+001776
Moving to projects directory: /home/amar/slots/0/ICMUAh7zg+001776
  12:30:39 STEP 1777 H= 444:15 +CPU= 20.102
  12:31:13 STEP 1778 H= 444:30 +CPU= 20.279
  12:31:45 STEP 1779 H= 444:45 +CPU= 19.964
  12:32:19 STEP 1780 H= 445:00 +CPU= 20.434
  12:32:51 STEP 1781 H= 445:15 +CPU= 19.998
  12:33:25 STEP 1782 H= 445:30 +CPU= 20.072
  12:33:58 STEP 1783 H= 445:45 +CPU= 20.020
  12:34:33 STEP 1784 H= 446:00 +CPU= 20.426
  12:35:08 STEP 1785 H= 446:15 +CPU= 20.123
  12:35:41 STEP 1786 H= 446:30 +CPU= 20.024
  12:36:15 STEP 1787 H= 446:45 +CPU= 20.141
  12:36:57 STEP 1788 H= 447:00 +CPU= 25.075
Moving to projects directory: /home/amar/slots/0/ICMGGh7zg+001788
Moving to projects directory: /home/amar/slots/0/ICMSHh7zg+001788
Moving to projects directory: /home/amar/slots/0/ICMUAh7zg+001788
  12:37:32 STEP 1789 H= 447:15 +CPU= 20.311
  12:38:07 STEP 1790 H= 447:30 +CPU= 20.136
  12:38:38 STEP 1791 H= 447:45 +CPU= 19.946
  12:39:13 STEP 1792 H= 448:00 +CPU= 20.504
  12:39:46 STEP 1793 H= 448:15 +CPU= 20.035
  12:40:18 STEP 1794 H= 448:30 +CPU= 20.058
  12:40:53 STEP 1795 H= 448:45 +CPU= 20.326
  12:41:27 STEP 1796 H= 449:00 +CPU= 20.621
  12:42:03 STEP 1797 H= 449:15 +CPU= 20.360
  12:42:37 STEP 1798 H= 449:30 +CPU= 20.066
  12:43:12 STEP 1799 H= 449:45 +CPU= 20.350
  12:43:54 STEP 1800 H= 450:00 +CPU= 25.034
Moving to projects directory: /home/amar/slots/0/ICMGGh7zg+001800
Moving to projects directory: /home/amar/slots/0/ICMSHh7zg+001800
Moving to projects directory: /home/amar/slots/0/ICMUAh7zg+001800
  12:44:28 STEP 1801 H= 450:15 +CPU= 20.231
  12:45:03 STEP 1802 H= 450:30 +CPU= 20.217
  12:45:37 STEP 1803 H= 450:45 +CPU= 20.041
  12:46:13 STEP 1804 H= 451:00 +CPU= 20.573
  12:46:47 STEP 1805 H= 451:15 +CPU= 20.039
  12:47:19 STEP 1806 H= 451:30 +CPU= 20.095
  12:47:54 STEP 1807 H= 451:45 +CPU= 20.075
  12:48:29 STEP 1808 H= 452:00 +CPU= 20.572
  12:49:03 STEP 1809 H= 452:15 +CPU= 20.242
  12:49:37 STEP 1810 H= 452:30 +CPU= 19.987
  12:50:11 STEP 1811 H= 452:45 +CPU= 20.136
  12:50:52 STEP 1812 H= 453:00 +CPU= 25.074
Moving to projects directory: /home/amar/slots/0/ICMGGh7zg+001812
Moving to projects directory: /home/amar/slots/0/ICMSHh7zg+001812
Moving to projects directory: /home/amar/slots/0/ICMUAh7zg+001812
  12:51:25 STEP 1813 H= 453:15 +CPU= 20.065
  12:52:01 STEP 1814 H= 453:30 +CPU= 20.205
  12:52:35 STEP 1815 H= 453:45 +CPU= 20.173
  12:53:10 STEP 1816 H= 454:00 +CPU= 20.568
  12:53:43 STEP 1817 H= 454:15 +CPU= 20.057
  12:54:17 STEP 1818 H= 454:30 +CPU= 20.218
  12:54:50 STEP 1819 H= 454:45 +CPU= 20.040
  12:55:25 STEP 1820 H= 455:00 +CPU= 20.538
  12:55:59 STEP 1821 H= 455:15 +CPU= 20.212
  12:56:33 STEP 1822 H= 455:30 +CPU= 20.182
  12:57:06 STEP 1823 H= 455:45 +CPU= 20.122
  12:57:54 STEP 1824 H= 456:00 +CPU= 27.754
Moving to projects directory: /home/amar/slots/0/ICMGGh7zg+001824
Moving to projects directory: /home/amar/slots/0/ICMSHh7zg+001824
Moving to projects directory: /home/amar/slots/0/ICMUAh7zg+001824
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMGGh7zg+001728
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMSHh7zg+001728
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMUAh7zg+001728
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMGGh7zg+001740
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMSHh7zg+001740
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMUAh7zg+001740
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMGGh7zg+001752
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMSHh7zg+001752
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMUAh7zg+001752
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMGGh7zg+001764
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMSHh7zg+001764
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMUAh7zg+001764
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMGGh7zg+001776
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMSHh7zg+001776
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMUAh7zg+001776
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMGGh7zg+001788
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMSHh7zg+001788
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMUAh7zg+001788
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMGGh7zg+001800
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMSHh7zg+001800
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMUAh7zg+001800
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMGGh7zg+001812
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMSHh7zg+001812
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMUAh7zg+001812
Zipping up the intermediate file: /home/amar/projects/climateprediction.net/oifs_43r3_bl_a4it_2016092300_20_1018_12294188_0_r48016068_18.zip
Uploading the intermediate file: upload_file_18.zip
  12:58:34 STEP 1825 H= 456:15 +CPU= 21.007
  12:59:09 STEP 1826 H= 456:30 +CPU= 20.074
  12:59:42 STEP 1827 H= 456:45 +CPU= 20.054
  13:00:16 STEP 1828 H= 457:00 +CPU= 20.363
  13:00:50 STEP 1829 H= 457:15 +CPU= 20.070
  13:01:24 STEP 1830 H= 457:30 +CPU= 20.092
  13:01:57 STEP 1831 H= 457:45 +CPU= 19.992
  13:02:32 STEP 1832 H= 458:00 +CPU= 20.470
  13:03:05 STEP 1833 H= 458:15 +CPU= 20.047
  13:03:38 STEP 1834 H= 458:30 +CPU= 19.927
  13:04:12 STEP 1835 H= 458:45 +CPU= 20.099
  13:04:53 STEP 1836 H= 459:00 +CPU= 25.088
Moving to projects directory: /home/amar/slots/0/ICMGGh7zg+001836
Moving to projects directory: /home/amar/slots/0/ICMSHh7zg+001836
Moving to projects directory: /home/amar/slots/0/ICMUAh7zg+001836
  13:05:28 STEP 1837 H= 459:15 +CPU= 20.105
  13:06:03 STEP 1838 H= 459:30 +CPU= 20.340
  13:06:36 STEP 1839 H= 459:45 +CPU= 19.905
  13:07:10 STEP 1840 H= 460:00 +CPU= 20.599
  13:07:44 STEP 1841 H= 460:15 +CPU= 20.067
  13:08:17 STEP 1842 H= 460:30 +CPU= 20.109
  13:08:51 STEP 1843 H= 460:45 +CPU= 19.950
  13:09:26 STEP 1844 H= 461:00 +CPU= 20.552
  13:09:59 STEP 1845 H= 461:15 +CPU= 20.016
  13:10:33 STEP 1846 H= 461:30 +CPU= 20.104
  13:11:08 STEP 1847 H= 461:45 +CPU= 20.172
  13:11:49 STEP 1848 H= 462:00 +CPU= 25.021
Moving to projects directory: /home/amar/slots/0/ICMGGh7zg+001848
Moving to projects directory: /home/amar/slots/0/ICMSHh7zg+001848
Moving to projects directory: /home/amar/slots/0/ICMUAh7zg+001848
  13:12:22 STEP 1849 H= 462:15 +CPU= 20.116
  13:12:55 STEP 1850 H= 462:30 +CPU= 19.990
  13:13:28 STEP 1851 H= 462:45 +CPU= 20.144
  13:14:03 STEP 1852 H= 463:00 +CPU= 20.581
  13:14:36 STEP 1853 H= 463:15 +CPU= 19.969
  13:15:10 STEP 1854 H= 463:30 +CPU= 20.123
  13:15:45 STEP 1855 H= 463:45 +CPU= 20.140
  13:16:20 STEP 1856 H= 464:00 +CPU= 20.730
  13:16:54 STEP 1857 H= 464:15 +CPU= 20.003
  13:17:29 STEP 1858 H= 464:30 +CPU= 20.283
  13:18:02 STEP 1859 H= 464:45 +CPU= 20.062
  13:18:47 STEP 1860 H= 465:00 +CPU= 25.147
Moving to projects directory: /home/amar/slots/0/ICMGGh7zg+001860
Moving to projects directory: /home/amar/slots/0/ICMSHh7zg+001860
Moving to projects directory: /home/amar/slots/0/ICMUAh7zg+001860
  13:19:21 STEP 1861 H= 465:15 +CPU= 20.177
  13:19:54 STEP 1862 H= 465:30 +CPU= 19.956
  13:20:29 STEP 1863 H= 465:45 +CPU= 20.146
  13:21:04 STEP 1864 H= 466:00 +CPU= 20.473
  13:21:38 STEP 1865 H= 466:15 +CPU= 19.946
  13:22:12 STEP 1866 H= 466:30 +CPU= 20.183
  13:22:45 STEP 1867 H= 466:45 +CPU= 19.967
  13:23:20 STEP 1868 H= 467:00 +CPU= 20.474
  13:23:54 STEP 1869 H= 467:15 +CPU= 20.058
  13:24:28 STEP 1870 H= 467:30 +CPU= 20.206
  13:25:01 STEP 1871 H= 467:45 +CPU= 20.115
  13:25:49 STEP 1872 H= 468:00 +CPU= 27.970
Moving to projects directory: /home/amar/slots/0/ICMGGh7zg+001872
Moving to projects directory: /home/amar/slots/0/ICMSHh7zg+001872
Moving to projects directory: /home/amar/slots/0/ICMUAh7zg+001872
  13:26:24 STEP 1873 H= 468:15 +CPU= 20.202
  13:26:57 STEP 1874 H= 468:30 +CPU= 19.968
  13:27:31 STEP 1875 H= 468:45 +CPU= 20.112
  13:28:06 STEP 1876 H= 469:00 +CPU= 20.492
  13:28:40 STEP 1877 H= 469:15 +CPU= 20.044
  13:29:12 STEP 1878 H= 469:30 +CPU= 20.041
  13:29:47 STEP 1879 H= 469:45 +CPU= 19.979
  13:30:21 STEP 1880 H= 470:00 +CPU= 20.494
  13:30:55 STEP 1881 H= 470:15 +CPU= 19.965
  13:31:30 STEP 1882 H= 470:30 +CPU= 20.464
  13:32:05 STEP 1883 H= 470:45 +CPU= 20.472
  13:32:53 STEP 1884 H= 471:00 +CPU= 26.023
Moving to projects directory: /home/amar/slots/0/ICMGGh7zg+001884
Moving to projects directory: /home/amar/slots/0/ICMSHh7zg+001884
Moving to projects directory: /home/amar/slots/0/ICMUAh7zg+001884
  13:33:31 STEP 1885 H= 471:15 +CPU= 21.082
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_bl_1.13_x86_64-pc-linux-gnu
(argv1) start_date: 2016092300
(argv2) exptid: h7zg
(argv3) unique_member_id: a4it
(argv4) batchid: 1018
(argv5) wuid: 12294188
(argv6) fclen: 20
(argv7) app_name: oifs_43r3_bl
(argv8) nthreads: 1
Working directory is: /home/amar/slots/0
Project directory is: /home/amar/projects/climateprediction.net/
app name: oifs_43r3_bl
version: 1.13
Location of temp folder: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188
..mkdir for temp folder for results failed
Copying: /home/amar/projects/climateprediction.net/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip to: /home/amar/slots/0/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /home/amar/slots/0/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_684e23235da31c81198f21791492403d to: /home/amar/slots/0/oifs_43r3_bl_a4it_2016092300_20_1018_12294188.zip
Unzipping the namelist zip file: /home/amar/slots/0/oifs_43r3_bl_a4it_2016092300_20_1018_12294188.zip
ic_ancil_file: ic_ancil_12294188
ifsdata_file: ifsdata_12294188
climate_data_file: clim_data_12294188
horiz_resolution: 159
vert_resolution: 91
grid_type: l_2
upload_interval: 96
utstep: 900
nfrres: restart dump frequency (steps) 48
Copying IC ancils from: ../../projects/climateprediction.net/jf_f7ecfd27aa1e68bbe7e8d19800e25c73 to: /home/amar/slots/0/ic_ancil_12294188.zip
Unzipping the IC ancils zip file: /home/amar/slots/0/ic_ancil_12294188.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_14fb2bf2b3b7d65d1fa89aef6dfd0a7e to: /home/amar/slots/0/ifsdata/ifsdata_12294188.zip
Unzipping the ifsdata_zip file: /home/amar/slots/0/ifsdata/ifsdata_12294188.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /home/amar/slots/0/159l_2/clim_data_12294188.zip
Unzipping the climate data zip file: /home/amar/slots/0/159l_2/clim_data_12294188.zip
Checking for progress XML file: /home/amar/slots/0/progress_file_12294188.xml
Opened progress file ok : /home/amar/slots/0/progress_file_12294188.xml
-- Model is restarting --
Adjusting last_iter, 1886, to previous model restart step.
Creating progress file: /home/amar/slots/0/progress_file_12294188.xml
last_cpu_time: 39921
upload_file_number: 19
last_iter: 1873
last_upload: 1641600
model_completed: 0
total_length_of_simulation: 1728000
result_base_name: oifs_43r3_bl_a4it_2016092300_20_1018_12294188_0_r48016068
The child process has been launched with process id: 94224
Executing the command: /home/amar/slots/0/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:amar-HP-ENVY:1:1:94224:94224] [20240618:052540:1718706340:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2091] fp = 0x314f440
[EC_DRHOOK:amar-HP-ENVY:1:1:94224:94224] [20240618:052540:1718706340:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:amar-HP-ENVY:1:1:94224:94224] [20240618:052540:1718706340:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:amar-HP-ENVY:1:1:94224:94224] [20240618:052540:1718706340:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:amar-HP-ENVY:1:1:94224:94224] [20240618:052540:1718706340:0.001] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:amar-HP-ENVY:1:1:94224:94224] [20240618:052540:1718706340:0.001] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:amar-HP-ENVY:1:1:94224:94224] [20240618:052540:1718706340:0.001] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:amar-HP-ENVY:1:1:94224:94224] [20240618:052540:1718706340:0.001] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:amar-HP-ENVY:1:1:94224:94224] [20240618:052540:1718706340:0.001] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:amar-HP-ENVY:1:1:94224:94224] [20240618:052540:1718706340:0.001] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:amar-HP-ENVY:1:1:94224:94224] [20240618:052540:1718706340:0.001] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:amar-HP-ENVY:1:1:94224:94224] [20240618:052540:1718706340:0.001] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:amar-HP-ENVY:1:1:94224:94224] [20240618:052540:1718706340:0.001] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:amar-HP-ENVY:1:1:94224:94224] [20240618:052540:1718706340:0.001] [ignore_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:amar-HP-ENVY:1:1:94224:94224] [20240618:052540:1718706340:0.001] [restore_default_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:amar-HP-ENVY:1:1:94224:94224] [20240618:052540:1718706340:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:amar-HP-ENVY:1:1:94224:94224] [20240618:052540:1718706340:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:amar-HP-ENVY:1:1:94224:94224] [20240618:052540:1718706340:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:amar-HP-ENVY:1:1:94224:94224] [20240618:052540:1718706340:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x820cf0 (old at (nil))
[EC_DRHOOK:amar-HP-ENVY:1:1:94224:94224] [20240618:052540:1718706340:0.001] [trapfpe_treatment@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:amar-HP-ENVY:1:1:94224:94224] [20240618:052540:1718706340:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:amar-HP-ENVY:1:1:94224:94224] [20240618:052540:1718706340:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:amar-HP-ENVY:1:1:94224:94224] [20240618:052540:1718706340:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x820cf0 (old at (nil))
[EC_DRHOOK:amar-HP-ENVY:1:1:94224:94224] [20240618:052540:1718706340:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:amar-HP-ENVY:1:1:94224:94224] [20240618:052540:1718706340:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:amar-HP-ENVY:1:1:94224:94224] [20240618:052540:1718706340:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:amar-HP-ENVY:1:1:94224:94224] [20240618:052540:1718706340:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x820cf0 (old at (nil))
[EC_DRHOOK:amar-HP-ENVY:1:1:94224:94224] [20240618:052540:1718706340:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x820cf0 (old at (nil))
[EC_DRHOOK:amar-HP-ENVY:1:1:94224:94224] [20240618:052540:1718706340:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:amar-HP-ENVY:1:1:94224:94224] [20240618:052540:1718706340:0.001] [catch_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:27:28 STEP 1872 H= 468:00 +CPU= 29.292
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:30:17 STEP 1873 H= 468:15 +CPU= 28.586
  05:31:29 STEP 1874 H= 468:30 +CPU= 29.051
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:34:12 STEP 1875 H= 468:45 +CPU= 29.575
  05:35:24 STEP 1876 H= 469:00 +CPU= 29.377
  05:36:34 STEP 1877 H= 469:15 +CPU= 29.681
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:38:05 STEP 1878 H= 469:30 +CPU= 28.535
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:40:23 STEP 1879 H= 469:45 +CPU= 30.510
  05:41:25 STEP 1880 H= 470:00 +CPU= 28.783
  05:42:36 STEP 1881 H= 470:15 +CPU= 29.825
  05:43:36 STEP 1882 H= 470:30 +CPU= 28.990
  05:44:49 STEP 1883 H= 470:45 +CPU= 29.020
  05:46:17 STEP 1884 H= 471:00 +CPU= 36.874
Moving to projects directory: /home/amar/slots/0/ICMGGh7zg+001884
Moving to projects directory: /home/amar/slots/0/ICMSHh7zg+001884
Moving to projects directory: /home/amar/slots/0/ICMUAh7zg+001884
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:48:05 STEP 1885 H= 471:15 +CPU= 29.283
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:50:03 STEP 1886 H= 471:30 +CPU= 28.311
  05:51:12 STEP 1887 H= 471:45 +CPU= 28.196
  05:52:24 STEP 1888 H= 472:00 +CPU= 29.581
  05:53:33 STEP 1889 H= 472:15 +CPU= 29.581
  05:54:38 STEP 1890 H= 472:30 +CPU= 28.313
  05:55:43 STEP 1891 H= 472:45 +CPU= 28.267
  05:56:57 STEP 1892 H= 473:00 +CPU= 29.033
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:58:31 STEP 1893 H= 473:15 +CPU= 29.549
  05:59:43 STEP 1894 H= 473:30 +CPU= 28.714
  06:00:48 STEP 1895 H= 473:45 +CPU= 28.711
  06:02:13 STEP 1896 H= 474:00 +CPU= 37.035
Moving to projects directory: /home/amar/slots/0/ICMGGh7zg+001896
Moving to projects directory: /home/amar/slots/0/ICMSHh7zg+001896
Moving to projects directory: /home/amar/slots/0/ICMUAh7zg+001896
  06:03:24 STEP 1897 H= 474:15 +CPU= 30.037
  06:04:32 STEP 1898 H= 474:30 +CPU= 30.651
  06:05:31 STEP 1899 H= 474:45 +CPU= 27.775
  06:06:38 STEP 1900 H= 475:00 +CPU= 28.822
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:09:52 STEP 1901 H= 475:15 +CPU= 30.681
  06:11:01 STEP 1902 H= 475:30 +CPU= 29.918
  06:11:59 STEP 1903 H= 475:45 +CPU= 27.840
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:14:51 STEP 1904 H= 476:00 +CPU= 29.136
  06:16:00 STEP 1905 H= 476:15 +CPU= 28.472
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:17:20 STEP 1906 H= 476:30 +CPU= 28.861
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:20:04 STEP 1907 H= 476:45 +CPU= 29.197
  06:21:28 STEP 1908 H= 477:00 +CPU= 36.198
Moving to projects directory: /home/amar/slots/0/ICMGGh7zg+001908
Moving to projects directory: /home/amar/slots/0/ICMSHh7zg+001908
Moving to projects directory: /home/amar/slots/0/ICMUAh7zg+001908
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:24:14 STEP 1909 H= 477:15 +CPU= 30.258
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:25:35 STEP 1910 H= 477:30 +CPU= 28.539
  06:26:48 STEP 1911 H= 477:45 +CPU= 30.003
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:29:35 STEP 1912 H= 478:00 +CPU= 29.307
  06:30:44 STEP 1913 H= 478:15 +CPU= 28.759
  06:31:56 STEP 1914 H= 478:30 +CPU= 29.171
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:33:54 STEP 1915 H= 478:45 +CPU= 29.376
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:35:45 STEP 1916 H= 479:00 +CPU= 28.562
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:38:17 STEP 1917 H= 479:15 +CPU= 31.679
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:39:45 STEP 1918 H= 479:30 +CPU= 28.169
  06:40:55 STEP 1919 H= 479:45 +CPU= 28.776
  06:42:20 STEP 1920 H= 480:00 +CPU= 37.389
..The child process terminated with status: 0
>>> Printing last 70 lines from file: NODE.001_01

 
DDH------------------------------------------------------------------------
I                                                                         I
I TIME =  480.000H DDH * Diagnostic Var.,     AREA                GLOBAL  I
I                                                                         I
I-------------------------------------------------------------------------I
I   VARIABLE   I      Initial value         I        Current value        I
I-------------------------------------------------------------------------I
I  OMEGA (P/S) I      0.2820318887E-21      I       0.5556376202E-07      I
ICloud fractionI      0.0000000000E+00      I       0.2024427794E-02      I
I  Relat. Hum. I      0.3929719276E+00      I       0.4033999369E+00      I
I  PASS 01     I      0.0000000000E+00      I       0.3787295608E+02      I
DDH------------------------------------------------------------------------
  
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
 NSTEP =  1920 SCAN2M_HPOS  P
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ICMGGh7zg+001920 MODE=a
 IO-STREAM CLOSED - ICMGGh7zg+001920
 MPI-TASK:   1 -    304372724 BYTES IN      7 RECORDS TRANSFERRED IN    0.0008 SECONDS380465.9 Mbytes/s, TOTAL TIME=    0.0427( 1.9%)
 06:42:20 STEP 1920 H= 480:00 +CPU= 37.389
 END CNT3
 NSTEP =  1920 CNT0   000000000  27.549  27.549  65.093   0.423               0               0               0               0               0
IO-STREAM STATISTICS, TOTAL NO OF REC/BYTES READ           46/  1069125708 WRITTEN         1430/************
 -TASK-OPENED-OPEN-RECS IN -KBYTE IN    -RECS OUT-KBYTE OUT-WALL   -WALL IN-WALL
  OU-TOT IN -TOT OUT
   1     28    0       46      1044068     1430 ************    30.7    23.8     6.9    25.9    13.8
===-=== START OF TIMING STATISTICS ===-===
 
STATS FOR ALL TASKS
 NUM ROUTINE                                     CALLS  MEAN(ms)   MAX(ms)   FRAC(%)  UNBAL(%)
   0 CNT0     - COMPLETE EXECUTION                   1 4599593.7 4599593.7    100.00      0.00
   1 CNT4     - FORWARD INTEGRATION                  1 4569171.2 4569171.2     99.34      0.00
   8 SCAN2M - GRID-POINT DYNAMICS                   53   17217.2   17217.2     19.84      0.00
   9 SPCM     - SPECTRAL COMP.                      48    2790.2    2790.2      2.91      0.00
  10 SCAN2M - PHYSICS                               49   38003.7   38003.7     40.49      0.00
  11 IOPACK   - OUTPUT P.P. RESULTS                  4    2466.1    2466.1      0.21      0.00
  12 SPNORM   - SPECTRAL NORM COMP.                  3      78.4      78.4      0.01      0.00
  17 GRIDFPOS IN CNT4                                4      73.9      73.9      0.01      0.00
  27 RESTART FILES - WRITING                         1    9864.2    9864.2      0.21      0.00
  28 RESTART FILES - READING                         1   28203.5   28203.5      0.61      0.00
  29 SU4FPOS IN CNT4                                 4       0.1       0.1      0.00      0.00
  30 DYNFPOS IN CNT4                                 4   13476.3   13476.3      1.17      0.00
  31 POSDDH IN STEPO                                 1       1.0       1.0      0.00      0.00
  37 CPGLAG   - SL COMPUTATIONS                     49   23737.0   23737.0     25.29      0.00
  39 SU0YOMB                                         1    1280.9    1280.9      0.03      0.00
  51 SCAN2M   - SL COMM. PART 1                     49      68.4      68.4      0.07      0.00
  54 SPCM     - M TO S/S TO M TRANSP.               48     936.6     936.6      0.98      0.00
  55 SPCIMPF  - S TO M/M TO S TRANSP.               48     302.5     302.5      0.32      0.00
  56 SPNORM   - SPECTRAL NORM COMM.                  3       0.4       0.4      0.00      0.00
 102 LTINV_CTL   - INVERSE LEGENDRE TRANSFORM      108    1418.2    1418.2      3.33      0.00
 103 LTDIR_CTL   - DIRECT LEGENDRE TRANSFORM       100     667.2     667.2      1.45      0.00
 106 FTDIR_CTL   - DIRECT FOURIER TRANSFORM        100     241.0     241.0      0.52      0.00
 107 FTINV_CTL   - INVERSE FOURIER TRANSFORM       108     600.3     600.3      1.41      0.00
 140 SULEG       - COMP. OF LEGENDRE POL.            1     203.3     203.3      0.00      0.00
 152 LTINV_CTL   - M TO L TRANSPOSITION            108     355.8     355.8      0.84      0.00
 153 LTDIR_CTL   - L TO M TRANSPOSITION            100      75.5      75.5      0.16      0.00
 157 FTINV_CTL   - L TO G TRANSPOSITION            108     123.3     123.3      0.29      0.00
 158 FTDIR_CTL   - G TO L TRANSPOSITION            100      58.4      58.4      0.13      0.00
 400 GSTATS                                       6667       0.0       0.0      0.00      0.00
 401 GSTATS HOOK                                  6149       0.0       0.0      0.00      0.00
TOTAL MEASURED IMBALANCE =       0.0 SECONDS,  0.0 PERCENT
TOTAL WALLCLOCK TIME     4599.6 CPU TIME    1464.7 VECTOR TIME     1464.7


===-=== END   OF TIMING STATISTICS ===-===
FORECAST DAYS PER DAY      378.2
  *** END CNT0 *** 
------------------------------------------------
Moving to projects directory: /home/amar/slots/0/ICMGGh7zg+001920
Moving to projects directory: /home/amar/slots/0/ICMSHh7zg+001920
Moving to projects directory: /home/amar/slots/0/ICMUAh7zg+001920
Adding to the zip: /home/amar/slots/0/NODE.001_01
Adding to the zip: /home/amar/slots/0/ifs.stat
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMGGh7zg+001872
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMUAh7zg+001872
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMSHh7zg+001860
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMSHh7zg+001872
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMGGh7zg+001908
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMUAh7zg+001860
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMSHh7zg+001884
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMSHh7zg+001836
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMSHh7zg+001896
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMGGh7zg+001848
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMSHh7zg+001824
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMUAh7zg+001824
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMUAh7zg+001908
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMUAh7zg+001848
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMGGh7zg+001920
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMGGh7zg+001824
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMSHh7zg+001848
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMGGh7zg+001884
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMUAh7zg+001884
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMSHh7zg+001920
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMGGh7zg+001836
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMGGh7zg+001896
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMUAh7zg+001896
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMGGh7zg+001860
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMSHh7zg+001908
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMUAh7zg+001920
Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMUAh7zg+001836
Zipping up the final file: /home/amar/projects/climateprediction.net/oifs_43r3_bl_a4it_2016092300_20_1018_12294188_0_r48016068_19.zip
Uploading the final file: upload_file_19.zip
Uploading trickle at timestep: 1727100
06:46:20 (94208): called boinc_finish(0)

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
18 Jun 2024 14:10:38 1542649 22447805 oifs_43r3_bl_a4it_2016092300_20_1018_12294188_0 1,727,100 41,385 0.0240
15 Jun 2024 18:36:15 1542649 22447805 oifs_43r3_bl_a4it_2016092300_20_1018_12294188_0 864,000 20,870 0.0242


©2024 cpdn.org