climateprediction.net (CPDN) home page
Task 22444059

Task 22444059

Name oifs_43r3_bl_a1nf_2016092300_20_1018_12290466_0
Workunit 12290466
Created 12 Jun 2024, 23:03:23 UTC
Sent 13 Jun 2024, 20:08:00 UTC
Report deadline 12 Aug 2024, 20:08:00 UTC
Received 15 Jun 2024, 22:18:12 UTC
Server state Over
Outcome Success
Client state Done
Exit status 0 (0x00000000)
Computer ID 1532599
Run time 15 hours 57 min 40 sec
CPU time 15 hours 57 min 40 sec
Validate state Valid
Credit 2,635.55
Device peak FLOPS 3.11 GFLOPS
Application version OpenIFS 43r3 Baroclinic Lifecycle v1.13
x86_64-pc-linux-gnu
Peak working set size 4.91 GB
Peak swap size 5.84 GB
Peak disk usage 1.32 GB
Stderr
<core_client_version>8.1.0</core_client_version>
<![CDATA[
<stderr_txt>
5 H= 383:45 +CPU= 29.051
  13:28:22 STEP 1536 H= 384:00 +CPU= 40.033
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGh7zg+001536
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHh7zg+001536
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAh7zg+001536
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMGGh7zg+001440
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMSHh7zg+001440
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMUAh7zg+001440
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMGGh7zg+001452
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMSHh7zg+001452
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMUAh7zg+001452
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMGGh7zg+001464
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMSHh7zg+001464
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMUAh7zg+001464
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMGGh7zg+001476
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMSHh7zg+001476
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMUAh7zg+001476
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMGGh7zg+001488
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMSHh7zg+001488
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMUAh7zg+001488
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMGGh7zg+001500
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMSHh7zg+001500
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMUAh7zg+001500
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMGGh7zg+001512
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMSHh7zg+001512
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMUAh7zg+001512
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMGGh7zg+001524
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMSHh7zg+001524
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMUAh7zg+001524
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a1nf_2016092300_20_1018_12290466_0_r1605132715_15.zip
Uploading the intermediate file: upload_file_15.zip
  13:28:50 STEP 1537 H= 384:15 +CPU= 28.108
  13:29:19 STEP 1538 H= 384:30 +CPU= 28.963
  13:29:47 STEP 1539 H= 384:45 +CPU= 27.765
  13:30:16 STEP 1540 H= 385:00 +CPU= 29.066
  13:30:44 STEP 1541 H= 385:15 +CPU= 27.489
  13:31:12 STEP 1542 H= 385:30 +CPU= 28.101
  13:31:42 STEP 1543 H= 385:45 +CPU= 29.767
  13:32:11 STEP 1544 H= 386:00 +CPU= 29.038
  13:32:39 STEP 1545 H= 386:15 +CPU= 28.065
  13:33:08 STEP 1546 H= 386:30 +CPU= 28.946
  13:33:38 STEP 1547 H= 386:45 +CPU= 30.104
  13:34:12 STEP 1548 H= 387:00 +CPU= 34.024
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGh7zg+001548
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHh7zg+001548
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAh7zg+001548
  13:34:39 STEP 1549 H= 387:15 +CPU= 26.740
  13:35:07 STEP 1550 H= 387:30 +CPU= 27.956
  13:35:36 STEP 1551 H= 387:45 +CPU= 28.666
  13:36:05 STEP 1552 H= 388:00 +CPU= 29.189
  13:36:35 STEP 1553 H= 388:15 +CPU= 29.168
  13:37:04 STEP 1554 H= 388:30 +CPU= 29.017
  13:37:32 STEP 1555 H= 388:45 +CPU= 28.228
  13:38:01 STEP 1556 H= 389:00 +CPU= 28.771
  13:38:29 STEP 1557 H= 389:15 +CPU= 28.114
  13:38:58 STEP 1558 H= 389:30 +CPU= 28.442
  13:39:27 STEP 1559 H= 389:45 +CPU= 29.806
  13:40:04 STEP 1560 H= 390:00 +CPU= 36.593
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGh7zg+001560
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHh7zg+001560
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAh7zg+001560
  13:40:35 STEP 1561 H= 390:15 +CPU= 30.674
  13:41:05 STEP 1562 H= 390:30 +CPU= 30.107
  13:41:34 STEP 1563 H= 390:45 +CPU= 28.726
  13:42:04 STEP 1564 H= 391:00 +CPU= 29.949
  13:42:32 STEP 1565 H= 391:15 +CPU= 27.715
  13:43:01 STEP 1566 H= 391:30 +CPU= 29.420
  13:43:31 STEP 1567 H= 391:45 +CPU= 29.679
  13:44:00 STEP 1568 H= 392:00 +CPU= 28.904
  13:44:28 STEP 1569 H= 392:15 +CPU= 28.437
  13:44:57 STEP 1570 H= 392:30 +CPU= 28.920
  13:45:28 STEP 1571 H= 392:45 +CPU= 30.419
  13:46:03 STEP 1572 H= 393:00 +CPU= 35.414
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGh7zg+001572
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHh7zg+001572
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAh7zg+001572
  13:46:32 STEP 1573 H= 393:15 +CPU= 28.418
  13:47:00 STEP 1574 H= 393:30 +CPU= 28.087
  13:47:29 STEP 1575 H= 393:45 +CPU= 29.107
  13:47:59 STEP 1576 H= 394:00 +CPU= 29.507
  13:48:27 STEP 1577 H= 394:15 +CPU= 28.383
  13:48:56 STEP 1578 H= 394:30 +CPU= 29.012
  13:49:25 STEP 1579 H= 394:45 +CPU= 28.417
  13:49:54 STEP 1580 H= 395:00 +CPU= 29.466
  13:50:23 STEP 1581 H= 395:15 +CPU= 29.021
  13:50:52 STEP 1582 H= 395:30 +CPU= 28.800
  13:51:21 STEP 1583 H= 395:45 +CPU= 28.734
  13:52:00 STEP 1584 H= 396:00 +CPU= 39.247
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGh7zg+001584
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHh7zg+001584
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAh7zg+001584
  13:52:29 STEP 1585 H= 396:15 +CPU= 28.243
  13:52:57 STEP 1586 H= 396:30 +CPU= 28.526
  13:53:26 STEP 1587 H= 396:45 +CPU= 28.306
  13:53:54 STEP 1588 H= 397:00 +CPU= 28.855
  13:54:24 STEP 1589 H= 397:15 +CPU= 29.474
  13:54:55 STEP 1590 H= 397:30 +CPU= 30.482
  13:55:22 STEP 1591 H= 397:45 +CPU= 27.787
  13:55:53 STEP 1592 H= 398:00 +CPU= 30.155
  13:56:21 STEP 1593 H= 398:15 +CPU= 27.883
  13:56:48 STEP 1594 H= 398:30 +CPU= 27.698
  13:57:19 STEP 1595 H= 398:45 +CPU= 30.456
  13:57:54 STEP 1596 H= 399:00 +CPU= 34.804
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGh7zg+001596
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHh7zg+001596
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAh7zg+001596
  13:58:23 STEP 1597 H= 399:15 +CPU= 28.731
  13:58:51 STEP 1598 H= 399:30 +CPU= 28.518
  13:59:22 STEP 1599 H= 399:45 +CPU= 30.230
  13:59:52 STEP 1600 H= 400:00 +CPU= 30.231
  14:00:20 STEP 1601 H= 400:15 +CPU= 28.463
  14:00:50 STEP 1602 H= 400:30 +CPU= 29.825
  14:01:19 STEP 1603 H= 400:45 +CPU= 28.301
  14:01:48 STEP 1604 H= 401:00 +CPU= 29.523
  14:02:16 STEP 1605 H= 401:15 +CPU= 27.277
  14:02:44 STEP 1606 H= 401:30 +CPU= 28.107
  14:03:12 STEP 1607 H= 401:45 +CPU= 28.242
  14:03:48 STEP 1608 H= 402:00 +CPU= 35.978
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGh7zg+001608
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHh7zg+001608
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAh7zg+001608
  14:04:18 STEP 1609 H= 402:15 +CPU= 29.622
  14:04:46 STEP 1610 H= 402:30 +CPU= 27.937
  14:05:15 STEP 1611 H= 402:45 +CPU= 29.251
  14:05:44 STEP 1612 H= 403:00 +CPU= 28.268
  14:06:12 STEP 1613 H= 403:15 +CPU= 28.061
  14:06:40 STEP 1614 H= 403:30 +CPU= 27.865
  14:07:09 STEP 1615 H= 403:45 +CPU= 29.055
  14:07:40 STEP 1616 H= 404:00 +CPU= 30.797
  14:08:09 STEP 1617 H= 404:15 +CPU= 29.409
  14:08:38 STEP 1618 H= 404:30 +CPU= 28.618
  14:09:06 STEP 1619 H= 404:45 +CPU= 28.369
  14:09:43 STEP 1620 H= 405:00 +CPU= 36.473
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGh7zg+001620
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHh7zg+001620
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAh7zg+001620
  14:10:12 STEP 1621 H= 405:15 +CPU= 29.210
  14:10:41 STEP 1622 H= 405:30 +CPU= 29.015
  14:11:13 STEP 1623 H= 405:45 +CPU= 31.347
  14:11:42 STEP 1624 H= 406:00 +CPU= 28.846
  14:12:12 STEP 1625 H= 406:15 +CPU= 29.846
  14:12:40 STEP 1626 H= 406:30 +CPU= 28.746
  14:13:09 STEP 1627 H= 406:45 +CPU= 28.456
  14:13:39 STEP 1628 H= 407:00 +CPU= 29.511
  14:14:07 STEP 1629 H= 407:15 +CPU= 28.278
  14:14:37 STEP 1630 H= 407:30 +CPU= 29.875
  14:15:05 STEP 1631 H= 407:45 +CPU= 27.912
  14:15:45 STEP 1632 H= 408:00 +CPU= 39.862
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGh7zg+001632
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHh7zg+001632
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAh7zg+001632
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMGGh7zg+001536
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMSHh7zg+001536
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMUAh7zg+001536
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMGGh7zg+001548
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMSHh7zg+001548
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMUAh7zg+001548
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMGGh7zg+001560
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMSHh7zg+001560
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMUAh7zg+001560
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMGGh7zg+001572
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMSHh7zg+001572
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMUAh7zg+001572
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMGGh7zg+001584
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMSHh7zg+001584
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMUAh7zg+001584
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMGGh7zg+001596
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMSHh7zg+001596
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMUAh7zg+001596
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMGGh7zg+001608
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMSHh7zg+001608
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMUAh7zg+001608
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMGGh7zg+001620
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMSHh7zg+001620
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMUAh7zg+001620
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a1nf_2016092300_20_1018_12290466_0_r1605132715_16.zip
Uploading the intermediate file: upload_file_16.zip
  14:16:14 STEP 1633 H= 408:15 +CPU= 29.465
  14:16:43 STEP 1634 H= 408:30 +CPU= 28.589
  14:17:12 STEP 1635 H= 408:45 +CPU= 29.336
  14:17:42 STEP 1636 H= 409:00 +CPU= 29.162
  14:18:10 STEP 1637 H= 409:15 +CPU= 28.153
  14:18:39 STEP 1638 H= 409:30 +CPU= 29.546
  14:19:10 STEP 1639 H= 409:45 +CPU= 30.830
  14:19:40 STEP 1640 H= 410:00 +CPU= 29.135
  14:20:09 STEP 1641 H= 410:15 +CPU= 28.999
  14:20:38 STEP 1642 H= 410:30 +CPU= 29.176
  14:21:07 STEP 1643 H= 410:45 +CPU= 28.989
  14:21:42 STEP 1644 H= 411:00 +CPU= 34.763
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGh7zg+001644
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHh7zg+001644
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAh7zg+001644
  14:22:11 STEP 1645 H= 411:15 +CPU= 29.019
  14:22:40 STEP 1646 H= 411:30 +CPU= 28.996
  14:23:09 STEP 1647 H= 411:45 +CPU= 28.542
  14:23:37 STEP 1648 H= 412:00 +CPU= 28.692
  14:24:07 STEP 1649 H= 412:15 +CPU= 29.294
  14:24:37 STEP 1650 H= 412:30 +CPU= 29.938
  14:25:06 STEP 1651 H= 412:45 +CPU= 29.059
  14:25:36 STEP 1652 H= 413:00 +CPU= 29.710
  14:26:06 STEP 1653 H= 413:15 +CPU= 30.001
  14:26:36 STEP 1654 H= 413:30 +CPU= 29.718
  14:27:05 STEP 1655 H= 413:45 +CPU= 29.328
  14:27:40 STEP 1656 H= 414:00 +CPU= 35.379
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGh7zg+001656
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHh7zg+001656
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAh7zg+001656
  14:28:10 STEP 1657 H= 414:15 +CPU= 29.157
  14:28:39 STEP 1658 H= 414:30 +CPU= 29.287
  14:29:06 STEP 1659 H= 414:45 +CPU= 27.310
  14:29:35 STEP 1660 H= 415:00 +CPU= 28.270
  14:30:03 STEP 1661 H= 415:15 +CPU= 28.066
  14:30:30 STEP 1662 H= 415:30 +CPU= 27.408
  14:31:00 STEP 1663 H= 415:45 +CPU= 29.092
  14:31:29 STEP 1664 H= 416:00 +CPU= 28.970
  14:31:56 STEP 1665 H= 416:15 +CPU= 27.772
  14:32:25 STEP 1666 H= 416:30 +CPU= 28.292
  14:32:53 STEP 1667 H= 416:45 +CPU= 28.138
  14:33:28 STEP 1668 H= 417:00 +CPU= 35.040
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGh7zg+001668
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHh7zg+001668
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAh7zg+001668
  14:33:57 STEP 1669 H= 417:15 +CPU= 29.131
  14:34:27 STEP 1670 H= 417:30 +CPU= 29.263
  14:34:56 STEP 1671 H= 417:45 +CPU= 29.084
  14:35:26 STEP 1672 H= 418:00 +CPU= 30.093
  14:35:57 STEP 1673 H= 418:15 +CPU= 30.877
  14:36:27 STEP 1674 H= 418:30 +CPU= 29.589
  14:36:54 STEP 1675 H= 418:45 +CPU= 27.632
  14:37:24 STEP 1676 H= 419:00 +CPU= 28.981
  14:37:53 STEP 1677 H= 419:15 +CPU= 29.934
  14:38:23 STEP 1678 H= 419:30 +CPU= 29.257
  14:38:52 STEP 1679 H= 419:45 +CPU= 29.098
  14:39:32 STEP 1680 H= 420:00 +CPU= 40.064
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGh7zg+001680
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHh7zg+001680
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAh7zg+001680
  14:40:01 STEP 1681 H= 420:15 +CPU= 28.775
  14:40:29 STEP 1682 H= 420:30 +CPU= 28.281
  14:40:58 STEP 1683 H= 420:45 +CPU= 28.097
  14:41:28 STEP 1684 H= 421:00 +CPU= 30.265
  14:41:55 STEP 1685 H= 421:15 +CPU= 27.275
  14:42:25 STEP 1686 H= 421:30 +CPU= 29.822
  14:42:54 STEP 1687 H= 421:45 +CPU= 28.367
  14:43:24 STEP 1688 H= 422:00 +CPU= 30.392
  14:43:53 STEP 1689 H= 422:15 +CPU= 29.208
  14:44:23 STEP 1690 H= 422:30 +CPU= 30.068
  14:44:53 STEP 1691 H= 422:45 +CPU= 29.586
  14:45:29 STEP 1692 H= 423:00 +CPU= 36.163
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGh7zg+001692
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHh7zg+001692
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAh7zg+001692
  14:45:58 STEP 1693 H= 423:15 +CPU= 28.270
  14:46:26 STEP 1694 H= 423:30 +CPU= 27.907
  14:46:53 STEP 1695 H= 423:45 +CPU= 27.594
  14:47:24 STEP 1696 H= 424:00 +CPU= 30.567
  14:47:51 STEP 1697 H= 424:15 +CPU= 27.201
  14:48:20 STEP 1698 H= 424:30 +CPU= 28.779
  14:48:48 STEP 1699 H= 424:45 +CPU= 27.981
  14:49:16 STEP 1700 H= 425:00 +CPU= 27.741
  14:49:45 STEP 1701 H= 425:15 +CPU= 28.925
  14:50:13 STEP 1702 H= 425:30 +CPU= 27.808
  14:50:41 STEP 1703 H= 425:45 +CPU= 27.823
  14:51:17 STEP 1704 H= 426:00 +CPU= 35.735
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGh7zg+001704
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHh7zg+001704
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAh7zg+001704
  14:51:46 STEP 1705 H= 426:15 +CPU= 29.269
  14:52:14 STEP 1706 H= 426:30 +CPU= 28.317
  14:52:42 STEP 1707 H= 426:45 +CPU= 27.865
  14:53:12 STEP 1708 H= 427:00 +CPU= 29.395
  14:53:39 STEP 1709 H= 427:15 +CPU= 26.741
  14:54:08 STEP 1710 H= 427:30 +CPU= 29.676
  14:54:36 STEP 1711 H= 427:45 +CPU= 27.461
  14:55:05 STEP 1712 H= 428:00 +CPU= 29.136
  14:55:33 STEP 1713 H= 428:15 +CPU= 28.051
  14:56:01 STEP 1714 H= 428:30 +CPU= 28.147
  14:56:31 STEP 1715 H= 428:45 +CPU= 29.653
  14:57:06 STEP 1716 H= 429:00 +CPU= 34.824
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGh7zg+001716
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHh7zg+001716
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAh7zg+001716
  14:57:37 STEP 1717 H= 429:15 +CPU= 30.400
  14:58:06 STEP 1718 H= 429:30 +CPU= 29.593
  14:58:33 STEP 1719 H= 429:45 +CPU= 26.830
  14:59:03 STEP 1720 H= 430:00 +CPU= 29.832
  14:59:31 STEP 1721 H= 430:15 +CPU= 27.956
  15:00:01 STEP 1722 H= 430:30 +CPU= 29.369
  15:00:29 STEP 1723 H= 430:45 +CPU= 28.620
  15:00:59 STEP 1724 H= 431:00 +CPU= 29.323
  15:01:27 STEP 1725 H= 431:15 +CPU= 28.411
  15:01:56 STEP 1726 H= 431:30 +CPU= 28.627
  15:02:25 STEP 1727 H= 431:45 +CPU= 29.453
  15:03:06 STEP 1728 H= 432:00 +CPU= 40.800
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGh7zg+001728
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHh7zg+001728
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAh7zg+001728
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMGGh7zg+001632
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMSHh7zg+001632
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMUAh7zg+001632
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMGGh7zg+001644
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMSHh7zg+001644
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMUAh7zg+001644
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMGGh7zg+001656
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMSHh7zg+001656
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMUAh7zg+001656
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMGGh7zg+001668
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMSHh7zg+001668
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMUAh7zg+001668
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMGGh7zg+001680
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMSHh7zg+001680
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMUAh7zg+001680
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMGGh7zg+001692
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMSHh7zg+001692
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMUAh7zg+001692
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMGGh7zg+001704
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMSHh7zg+001704
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMUAh7zg+001704
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMGGh7zg+001716
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMSHh7zg+001716
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMUAh7zg+001716
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a1nf_2016092300_20_1018_12290466_0_r1605132715_17.zip
Uploading the intermediate file: upload_file_17.zip
  15:03:35 STEP 1729 H= 432:15 +CPU= 28.554
  15:04:03 STEP 1730 H= 432:30 +CPU= 27.764
  15:04:33 STEP 1731 H= 432:45 +CPU= 30.093
  15:05:04 STEP 1732 H= 433:00 +CPU= 30.904
  15:05:33 STEP 1733 H= 433:15 +CPU= 29.004
  15:06:02 STEP 1734 H= 433:30 +CPU= 29.346
  15:06:33 STEP 1735 H= 433:45 +CPU= 30.094
  15:07:03 STEP 1736 H= 434:00 +CPU= 29.878
  15:07:30 STEP 1737 H= 434:15 +CPU= 27.847
  15:07:58 STEP 1738 H= 434:30 +CPU= 27.553
  15:08:27 STEP 1739 H= 434:45 +CPU= 28.748
  15:09:03 STEP 1740 H= 435:00 +CPU= 36.193
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGh7zg+001740
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHh7zg+001740
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAh7zg+001740
  15:09:32 STEP 1741 H= 435:15 +CPU= 28.973
  15:10:02 STEP 1742 H= 435:30 +CPU= 30.168
  15:10:33 STEP 1743 H= 435:45 +CPU= 30.057
  15:11:02 STEP 1744 H= 436:00 +CPU= 29.453
  15:11:32 STEP 1745 H= 436:15 +CPU= 29.536
  15:12:01 STEP 1746 H= 436:30 +CPU= 29.349
  15:12:30 STEP 1747 H= 436:45 +CPU= 28.557
  15:12:59 STEP 1748 H= 437:00 +CPU= 29.545
  15:13:27 STEP 1749 H= 437:15 +CPU= 27.720
  15:13:56 STEP 1750 H= 437:30 +CPU= 29.054
  15:14:25 STEP 1751 H= 437:45 +CPU= 28.684
  15:15:00 STEP 1752 H= 438:00 +CPU= 35.274
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGh7zg+001752
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHh7zg+001752
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAh7zg+001752
  15:15:30 STEP 1753 H= 438:15 +CPU= 29.847
  15:16:00 STEP 1754 H= 438:30 +CPU= 29.192
  15:16:29 STEP 1755 H= 438:45 +CPU= 29.383
  15:16:57 STEP 1756 H= 439:00 +CPU= 28.058
  15:17:27 STEP 1757 H= 439:15 +CPU= 30.225
  15:17:57 STEP 1758 H= 439:30 +CPU= 29.157
  15:18:24 STEP 1759 H= 439:45 +CPU= 27.360
  15:18:53 STEP 1760 H= 440:00 +CPU= 29.173
  15:19:23 STEP 1761 H= 440:15 +CPU= 29.918
  15:19:51 STEP 1762 H= 440:30 +CPU= 27.099
  15:20:19 STEP 1763 H= 440:45 +CPU= 28.638
  15:20:55 STEP 1764 H= 441:00 +CPU= 35.580
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGh7zg+001764
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHh7zg+001764
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAh7zg+001764
  15:21:23 STEP 1765 H= 441:15 +CPU= 27.936
  15:21:52 STEP 1766 H= 441:30 +CPU= 29.127
  15:22:23 STEP 1767 H= 441:45 +CPU= 30.131
  15:22:53 STEP 1768 H= 442:00 +CPU= 29.635
  15:23:24 STEP 1769 H= 442:15 +CPU= 31.250
  15:23:59 STEP 1770 H= 442:30 +CPU= 33.180
  15:24:29 STEP 1771 H= 442:45 +CPU= 30.429
  15:24:59 STEP 1772 H= 443:00 +CPU= 29.504
  15:25:28 STEP 1773 H= 443:15 +CPU= 28.412
  15:25:58 STEP 1774 H= 443:30 +CPU= 30.425
  15:26:28 STEP 1775 H= 443:45 +CPU= 29.788
  15:27:12 STEP 1776 H= 444:00 +CPU= 42.475
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGh7zg+001776
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHh7zg+001776
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAh7zg+001776
  15:27:45 STEP 1777 H= 444:15 +CPU= 32.506
  15:28:14 STEP 1778 H= 444:30 +CPU= 28.951
  15:28:47 STEP 1779 H= 444:45 +CPU= 32.249
  15:29:20 STEP 1780 H= 445:00 +CPU= 32.055
  15:29:50 STEP 1781 H= 445:15 +CPU= 29.721
  15:30:20 STEP 1782 H= 445:30 +CPU= 29.924
  15:30:56 STEP 1783 H= 445:45 +CPU= 36.018
  15:31:33 STEP 1784 H= 446:00 +CPU= 35.699
  15:32:03 STEP 1785 H= 446:15 +CPU= 30.287
  15:32:34 STEP 1786 H= 446:30 +CPU= 30.510
  15:33:05 STEP 1787 H= 446:45 +CPU= 30.470
  15:33:44 STEP 1788 H= 447:00 +CPU= 38.861
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGh7zg+001788
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHh7zg+001788
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAh7zg+001788
  15:34:17 STEP 1789 H= 447:15 +CPU= 32.534
  15:34:51 STEP 1790 H= 447:30 +CPU= 33.270
  15:35:24 STEP 1791 H= 447:45 +CPU= 32.604
  15:35:56 STEP 1792 H= 448:00 +CPU= 32.070
  15:36:28 STEP 1793 H= 448:15 +CPU= 31.220
  15:37:03 STEP 1794 H= 448:30 +CPU= 33.990
  15:37:35 STEP 1795 H= 448:45 +CPU= 31.639
  15:38:07 STEP 1796 H= 449:00 +CPU= 31.683
  15:38:39 STEP 1797 H= 449:15 +CPU= 31.882
  15:39:15 STEP 1798 H= 449:30 +CPU= 35.601
  15:39:47 STEP 1799 H= 449:45 +CPU= 31.562
  15:40:24 STEP 1800 H= 450:00 +CPU= 36.980
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGh7zg+001800
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHh7zg+001800
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAh7zg+001800
  15:40:58 STEP 1801 H= 450:15 +CPU= 32.911
  15:41:31 STEP 1802 H= 450:30 +CPU= 32.420
  15:42:04 STEP 1803 H= 450:45 +CPU= 32.819
  15:42:38 STEP 1804 H= 451:00 +CPU= 33.878
  15:43:10 STEP 1805 H= 451:15 +CPU= 31.865
  15:43:40 STEP 1806 H= 451:30 +CPU= 30.564
  15:44:15 STEP 1807 H= 451:45 +CPU= 34.163
[EC_DRHOOK:SuperBobH1:1:1:729005:729005] [20240615:154433:1718480673:41546.965] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1538] Received signal#15 (SIGTERM) :: 3958MB (heap), 4796MB (maxrss), 0MB (maxstack), 0 (paging), nsigs = 1
[EC_DRHOOK:SuperBobH1:1:1:729005:729005] [20240615:154433:1718480673:41546.967] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1542] Also activating Harakiri-alarm (SIGALRM=14) to expire after 500s elapsed to prevent hangs, nsigs = 1
[EC_DRHOOK:SuperBobH1:1:1:729005:729005] [20240615:154433:1718480673:41546.967] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1544] Harakiri signal handler 'signal_harakiri' for signal#14 (SIGALRM) installed at 0x820890 (old at (nil))
[EC_DRHOOK:SuperBobH1:1:1:729005:729005] [20240615:154433:1718480673:41546.968] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1617] Signal#15 was caused by unrecognized si_code [memaddr=0x1], nsigs = 1
[EC_DRHOOK:SuperBobH1:1:1:729005:729005] [20240615:154433:1718480673:41546.968] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1686] Starting DrHook backtrace for signal#15, nsigs = 1
[EC_DRHOOK:SuperBobH1:1:1:729005:729005] [20240615:154433:1718480673:41546.968] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3843] 3958 MB (maxheap), 4796 MB (maxrss), 0 MB (maxstack)
[EC_DRHOOK:SuperBobH1:1:1:729005:729005] [20240615:154433:1718480673:41546.968] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :  MASTER 
[EC_DRHOOK:SuperBobH1:1:1:729005:729005] [20240615:154433:1718480673:41546.969] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :   CNT0 
[EC_DRHOOK:SuperBobH1:1:1:729005:729005] [20240615:154433:1718480673:41546.969] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :    CNT1 
[EC_DRHOOK:SuperBobH1:1:1:729005:729005] [20240615:154433:1718480673:41546.969] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :     CNT2 
[EC_DRHOOK:SuperBobH1:1:1:729005:729005] [20240615:154433:1718480673:41546.969] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :      CNT3 
[EC_DRHOOK:SuperBobH1:1:1:729005:729005] [20240615:154433:1718480673:41546.969] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :       CNT4 
[EC_DRHOOK:SuperBobH1:1:1:729005:729005] [20240615:154433:1718480673:41546.969] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :        STEPO 
[EC_DRHOOK:SuperBobH1:1:1:729005:729005] [20240615:154433:1718480673:41546.969] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :         SCAN2M 
[EC_DRHOOK:SuperBobH1:1:1:729005:729005] [20240615:154433:1718480673:41546.969] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :          GP_MODEL_HEAP 
[EC_DRHOOK:SuperBobH1:1:1:729005:729005] [20240615:154433:1718480673:41546.969] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :           GP_MODEL 
[EC_DRHOOK:SuperBobH1:1:1:729005:729005] [20240615:154433:1718480673:41546.969] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :            EC_PHYS_DRV 
[EC_DRHOOK:SuperBobH1:1:1:729005:729005] [20240615:154433:1718480673:41546.969] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :             >OMP-PHYSICS              (1001) 
[EC_DRHOOK:SuperBobH1:1:1:729005:729005] [20240615:154433:1718480673:41546.969] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :              EC_PHYS 
[EC_DRHOOK:SuperBobH1:1:1:729005:729005] [20240615:154433:1718480673:41546.969] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :               CALLPAR 
[EC_DRHOOK:SuperBobH1:1:1:729005:729005] [20240615:154433:1718480673:41546.969] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :                CLOUD_LAYER 
[EC_DRHOOK:SuperBobH1:1:1:729005:729005] [20240615:154433:1718480673:41546.969] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :                 CLOUDSC 
[EC_DRHOOK:SuperBobH1:1:1:729005:729005] [20240615:154434:1718480674:41547.969] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1734] DrHook backtrace done for signal#15, nsigs = 1
[EC_DRHOOK:SuperBobH1:1:1:729005:729005] [20240615:154434:1718480674:41547.969] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1785] Calling previous signal handler at 0x1cf8da0 for signal#15, nsigs = 1
forrtl: error (78): process killed (SIGTERM)
Image              PC                Routine            Line        Source             
oifs_43r3_model.e  0000000001CF935B  Unknown               Unknown  Unknown
oifs_43r3_model.e  00000000008214C1  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001DD5490  Unknown               Unknown  Unknown
oifs_43r3_model.e  000000000117FE68  cloudsc_                 3033  cloudsc.F90
oifs_43r3_model.e  000000000115C847  cloud_layer_              131  cloud_layer.F90
oifs_43r3_model.e  0000000000FBB064  callpar_                 1223  callpar.F90
oifs_43r3_model.e  0000000000F5FA6B  ec_phys_                  670  ec_phys.F90
oifs_43r3_model.e  0000000000F4DBC4  ec_phys_drv_              599  ec_phys_drv.F90
oifs_43r3_model.e  0000000000BD7131  gp_model_                 613  gp_model.F90
oifs_43r3_model.e  00000000012C5002  gp_model_heap_             74  gp_model_heap.F90
oifs_43r3_model.e  0000000000BACBAE  scan2m_                   535  scan2m.F90
oifs_43r3_model.e  0000000000688153  stepo_                    327  stepo.F90
oifs_43r3_model.e  00000000004CD1DC  cnt4_                    1133  cnt4.F90
oifs_43r3_model.e  00000000004B5D85  cnt3_                     267  cnt3.F90
oifs_43r3_model.e  00000000004B4F52  cnt2_                      88  cnt2.F90
oifs_43r3_model.e  000000000049D638  cnt1_                      92  cnt1.F90
oifs_43r3_model.e  0000000000406FBD  cnt0_                     146  cnt0.F90
oifs_43r3_model.e  0000000000401E7F  MAIN__                     96  master.F90
oifs_43r3_model.e  0000000000401E12  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001DCC34A  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001DCDBE7  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000000401CD5  Unknown               Unknown  Unknown
(argv0) ../../projects/climateprediction.net/oifs_43r3_bl_1.13_x86_64-pc-linux-gnu
(argv1) start_date: 2016092300
(argv2) exptid: h7zg
(argv3) unique_member_id: a1nf
(argv4) batchid: 1018
(argv5) wuid: 12290466
(argv6) fclen: 20
(argv7) app_name: oifs_43r3_bl
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/10
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_bl
version: 1.13
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/10/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/10/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_badaf24bf5f91bb2cf8a16db059ebf21 to: /var/lib/boinc-client/slots/10/oifs_43r3_bl_a1nf_2016092300_20_1018_12290466.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/10/oifs_43r3_bl_a1nf_2016092300_20_1018_12290466.zip
ic_ancil_file: ic_ancil_12290466
ifsdata_file: ifsdata_12290466
climate_data_file: clim_data_12290466
horiz_resolution: 159
vert_resolution: 91
grid_type: l_2
upload_interval: 96
utstep: 900
nfrres: restart dump frequency (steps) 48
Copying IC ancils from: ../../projects/climateprediction.net/jf_f7ecfd27aa1e68bbe7e8d19800e25c73 to: /var/lib/boinc-client/slots/10/ic_ancil_12290466.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/10/ic_ancil_12290466.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_6aab511b6642eae610d38ab0748de549 to: /var/lib/boinc-client/slots/10/ifsdata/ifsdata_12290466.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/10/ifsdata/ifsdata_12290466.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/10/159l_2/clim_data_12290466.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/10/159l_2/clim_data_12290466.zip
Checking for progress XML file: /var/lib/boinc-client/slots/10/progress_file_12290466.xml
Opened progress file ok : /var/lib/boinc-client/slots/10/progress_file_12290466.xml
-- Model is restarting --
Adjusting last_iter, 1808, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/10/progress_file_12290466.xml
last_cpu_time: 52370
upload_file_number: 18
last_iter: 1777
last_upload: 1555200
model_completed: 0
total_length_of_simulation: 1728000
result_base_name: oifs_43r3_bl_a1nf_2016092300_20_1018_12290466_0_r1605132715
The child process has been launched with process id: 3275
Executing the command: /var/lib/boinc-client/slots/10/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:SuperBobH1:1:1:3275:3275] [20240615:154702:1718480822:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2091] fp = 0x314f440
[EC_DRHOOK:SuperBobH1:1:1:3275:3275] [20240615:154702:1718480822:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:SuperBobH1:1:1:3275:3275] [20240615:154702:1718480822:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:SuperBobH1:1:1:3275:3275] [20240615:154702:1718480822:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:SuperBobH1:1:1:3275:3275] [20240615:154702:1718480822:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:SuperBobH1:1:1:3275:3275] [20240615:154702:1718480822:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:SuperBobH1:1:1:3275:3275] [20240615:154702:1718480822:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:SuperBobH1:1:1:3275:3275] [20240615:154702:1718480822:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:SuperBobH1:1:1:3275:3275] [20240615:154702:1718480822:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:SuperBobH1:1:1:3275:3275] [20240615:154702:1718480822:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:SuperBobH1:1:1:3275:3275] [20240615:154702:1718480822:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:SuperBobH1:1:1:3275:3275] [20240615:154702:1718480822:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:SuperBobH1:1:1:3275:3275] [20240615:154702:1718480822:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:SuperBobH1:1:1:3275:3275] [20240615:154702:1718480822:0.000] [ignore_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:SuperBobH1:1:1:3275:3275] [20240615:154702:1718480822:0.000] [restore_default_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:SuperBobH1:1:1:3275:3275] [20240615:154702:1718480822:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:SuperBobH1:1:1:3275:3275] [20240615:154702:1718480822:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:SuperBobH1:1:1:3275:3275] [20240615:154702:1718480822:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:SuperBobH1:1:1:3275:3275] [20240615:154702:1718480822:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x820cf0 (old at (nil))
[EC_DRHOOK:SuperBobH1:1:1:3275:3275] [20240615:154702:1718480822:0.000] [trapfpe_treatment@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:SuperBobH1:1:1:3275:3275] [20240615:154702:1718480822:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:SuperBobH1:1:1:3275:3275] [20240615:154702:1718480822:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:SuperBobH1:1:1:3275:3275] [20240615:154702:1718480822:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x820cf0 (old at (nil))
[EC_DRHOOK:SuperBobH1:1:1:3275:3275] [20240615:154702:1718480822:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:SuperBobH1:1:1:3275:3275] [20240615:154702:1718480822:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:SuperBobH1:1:1:3275:3275] [20240615:154702:1718480822:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:SuperBobH1:1:1:3275:3275] [20240615:154702:1718480822:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x820cf0 (old at (nil))
[EC_DRHOOK:SuperBobH1:1:1:3275:3275] [20240615:154702:1718480822:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x820cf0 (old at (nil))
[EC_DRHOOK:SuperBobH1:1:1:3275:3275] [20240615:154702:1718480822:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:SuperBobH1:1:1:3275:3275] [20240615:154702:1718480822:0.000] [catch_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  15:47:26 STEP 1776 H= 444:00 +CPU= 20.093
  15:47:56 STEP 1777 H= 444:15 +CPU= 26.547
  15:48:17 STEP 1778 H= 444:30 +CPU= 20.865
  15:48:37 STEP 1779 H= 444:45 +CPU= 19.818
  15:48:57 STEP 1780 H= 445:00 +CPU= 19.570
  15:49:16 STEP 1781 H= 445:15 +CPU= 19.651
  15:49:36 STEP 1782 H= 445:30 +CPU= 19.925
  15:49:57 STEP 1783 H= 445:45 +CPU= 19.985
  15:50:19 STEP 1784 H= 446:00 +CPU= 22.450
  15:50:39 STEP 1785 H= 446:15 +CPU= 19.786
  15:50:59 STEP 1786 H= 446:30 +CPU= 19.731
  15:51:19 STEP 1787 H= 446:45 +CPU= 19.761
  15:51:43 STEP 1788 H= 447:00 +CPU= 23.653
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGh7zg+001788
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHh7zg+001788
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAh7zg+001788
  15:52:14 STEP 1789 H= 447:15 +CPU= 30.980
  15:52:47 STEP 1790 H= 447:30 +CPU= 33.785
  15:53:23 STEP 1791 H= 447:45 +CPU= 35.134
  15:54:00 STEP 1792 H= 448:00 +CPU= 36.912
  15:54:38 STEP 1793 H= 448:15 +CPU= 38.082
  15:55:12 STEP 1794 H= 448:30 +CPU= 33.688
  15:55:42 STEP 1795 H= 448:45 +CPU= 30.806
  15:56:15 STEP 1796 H= 449:00 +CPU= 32.101
  15:56:44 STEP 1797 H= 449:15 +CPU= 29.244
  15:57:12 STEP 1798 H= 449:30 +CPU= 28.440
  15:57:42 STEP 1799 H= 449:45 +CPU= 29.991
  15:58:20 STEP 1800 H= 450:00 +CPU= 37.540
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGh7zg+001800
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHh7zg+001800
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAh7zg+001800
  15:58:50 STEP 1801 H= 450:15 +CPU= 30.045
  15:59:22 STEP 1802 H= 450:30 +CPU= 32.269
  15:59:54 STEP 1803 H= 450:45 +CPU= 31.342
  16:00:28 STEP 1804 H= 451:00 +CPU= 33.638
  16:01:03 STEP 1805 H= 451:15 +CPU= 35.375
  16:01:41 STEP 1806 H= 451:30 +CPU= 36.688
  16:02:14 STEP 1807 H= 451:45 +CPU= 33.397
  16:02:49 STEP 1808 H= 452:00 +CPU= 34.269
  16:03:22 STEP 1809 H= 452:15 +CPU= 33.656
  16:03:57 STEP 1810 H= 452:30 +CPU= 34.411
  16:04:30 STEP 1811 H= 452:45 +CPU= 32.290
  16:05:10 STEP 1812 H= 453:00 +CPU= 40.258
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGh7zg+001812
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHh7zg+001812
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAh7zg+001812
  16:05:43 STEP 1813 H= 453:15 +CPU= 32.163
  16:06:15 STEP 1814 H= 453:30 +CPU= 32.022
  16:06:49 STEP 1815 H= 453:45 +CPU= 33.685
  16:07:25 STEP 1816 H= 454:00 +CPU= 35.518
  16:07:59 STEP 1817 H= 454:15 +CPU= 34.341
  16:08:33 STEP 1818 H= 454:30 +CPU= 33.361
  16:09:06 STEP 1819 H= 454:45 +CPU= 33.471
  16:09:41 STEP 1820 H= 455:00 +CPU= 34.428
  16:10:18 STEP 1821 H= 455:15 +CPU= 36.430
  16:10:55 STEP 1822 H= 455:30 +CPU= 35.754
  16:11:31 STEP 1823 H= 455:45 +CPU= 36.032
  16:12:22 STEP 1824 H= 456:00 +CPU= 49.667
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGh7zg+001824
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHh7zg+001824
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAh7zg+001824
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMGGh7zg+001728
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMSHh7zg+001728
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMUAh7zg+001728
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMGGh7zg+001740
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMSHh7zg+001740
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMUAh7zg+001740
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMGGh7zg+001752
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMSHh7zg+001752
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMUAh7zg+001752
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMGGh7zg+001764
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMSHh7zg+001764
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMUAh7zg+001764
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMGGh7zg+001776
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMSHh7zg+001776
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMUAh7zg+001776
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMGGh7zg+001788
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMSHh7zg+001788
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMUAh7zg+001788
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMGGh7zg+001800
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMSHh7zg+001800
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMUAh7zg+001800
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMGGh7zg+001812
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMSHh7zg+001812
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMUAh7zg+001812
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a1nf_2016092300_20_1018_12290466_0_r1605132715_18.zip
Uploading the intermediate file: upload_file_18.zip
  16:12:59 STEP 1825 H= 456:15 +CPU= 36.226
  16:13:40 STEP 1826 H= 456:30 +CPU= 39.600
  16:14:19 STEP 1827 H= 456:45 +CPU= 38.024
  16:14:59 STEP 1828 H= 457:00 +CPU= 39.750
  16:15:38 STEP 1829 H= 457:15 +CPU= 37.739
  16:16:17 STEP 1830 H= 457:30 +CPU= 39.025
  16:16:56 STEP 1831 H= 457:45 +CPU= 37.562
  16:17:35 STEP 1832 H= 458:00 +CPU= 38.831
  16:18:14 STEP 1833 H= 458:15 +CPU= 38.216
  16:18:52 STEP 1834 H= 458:30 +CPU= 36.843
  16:19:36 STEP 1835 H= 458:45 +CPU= 42.946
  16:20:27 STEP 1836 H= 459:00 +CPU= 49.189
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGh7zg+001836
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHh7zg+001836
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAh7zg+001836
  16:21:03 STEP 1837 H= 459:15 +CPU= 35.807
  16:21:41 STEP 1838 H= 459:30 +CPU= 37.245
  16:22:24 STEP 1839 H= 459:45 +CPU= 42.543
  16:23:05 STEP 1840 H= 460:00 +CPU= 39.613
  16:23:39 STEP 1841 H= 460:15 +CPU= 34.471
  16:24:14 STEP 1842 H= 460:30 +CPU= 34.671
  16:24:50 STEP 1843 H= 460:45 +CPU= 35.894
  16:25:27 STEP 1844 H= 461:00 +CPU= 35.931
  16:26:05 STEP 1845 H= 461:15 +CPU= 37.627
  16:26:47 STEP 1846 H= 461:30 +CPU= 40.914
  16:27:26 STEP 1847 H= 461:45 +CPU= 38.485
  16:28:16 STEP 1848 H= 462:00 +CPU= 48.998
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGh7zg+001848
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHh7zg+001848
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAh7zg+001848
  16:28:56 STEP 1849 H= 462:15 +CPU= 39.007
  16:29:39 STEP 1850 H= 462:30 +CPU= 41.941
  16:30:26 STEP 1851 H= 462:45 +CPU= 44.610
  16:31:07 STEP 1852 H= 463:00 +CPU= 40.104
  16:31:44 STEP 1853 H= 463:15 +CPU= 36.302
  16:32:20 STEP 1854 H= 463:30 +CPU= 35.150
  16:33:08 STEP 1855 H= 463:45 +CPU= 45.238
  16:33:50 STEP 1856 H= 464:00 +CPU= 41.297
  16:34:24 STEP 1857 H= 464:15 +CPU= 34.175
  16:34:58 STEP 1858 H= 464:30 +CPU= 33.325
  16:35:31 STEP 1859 H= 464:45 +CPU= 32.302
  16:36:16 STEP 1860 H= 465:00 +CPU= 44.077
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGh7zg+001860
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHh7zg+001860
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAh7zg+001860
  16:36:56 STEP 1861 H= 465:15 +CPU= 39.742
  16:37:36 STEP 1862 H= 465:30 +CPU= 40.049
  16:38:19 STEP 1863 H= 465:45 +CPU= 42.105
  16:39:01 STEP 1864 H= 466:00 +CPU= 41.652
  16:39:36 STEP 1865 H= 466:15 +CPU= 34.130
  16:40:11 STEP 1866 H= 466:30 +CPU= 34.757
  16:40:45 STEP 1867 H= 466:45 +CPU= 33.849
  16:41:20 STEP 1868 H= 467:00 +CPU= 35.108
  16:41:55 STEP 1869 H= 467:15 +CPU= 33.783
  16:42:30 STEP 1870 H= 467:30 +CPU= 35.183
  16:43:04 STEP 1871 H= 467:45 +CPU= 33.719
  16:43:57 STEP 1872 H= 468:00 +CPU= 52.020
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGh7zg+001872
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHh7zg+001872
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAh7zg+001872
  16:44:36 STEP 1873 H= 468:15 +CPU= 38.177
  16:45:16 STEP 1874 H= 468:30 +CPU= 39.137
  16:45:58 STEP 1875 H= 468:45 +CPU= 40.938
  16:46:34 STEP 1876 H= 469:00 +CPU= 35.910
  16:47:09 STEP 1877 H= 469:15 +CPU= 34.086
  16:47:42 STEP 1878 H= 469:30 +CPU= 33.023
  16:48:15 STEP 1879 H= 469:45 +CPU= 33.176
  16:48:51 STEP 1880 H= 470:00 +CPU= 35.030
  16:49:26 STEP 1881 H= 470:15 +CPU= 35.267
  16:50:01 STEP 1882 H= 470:30 +CPU= 34.277
  16:50:33 STEP 1883 H= 470:45 +CPU= 31.455
  16:51:14 STEP 1884 H= 471:00 +CPU= 40.399
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGh7zg+001884
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHh7zg+001884
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAh7zg+001884
  16:51:47 STEP 1885 H= 471:15 +CPU= 32.229
  16:52:21 STEP 1886 H= 471:30 +CPU= 33.568
  16:52:56 STEP 1887 H= 471:45 +CPU= 35.001
  16:53:32 STEP 1888 H= 472:00 +CPU= 35.182
  16:54:06 STEP 1889 H= 472:15 +CPU= 33.395
  16:54:40 STEP 1890 H= 472:30 +CPU= 33.041
  16:55:14 STEP 1891 H= 472:45 +CPU= 34.101
  16:55:49 STEP 1892 H= 473:00 +CPU= 34.787
  16:56:24 STEP 1893 H= 473:15 +CPU= 33.644
  16:56:56 STEP 1894 H= 473:30 +CPU= 32.372
  16:57:32 STEP 1895 H= 473:45 +CPU= 34.712
  16:58:19 STEP 1896 H= 474:00 +CPU= 41.253
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGh7zg+001896
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHh7zg+001896
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAh7zg+001896
  16:58:53 STEP 1897 H= 474:15 +CPU= 34.487
  16:59:28 STEP 1898 H= 474:30 +CPU= 33.874
  17:00:04 STEP 1899 H= 474:45 +CPU= 35.927
  17:00:40 STEP 1900 H= 475:00 +CPU= 35.154
  17:01:15 STEP 1901 H= 475:15 +CPU= 34.499
  17:01:49 STEP 1902 H= 475:30 +CPU= 33.381
  17:02:22 STEP 1903 H= 475:45 +CPU= 32.842
  17:02:59 STEP 1904 H= 476:00 +CPU= 35.739
  17:03:35 STEP 1905 H= 476:15 +CPU= 36.209
  17:04:13 STEP 1906 H= 476:30 +CPU= 36.424
  17:04:59 STEP 1907 H= 476:45 +CPU= 38.537
  17:05:50 STEP 1908 H= 477:00 +CPU= 49.707
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGh7zg+001908
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHh7zg+001908
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAh7zg+001908
  17:06:30 STEP 1909 H= 477:15 +CPU= 38.510
  17:07:09 STEP 1910 H= 477:30 +CPU= 38.243
  17:07:49 STEP 1911 H= 477:45 +CPU= 36.349
  17:08:28 STEP 1912 H= 478:00 +CPU= 38.292
  17:09:11 STEP 1913 H= 478:15 +CPU= 37.902
  17:09:53 STEP 1914 H= 478:30 +CPU= 38.009
  17:10:33 STEP 1915 H= 478:45 +CPU= 36.790
  17:11:13 STEP 1916 H= 479:00 +CPU= 35.333
  17:11:50 STEP 1917 H= 479:15 +CPU= 34.108
  17:12:31 STEP 1918 H= 479:30 +CPU= 34.147
  17:13:05 STEP 1919 H= 479:45 +CPU= 33.787
  17:13:50 STEP 1920 H= 480:00 +CPU= 41.607
..The child process terminated with status: 0
>>> Printing last 70 lines from file: NODE.001_01

 
DDH------------------------------------------------------------------------
I                                                                         I
I TIME =  480.000H DDH * Diagnostic Var.,     AREA                GLOBAL  I
I                                                                         I
I-------------------------------------------------------------------------I
I   VARIABLE   I      Initial value         I        Current value        I
I-------------------------------------------------------------------------I
I  OMEGA (P/S) I     -0.3030311212E-21      I       0.5091476490E-06      I
ICloud fractionI      0.0000000000E+00      I       0.3077386014E-01      I
I  Relat. Hum. I      0.9872840634E-01      I       0.1664074758E+00      I
I  PASS 01     I      0.0000000000E+00      I       0.1684860890E+03      I
DDH------------------------------------------------------------------------
  
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
 NSTEP =  1920 SCAN2M_HPOS  P
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ICMGGh7zg+001920 MODE=a
 IO-STREAM CLOSED - ICMGGh7zg+001920
 MPI-TASK:   1 -    311104836 BYTES IN      7 RECORDS TRANSFERRED IN    0.0008 SECONDS388881.0 Mbytes/s, TOTAL TIME=    0.0148( 5.4%)
 17:13:50 STEP 1920 H= 480:00 +CPU= 41.607
 END CNT3
 NSTEP =  1920 CNT0   000000000  30.851  30.851  31.336   0.985               0               0               0               0               0
IO-STREAM STATISTICS, TOTAL NO OF REC/BYTES READ           46/  1069125708 WRITTEN         4290/************
 -TASK-OPENED-OPEN-RECS IN -KBYTE IN    -RECS OUT-KBYTE OUT-WALL   -WALL IN-WALL
  OU-TOT IN -TOT OUT
   1     78    0       46      1044068     4290 ************    12.1     1.9    10.2     2.8    21.2
===-=== START OF TIMING STATISTICS ===-===
 
STATS FOR ALL TASKS
 NUM ROUTINE                                     CALLS  MEAN(ms)   MAX(ms)   FRAC(%)  UNBAL(%)
   0 CNT0     - COMPLETE EXECUTION                   1 5208247.1 5208247.1    100.00      0.00
   1 CNT4     - FORWARD INTEGRATION                  1 5204117.0 5204117.0     99.92      0.00
   8 SCAN2M - GRID-POINT DYNAMICS                  157    6898.4    6898.4     20.79      0.00
   9 SPCM     - SPECTRAL COMP.                     144    1222.5    1222.5      3.38      0.00
  10 SCAN2M - PHYSICS                              145   14990.9   14990.9     41.74      0.00
  11 IOPACK   - OUTPUT P.P. RESULTS                 12    1279.4    1279.4      0.29      0.00
  12 SPNORM   - SPECTRAL NORM COMP.                  7      79.9      79.9      0.01      0.00
  17 GRIDFPOS IN CNT4                               12      40.8      40.8      0.01      0.00
  27 RESTART FILES - WRITING                         3    5117.6    5117.6      0.29      0.00
  28 RESTART FILES - READING                         1    3627.8    3627.8      0.07      0.00
  29 SU4FPOS IN CNT4                                12       0.0       0.0      0.00      0.00
  30 DYNFPOS IN CNT4                                12    6871.7    6871.7      1.58      0.00
  31 POSDDH IN STEPO                                 1       4.2       4.2      0.00      0.00
  37 CPGLAG   - SL COMPUTATIONS                    145    7906.8    7906.8     22.01      0.00
  39 SU0YOMB                                         1     302.4     302.4      0.01      0.00
  51 SCAN2M   - SL COMM. PART 1                    145      33.0      33.0      0.09      0.00
  54 SPCM     - M TO S/S TO M TRANSP.              144     699.6     699.6      1.93      0.00
  55 SPCIMPF  - S TO M/M TO S TRANSP.              144     113.5     113.5      0.31      0.00
  56 SPNORM   - SPECTRAL NORM COMM.                  7       0.4       0.4      0.00      0.00
 102 LTINV_CTL   - INVERSE LEGENDRE TRANSFORM      316     487.1     487.1      2.96      0.00
 103 LTDIR_CTL   - DIRECT LEGENDRE TRANSFORM       300     267.3     267.3      1.54      0.00
 106 FTDIR_CTL   - DIRECT FOURIER TRANSFORM        300      91.4      91.4      0.53      0.00
 107 FTINV_CTL   - INVERSE FOURIER TRANSFORM       316     137.8     137.8      0.84      0.00
 140 SULEG       - COMP. OF LEGENDRE POL.            1      29.5      29.5      0.00      0.00
 152 LTINV_CTL   - M TO L TRANSPOSITION            316      96.1      96.1      0.58      0.00
 153 LTDIR_CTL   - L TO M TRANSPOSITION            300      54.6      54.6      0.31      0.00
 157 FTINV_CTL   - L TO G TRANSPOSITION            316     118.7     118.7      0.72      0.00
 158 FTDIR_CTL   - G TO L TRANSPOSITION            300      58.2      58.2      0.33      0.00
 400 GSTATS                                      17763       0.0       0.0      0.00      0.00
 401 GSTATS HOOK                                 16473       0.0       0.0      0.00      0.00
TOTAL MEASURED IMBALANCE =       0.0 SECONDS,  0.0 PERCENT
TOTAL WALLCLOCK TIME     5208.2 CPU TIME    5090.6 VECTOR TIME     5090.6


===-=== END   OF TIMING STATISTICS ===-===
FORECAST DAYS PER DAY      332.0
  *** END CNT0 *** 
------------------------------------------------
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGh7zg+001920
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHh7zg+001920
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAh7zg+001920
Adding to the zip: /var/lib/boinc-client/slots/10/NODE.001_01
Adding to the zip: /var/lib/boinc-client/slots/10/ifs.stat
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMUAh7zg+001824
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMGGh7zg+001884
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMSHh7zg+001860
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMSHh7zg+001896
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMGGh7zg+001836
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMSHh7zg+001908
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMSHh7zg+001872
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMUAh7zg+001860
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMGGh7zg+001896
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMUAh7zg+001836
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMUAh7zg+001872
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMGGh7zg+001848
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMSHh7zg+001848
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMGGh7zg+001860
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMGGh7zg+001920
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMSHh7zg+001824
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMUAh7zg+001920
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMUAh7zg+001896
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMUAh7zg+001884
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMSHh7zg+001836
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMUAh7zg+001848
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMGGh7zg+001824
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMSHh7zg+001920
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMGGh7zg+001908
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMUAh7zg+001908
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMGGh7zg+001872
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290466/ICMSHh7zg+001884
Zipping up the final file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a1nf_2016092300_20_1018_12290466_0_r1605132715_19.zip
Uploading the final file: upload_file_19.zip
Uploading trickle at timestep: 1727100
17:17:18 (3226): called boinc_finish(0)

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
15 Jun 2024 21:16:47 1532599 22444059 oifs_43r3_bl_a1nf_2016092300_20_1018_12290466_0 1,727,100 57,460 0.0333
15 Jun 2024 16:04:50 1532599 22444059 oifs_43r3_bl_a1nf_2016092300_20_1018_12290466_0 1,209,600 38,296 0.0317


©2025 cpdn.org