climateprediction.net home page
Task 22449782

Task 22449782

Name oifs_43r3_bl_a0bb_2016092300_20_1018_12288734_1
Workunit 12288734
Created 18 Jun 2024, 6:12:09 UTC
Sent 18 Jun 2024, 6:12:55 UTC
Report deadline 17 Aug 2024, 6:12:55 UTC
Received 17 Jul 2024, 15:14:46 UTC
Server state Over
Outcome Success
Client state Done
Exit status 0 (0x00000000)
Computer ID 1541000
Run time 10 hours 26 min 51 sec
CPU time 10 hours 26 min 51 sec
Validate state Valid
Credit 2,635.55
Device peak FLOPS 4.91 GFLOPS
Application version OpenIFS 43r3 Baroclinic Lifecycle v1.13
x86_64-pc-linux-gnu
Peak working set size 5,023.80 MB
Peak swap size 5,981.08 MB
Peak disk usage 2,998.39 MB
Stderr
<core_client_version>8.0.2</core_client_version>
<![CDATA[
<stderr_txt>
b/boinc-client/slots/3/ICMSHh7zg+001548
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAh7zg+001548
  13:23:36 STEP 1549 H= 387:15 +CPU= 14.594
  13:23:50 STEP 1550 H= 387:30 +CPU= 14.639
  13:24:05 STEP 1551 H= 387:45 +CPU= 14.592
  13:24:20 STEP 1552 H= 388:00 +CPU= 15.462
  13:24:35 STEP 1553 H= 388:15 +CPU= 14.418
  13:24:49 STEP 1554 H= 388:30 +CPU= 14.224
  13:25:04 STEP 1555 H= 388:45 +CPU= 14.523
  13:25:18 STEP 1556 H= 389:00 +CPU= 14.668
  13:25:33 STEP 1557 H= 389:15 +CPU= 14.550
  13:25:47 STEP 1558 H= 389:30 +CPU= 14.232
  13:26:01 STEP 1559 H= 389:45 +CPU= 14.410
  13:26:19 STEP 1560 H= 390:00 +CPU= 17.746
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGh7zg+001560
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHh7zg+001560
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAh7zg+001560
  13:26:34 STEP 1561 H= 390:15 +CPU= 14.867
  13:26:49 STEP 1562 H= 390:30 +CPU= 14.648
  13:27:03 STEP 1563 H= 390:45 +CPU= 14.707
  13:27:19 STEP 1564 H= 391:00 +CPU= 15.062
  13:27:33 STEP 1565 H= 391:15 +CPU= 14.856
  13:27:48 STEP 1566 H= 391:30 +CPU= 14.833
  13:28:03 STEP 1567 H= 391:45 +CPU= 14.710
  13:28:18 STEP 1568 H= 392:00 +CPU= 14.925
  13:28:33 STEP 1569 H= 392:15 +CPU= 14.789
  13:28:47 STEP 1570 H= 392:30 +CPU= 14.587
  13:29:02 STEP 1571 H= 392:45 +CPU= 14.875
  13:29:20 STEP 1572 H= 393:00 +CPU= 18.066
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGh7zg+001572
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHh7zg+001572
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAh7zg+001572
  13:29:35 STEP 1573 H= 393:15 +CPU= 14.750
  13:29:50 STEP 1574 H= 393:30 +CPU= 14.781
  13:30:04 STEP 1575 H= 393:45 +CPU= 14.583
  13:30:19 STEP 1576 H= 394:00 +CPU= 14.973
  13:30:34 STEP 1577 H= 394:15 +CPU= 14.736
  13:30:49 STEP 1578 H= 394:30 +CPU= 14.623
  13:31:04 STEP 1579 H= 394:45 +CPU= 14.760
  13:31:19 STEP 1580 H= 395:00 +CPU= 15.162
  13:31:33 STEP 1581 H= 395:15 +CPU= 14.527
  13:31:48 STEP 1582 H= 395:30 +CPU= 14.647
  13:32:03 STEP 1583 H= 395:45 +CPU= 15.016
  13:32:23 STEP 1584 H= 396:00 +CPU= 19.688
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGh7zg+001584
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHh7zg+001584
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAh7zg+001584
  13:32:37 STEP 1585 H= 396:15 +CPU= 14.738
  13:32:52 STEP 1586 H= 396:30 +CPU= 14.665
  13:33:07 STEP 1587 H= 396:45 +CPU= 14.738
  13:33:22 STEP 1588 H= 397:00 +CPU= 14.945
  13:33:37 STEP 1589 H= 397:15 +CPU= 14.686
  13:33:51 STEP 1590 H= 397:30 +CPU= 14.543
  13:34:06 STEP 1591 H= 397:45 +CPU= 14.517
  13:34:20 STEP 1592 H= 398:00 +CPU= 14.805
  13:34:35 STEP 1593 H= 398:15 +CPU= 14.793
  13:34:50 STEP 1594 H= 398:30 +CPU= 14.732
  13:35:05 STEP 1595 H= 398:45 +CPU= 14.766
  13:35:23 STEP 1596 H= 399:00 +CPU= 18.102
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGh7zg+001596
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHh7zg+001596
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAh7zg+001596
  13:35:38 STEP 1597 H= 399:15 +CPU= 14.733
  13:35:53 STEP 1598 H= 399:30 +CPU= 15.016
  13:36:08 STEP 1599 H= 399:45 +CPU= 14.909
  13:36:22 STEP 1600 H= 400:00 +CPU= 14.803
  13:36:37 STEP 1601 H= 400:15 +CPU= 14.802
  13:36:52 STEP 1602 H= 400:30 +CPU= 14.583
  13:37:07 STEP 1603 H= 400:45 +CPU= 14.887
  13:37:22 STEP 1604 H= 401:00 +CPU= 14.916
  13:37:36 STEP 1605 H= 401:15 +CPU= 14.562
  13:37:51 STEP 1606 H= 401:30 +CPU= 14.463
  13:38:05 STEP 1607 H= 401:45 +CPU= 14.592
  13:38:23 STEP 1608 H= 402:00 +CPU= 18.029
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGh7zg+001608
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHh7zg+001608
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAh7zg+001608
  13:38:38 STEP 1609 H= 402:15 +CPU= 14.367
  13:38:53 STEP 1610 H= 402:30 +CPU= 14.749
  13:39:07 STEP 1611 H= 402:45 +CPU= 14.446
  13:39:22 STEP 1612 H= 403:00 +CPU= 14.866
  13:39:36 STEP 1613 H= 403:15 +CPU= 14.468
  13:39:51 STEP 1614 H= 403:30 +CPU= 14.162
  13:40:05 STEP 1615 H= 403:45 +CPU= 14.930
  13:40:20 STEP 1616 H= 404:00 +CPU= 14.452
  13:40:34 STEP 1617 H= 404:15 +CPU= 14.248
  13:40:48 STEP 1618 H= 404:30 +CPU= 14.287
  13:41:03 STEP 1619 H= 404:45 +CPU= 14.206
  13:41:21 STEP 1620 H= 405:00 +CPU= 18.216
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGh7zg+001620
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHh7zg+001620
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAh7zg+001620
  13:41:36 STEP 1621 H= 405:15 +CPU= 14.894
  13:41:50 STEP 1622 H= 405:30 +CPU= 14.311
  13:42:05 STEP 1623 H= 405:45 +CPU= 14.548
  13:42:20 STEP 1624 H= 406:00 +CPU= 15.190
  13:42:34 STEP 1625 H= 406:15 +CPU= 14.468
  13:42:49 STEP 1626 H= 406:30 +CPU= 14.641
  13:43:04 STEP 1627 H= 406:45 +CPU= 14.867
  13:43:19 STEP 1628 H= 407:00 +CPU= 14.861
  13:43:33 STEP 1629 H= 407:15 +CPU= 14.606
  13:43:48 STEP 1630 H= 407:30 +CPU= 14.881
  13:44:03 STEP 1631 H= 407:45 +CPU= 14.605
  13:44:23 STEP 1632 H= 408:00 +CPU= 19.958
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGh7zg+001632
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHh7zg+001632
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAh7zg+001632
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288734/ICMGGh7zg+001536
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288734/ICMSHh7zg+001536
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288734/ICMUAh7zg+001536
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288734/ICMGGh7zg+001548
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288734/ICMSHh7zg+001548
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288734/ICMUAh7zg+001548
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288734/ICMGGh7zg+001560
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288734/ICMSHh7zg+001560
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288734/ICMUAh7zg+001560
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288734/ICMGGh7zg+001572
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288734/ICMSHh7zg+001572
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288734/ICMUAh7zg+001572
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288734/ICMGGh7zg+001584
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288734/ICMSHh7zg+001584
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288734/ICMUAh7zg+001584
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288734/ICMGGh7zg+001596
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288734/ICMSHh7zg+001596
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288734/ICMUAh7zg+001596
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288734/ICMGGh7zg+001608
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288734/ICMSHh7zg+001608
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288734/ICMUAh7zg+001608
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288734/ICMGGh7zg+001620
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288734/ICMSHh7zg+001620
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288734/ICMUAh7zg+001620
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a0bb_2016092300_20_1018_12288734_1_r240171188_16.zip
Uploading the intermediate file: upload_file_16.zip
  13:44:37 STEP 1633 H= 408:15 +CPU= 14.296
  13:44:52 STEP 1634 H= 408:30 +CPU= 14.611
  13:45:06 STEP 1635 H= 408:45 +CPU= 14.571
  13:45:21 STEP 1636 H= 409:00 +CPU= 14.711
  13:45:36 STEP 1637 H= 409:15 +CPU= 14.547
  13:45:51 STEP 1638 H= 409:30 +CPU= 14.990
  13:46:06 STEP 1639 H= 409:45 +CPU= 14.832
  13:46:20 STEP 1640 H= 410:00 +CPU= 14.463
  13:46:34 STEP 1641 H= 410:15 +CPU= 14.277
  13:46:49 STEP 1642 H= 410:30 +CPU= 14.998
  13:47:04 STEP 1643 H= 410:45 +CPU= 14.564
  13:47:22 STEP 1644 H= 411:00 +CPU= 17.850
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGh7zg+001644
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHh7zg+001644
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAh7zg+001644
  13:47:37 STEP 1645 H= 411:15 +CPU= 14.826
  13:47:52 STEP 1646 H= 411:30 +CPU= 14.927
  13:48:06 STEP 1647 H= 411:45 +CPU= 14.651
  13:48:21 STEP 1648 H= 412:00 +CPU= 14.547
  13:48:36 STEP 1649 H= 412:15 +CPU= 14.818
  13:48:50 STEP 1650 H= 412:30 +CPU= 14.471
  13:49:05 STEP 1651 H= 412:45 +CPU= 14.720
  13:49:20 STEP 1652 H= 413:00 +CPU= 15.004
  13:49:35 STEP 1653 H= 413:15 +CPU= 14.608
  13:49:49 STEP 1654 H= 413:30 +CPU= 14.669
  13:50:04 STEP 1655 H= 413:45 +CPU= 14.547
  13:50:22 STEP 1656 H= 414:00 +CPU= 18.117
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGh7zg+001656
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHh7zg+001656
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAh7zg+001656
  13:50:36 STEP 1657 H= 414:15 +CPU= 14.114
  13:50:50 STEP 1658 H= 414:30 +CPU= 14.090
  13:51:04 STEP 1659 H= 414:45 +CPU= 14.196
  13:51:19 STEP 1660 H= 415:00 +CPU= 14.392
  13:51:33 STEP 1661 H= 415:15 +CPU= 14.449
  13:51:47 STEP 1662 H= 415:30 +CPU= 14.186
  13:52:02 STEP 1663 H= 415:45 +CPU= 14.481
  13:52:17 STEP 1664 H= 416:00 +CPU= 14.738
  13:52:31 STEP 1665 H= 416:15 +CPU= 14.228
  13:52:46 STEP 1666 H= 416:30 +CPU= 14.910
  13:53:01 STEP 1667 H= 416:45 +CPU= 14.762
  13:53:18 STEP 1668 H= 417:00 +CPU= 17.744
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGh7zg+001668
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHh7zg+001668
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAh7zg+001668
  13:53:33 STEP 1669 H= 417:15 +CPU= 14.680
  13:53:47 STEP 1670 H= 417:30 +CPU= 14.168
  13:54:01 STEP 1671 H= 417:45 +CPU= 14.157
  13:54:16 STEP 1672 H= 418:00 +CPU= 14.963
  13:54:31 STEP 1673 H= 418:15 +CPU= 14.797
  13:54:46 STEP 1674 H= 418:30 +CPU= 14.403
  13:55:00 STEP 1675 H= 418:45 +CPU= 14.602
  13:55:15 STEP 1676 H= 419:00 +CPU= 15.012
  13:55:30 STEP 1677 H= 419:15 +CPU= 14.945
  13:55:44 STEP 1678 H= 419:30 +CPU= 14.102
  13:55:59 STEP 1679 H= 419:45 +CPU= 14.350
  13:56:18 STEP 1680 H= 420:00 +CPU= 19.565
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGh7zg+001680
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHh7zg+001680
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAh7zg+001680
  13:56:33 STEP 1681 H= 420:15 +CPU= 14.444
  13:56:48 STEP 1682 H= 420:30 +CPU= 14.809
  13:57:02 STEP 1683 H= 420:45 +CPU= 14.195
  13:57:16 STEP 1684 H= 421:00 +CPU= 14.430
  13:57:30 STEP 1685 H= 421:15 +CPU= 14.162
  13:57:44 STEP 1686 H= 421:30 +CPU= 14.170
  13:57:59 STEP 1687 H= 421:45 +CPU= 14.208
  13:58:13 STEP 1688 H= 422:00 +CPU= 14.427
  13:58:28 STEP 1689 H= 422:15 +CPU= 14.457
  13:58:42 STEP 1690 H= 422:30 +CPU= 14.514
  13:58:57 STEP 1691 H= 422:45 +CPU= 14.930
  13:59:15 STEP 1692 H= 423:00 +CPU= 17.844
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGh7zg+001692
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHh7zg+001692
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAh7zg+001692
  13:59:29 STEP 1693 H= 423:15 +CPU= 14.208
  13:59:44 STEP 1694 H= 423:30 +CPU= 14.910
  13:59:59 STEP 1695 H= 423:45 +CPU= 14.926
  14:00:14 STEP 1696 H= 424:00 +CPU= 15.118
  14:00:29 STEP 1697 H= 424:15 +CPU= 14.750
  14:00:44 STEP 1698 H= 424:30 +CPU= 14.709
  14:00:59 STEP 1699 H= 424:45 +CPU= 14.973
  14:01:14 STEP 1700 H= 425:00 +CPU= 14.952
  14:01:28 STEP 1701 H= 425:15 +CPU= 14.584
  14:01:42 STEP 1702 H= 425:30 +CPU= 14.092
  14:01:56 STEP 1703 H= 425:45 +CPU= 14.111
  14:02:14 STEP 1704 H= 426:00 +CPU= 18.052
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGh7zg+001704
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHh7zg+001704
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAh7zg+001704
  14:02:29 STEP 1705 H= 426:15 +CPU= 14.465
  14:02:43 STEP 1706 H= 426:30 +CPU= 14.142
  14:02:58 STEP 1707 H= 426:45 +CPU= 14.745
  14:03:13 STEP 1708 H= 427:00 +CPU= 14.865
  14:03:27 STEP 1709 H= 427:15 +CPU= 14.683
  14:03:42 STEP 1710 H= 427:30 +CPU= 14.794
  14:03:57 STEP 1711 H= 427:45 +CPU= 14.783
  14:04:12 STEP 1712 H= 428:00 +CPU= 14.713
  14:04:26 STEP 1713 H= 428:15 +CPU= 14.447
  14:04:41 STEP 1714 H= 428:30 +CPU= 14.644
  14:04:55 STEP 1715 H= 428:45 +CPU= 14.470
  14:05:13 STEP 1716 H= 429:00 +CPU= 17.903
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGh7zg+001716
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHh7zg+001716
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAh7zg+001716
  14:05:28 STEP 1717 H= 429:15 +CPU= 14.398
  14:05:42 STEP 1718 H= 429:30 +CPU= 14.746
  14:05:57 STEP 1719 H= 429:45 +CPU= 14.629
  14:06:12 STEP 1720 H= 430:00 +CPU= 14.840
  14:06:27 STEP 1721 H= 430:15 +CPU= 14.805
  14:06:41 STEP 1722 H= 430:30 +CPU= 14.493
  14:06:56 STEP 1723 H= 430:45 +CPU= 14.492
  14:07:10 STEP 1724 H= 431:00 +CPU= 14.564
  14:07:25 STEP 1725 H= 431:15 +CPU= 14.896
  14:07:40 STEP 1726 H= 431:30 +CPU= 14.350
  14:07:54 STEP 1727 H= 431:45 +CPU= 14.727
  14:08:14 STEP 1728 H= 432:00 +CPU= 20.082
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGh7zg+001728
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHh7zg+001728
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAh7zg+001728
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288734/ICMGGh7zg+001632
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288734/ICMSHh7zg+001632
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288734/ICMUAh7zg+001632
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288734/ICMGGh7zg+001644
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288734/ICMSHh7zg+001644
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288734/ICMUAh7zg+001644
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288734/ICMGGh7zg+001656
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288734/ICMSHh7zg+001656
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288734/ICMUAh7zg+001656
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288734/ICMGGh7zg+001668
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288734/ICMSHh7zg+001668
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288734/ICMUAh7zg+001668
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288734/ICMGGh7zg+001680
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288734/ICMSHh7zg+001680
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288734/ICMUAh7zg+001680
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288734/ICMGGh7zg+001692
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288734/ICMSHh7zg+001692
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288734/ICMUAh7zg+001692
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288734/ICMGGh7zg+001704
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288734/ICMSHh7zg+001704
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288734/ICMUAh7zg+001704
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288734/ICMGGh7zg+001716
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288734/ICMSHh7zg+001716
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288734/ICMUAh7zg+001716
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a0bb_2016092300_20_1018_12288734_1_r240171188_17.zip
Uploading the intermediate file: upload_file_17.zip
  14:08:29 STEP 1729 H= 432:15 +CPU= 14.611
  14:08:44 STEP 1730 H= 432:30 +CPU= 14.675
  14:08:58 STEP 1731 H= 432:45 +CPU= 14.395
  14:09:13 STEP 1732 H= 433:00 +CPU= 14.469
  14:09:27 STEP 1733 H= 433:15 +CPU= 14.165
  14:09:41 STEP 1734 H= 433:30 +CPU= 14.183
  14:09:55 STEP 1735 H= 433:45 +CPU= 14.198
  14:10:10 STEP 1736 H= 434:00 +CPU= 14.627
  14:10:24 STEP 1737 H= 434:15 +CPU= 14.605
  14:10:39 STEP 1738 H= 434:30 +CPU= 14.267
  14:10:53 STEP 1739 H= 434:45 +CPU= 14.379
  14:11:11 STEP 1740 H= 435:00 +CPU= 18.049
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGh7zg+001740
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHh7zg+001740
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAh7zg+001740
  14:11:26 STEP 1741 H= 435:15 +CPU= 14.575
  14:11:41 STEP 1742 H= 435:30 +CPU= 14.761
  14:11:55 STEP 1743 H= 435:45 +CPU= 14.454
  14:12:10 STEP 1744 H= 436:00 +CPU= 14.796
  14:12:24 STEP 1745 H= 436:15 +CPU= 14.471
  14:12:39 STEP 1746 H= 436:30 +CPU= 14.701
  14:12:53 STEP 1747 H= 436:45 +CPU= 14.303
  14:13:08 STEP 1748 H= 437:00 +CPU= 14.690
  14:13:22 STEP 1749 H= 437:15 +CPU= 14.316
  14:13:37 STEP 1750 H= 437:30 +CPU= 14.583
  14:13:51 STEP 1751 H= 437:45 +CPU= 14.396
  14:14:10 STEP 1752 H= 438:00 +CPU= 18.203
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGh7zg+001752
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHh7zg+001752
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAh7zg+001752
  14:14:24 STEP 1753 H= 438:15 +CPU= 14.417
  14:14:39 STEP 1754 H= 438:30 +CPU= 14.451
  14:14:53 STEP 1755 H= 438:45 +CPU= 14.315
  14:15:08 STEP 1756 H= 439:00 +CPU= 14.669
  14:15:22 STEP 1757 H= 439:15 +CPU= 14.262
  14:15:36 STEP 1758 H= 439:30 +CPU= 14.272
  14:15:50 STEP 1759 H= 439:45 +CPU= 14.356
  14:16:06 STEP 1760 H= 440:00 +CPU= 15.096
  14:16:20 STEP 1761 H= 440:15 +CPU= 14.565
  14:16:34 STEP 1762 H= 440:30 +CPU= 14.105
  14:16:49 STEP 1763 H= 440:45 +CPU= 14.586
  14:17:07 STEP 1764 H= 441:00 +CPU= 17.811
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGh7zg+001764
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHh7zg+001764
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAh7zg+001764
  14:17:21 STEP 1765 H= 441:15 +CPU= 14.840
  14:17:36 STEP 1766 H= 441:30 +CPU= 14.932
  14:17:51 STEP 1767 H= 441:45 +CPU= 14.706
  14:18:06 STEP 1768 H= 442:00 +CPU= 14.800
  14:18:21 STEP 1769 H= 442:15 +CPU= 14.642
  14:18:35 STEP 1770 H= 442:30 +CPU= 14.674
  14:18:50 STEP 1771 H= 442:45 +CPU= 14.746
  14:19:05 STEP 1772 H= 443:00 +CPU= 14.812
  14:19:19 STEP 1773 H= 443:15 +CPU= 14.384
  14:19:33 STEP 1774 H= 443:30 +CPU= 14.151
  14:19:48 STEP 1775 H= 443:45 +CPU= 14.410
  14:20:08 STEP 1776 H= 444:00 +CPU= 19.951
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGh7zg+001776
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHh7zg+001776
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAh7zg+001776
  14:20:22 STEP 1777 H= 444:15 +CPU= 14.556
  14:20:37 STEP 1778 H= 444:30 +CPU= 14.461
  14:20:51 STEP 1779 H= 444:45 +CPU= 14.499
  14:21:06 STEP 1780 H= 445:00 +CPU= 14.868
  14:21:21 STEP 1781 H= 445:15 +CPU= 14.239
  14:21:35 STEP 1782 H= 445:30 +CPU= 14.945
  14:21:50 STEP 1783 H= 445:45 +CPU= 14.738
  14:22:05 STEP 1784 H= 446:00 +CPU= 14.610
  14:22:19 STEP 1785 H= 446:15 +CPU= 14.451
  14:22:33 STEP 1786 H= 446:30 +CPU= 13.979
  14:22:47 STEP 1787 H= 446:45 +CPU= 13.865
  14:23:04 STEP 1788 H= 447:00 +CPU= 17.148
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGh7zg+001788
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHh7zg+001788
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAh7zg+001788
  14:23:19 STEP 1789 H= 447:15 +CPU= 14.361
  14:23:33 STEP 1790 H= 447:30 +CPU= 13.855
  14:23:47 STEP 1791 H= 447:45 +CPU= 14.006
  14:24:01 STEP 1792 H= 448:00 +CPU= 14.334
  14:24:15 STEP 1793 H= 448:15 +CPU= 13.844
  14:24:28 STEP 1794 H= 448:30 +CPU= 13.654
  14:24:42 STEP 1795 H= 448:45 +CPU= 13.725
  14:24:57 STEP 1796 H= 449:00 +CPU= 14.349
  14:25:11 STEP 1797 H= 449:15 +CPU= 14.320
  14:25:25 STEP 1798 H= 449:30 +CPU= 14.137
  14:25:40 STEP 1799 H= 449:45 +CPU= 14.528
  14:25:57 STEP 1800 H= 450:00 +CPU= 17.040
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGh7zg+001800
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHh7zg+001800
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAh7zg+001800
  14:26:10 STEP 1801 H= 450:15 +CPU= 13.291
  14:26:23 STEP 1802 H= 450:30 +CPU= 13.250
  14:26:36 STEP 1803 H= 450:45 +CPU= 13.247
  14:26:50 STEP 1804 H= 451:00 +CPU= 13.828
  14:27:04 STEP 1805 H= 451:15 +CPU= 14.099
  14:27:18 STEP 1806 H= 451:30 +CPU= 13.938
  14:27:33 STEP 1807 H= 451:45 +CPU= 14.235
  14:27:47 STEP 1808 H= 452:00 +CPU= 14.200
  14:28:00 STEP 1809 H= 452:15 +CPU= 13.561
  14:28:14 STEP 1810 H= 452:30 +CPU= 13.329
  14:28:28 STEP 1811 H= 452:45 +CPU= 13.910
  14:28:45 STEP 1812 H= 453:00 +CPU= 17.743
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGh7zg+001812
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHh7zg+001812
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAh7zg+001812
  14:29:00 STEP 1813 H= 453:15 +CPU= 14.073
  14:29:14 STEP 1814 H= 453:30 +CPU= 14.120
  14:29:27 STEP 1815 H= 453:45 +CPU= 13.569
  14:29:41 STEP 1816 H= 454:00 +CPU= 14.159
  14:29:55 STEP 1817 H= 454:15 +CPU= 13.290
  14:30:08 STEP 1818 H= 454:30 +CPU= 13.303
  14:30:21 STEP 1819 H= 454:45 +CPU= 13.348
  14:30:35 STEP 1820 H= 455:00 +CPU= 13.812
  14:30:49 STEP 1821 H= 455:15 +CPU= 14.064
  14:31:03 STEP 1822 H= 455:30 +CPU= 13.732
  14:31:17 STEP 1823 H= 455:45 +CPU= 13.922
  14:31:36 STEP 1824 H= 456:00 +CPU= 18.582
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGh7zg+001824
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHh7zg+001824
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAh7zg+001824
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288734/ICMGGh7zg+001728
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288734/ICMSHh7zg+001728
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288734/ICMUAh7zg+001728
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288734/ICMGGh7zg+001740
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288734/ICMSHh7zg+001740
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288734/ICMUAh7zg+001740
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288734/ICMGGh7zg+001752
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288734/ICMSHh7zg+001752
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288734/ICMUAh7zg+001752
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288734/ICMGGh7zg+001764
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288734/ICMSHh7zg+001764
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288734/ICMUAh7zg+001764
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288734/ICMGGh7zg+001776
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288734/ICMSHh7zg+001776
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288734/ICMUAh7zg+001776
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288734/ICMGGh7zg+001788
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288734/ICMSHh7zg+001788
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288734/ICMUAh7zg+001788
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288734/ICMGGh7zg+001800
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288734/ICMSHh7zg+001800
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288734/ICMUAh7zg+001800
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288734/ICMGGh7zg+001812
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288734/ICMSHh7zg+001812
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288734/ICMUAh7zg+001812
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a0bb_2016092300_20_1018_12288734_1_r240171188_18.zip
  14:31:50 STEP 1825 H= 456:15 +CPU= 14.072
Uploading the intermediate file: upload_file_18.zip
  14:32:04 STEP 1826 H= 456:30 +CPU= 13.972
  14:32:18 STEP 1827 H= 456:45 +CPU= 14.021
  14:32:32 STEP 1828 H= 457:00 +CPU= 14.157
  14:32:46 STEP 1829 H= 457:15 +CPU= 13.810
  14:33:00 STEP 1830 H= 457:30 +CPU= 13.961
  14:33:14 STEP 1831 H= 457:45 +CPU= 13.774
  14:33:27 STEP 1832 H= 458:00 +CPU= 13.800
  14:33:41 STEP 1833 H= 458:15 +CPU= 13.413
  14:33:55 STEP 1834 H= 458:30 +CPU= 13.955
  14:34:09 STEP 1835 H= 458:45 +CPU= 13.771
  14:34:26 STEP 1836 H= 459:00 +CPU= 17.296
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGh7zg+001836
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHh7zg+001836
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAh7zg+001836
  14:34:39 STEP 1837 H= 459:15 +CPU= 13.586
  14:34:54 STEP 1838 H= 459:30 +CPU= 14.074
  14:35:07 STEP 1839 H= 459:45 +CPU= 13.645
  14:35:21 STEP 1840 H= 460:00 +CPU= 13.751
  14:35:35 STEP 1841 H= 460:15 +CPU= 13.776
  14:35:49 STEP 1842 H= 460:30 +CPU= 14.072
  14:36:03 STEP 1843 H= 460:45 +CPU= 13.651
  14:36:17 STEP 1844 H= 461:00 +CPU= 14.519
  14:36:31 STEP 1845 H= 461:15 +CPU= 14.033
  14:36:45 STEP 1846 H= 461:30 +CPU= 13.863
  14:36:59 STEP 1847 H= 461:45 +CPU= 13.786
  14:37:16 STEP 1848 H= 462:00 +CPU= 17.344
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGh7zg+001848
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHh7zg+001848
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAh7zg+001848
  14:37:30 STEP 1849 H= 462:15 +CPU= 13.325
  14:37:44 STEP 1850 H= 462:30 +CPU= 13.947
  14:37:57 STEP 1851 H= 462:45 +CPU= 13.785
  14:38:11 STEP 1852 H= 463:00 +CPU= 13.674
  14:38:24 STEP 1853 H= 463:15 +CPU= 13.442
  14:38:38 STEP 1854 H= 463:30 +CPU= 13.705
  14:38:52 STEP 1855 H= 463:45 +CPU= 13.362
  14:39:05 STEP 1856 H= 464:00 +CPU= 13.648
  14:39:19 STEP 1857 H= 464:15 +CPU= 13.470
  14:39:32 STEP 1858 H= 464:30 +CPU= 13.731
  14:39:46 STEP 1859 H= 464:45 +CPU= 13.571
  14:40:03 STEP 1860 H= 465:00 +CPU= 17.365
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGh7zg+001860
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHh7zg+001860
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAh7zg+001860
  14:40:17 STEP 1861 H= 465:15 +CPU= 13.754
  14:40:31 STEP 1862 H= 465:30 +CPU= 13.399
  14:40:45 STEP 1863 H= 465:45 +CPU= 13.977
  14:40:59 STEP 1864 H= 466:00 +CPU= 14.439
  14:41:13 STEP 1865 H= 466:15 +CPU= 14.043
  14:41:27 STEP 1866 H= 466:30 +CPU= 14.116
  14:41:41 STEP 1867 H= 466:45 +CPU= 13.937
  14:41:56 STEP 1868 H= 467:00 +CPU= 14.519
  14:42:09 STEP 1869 H= 467:15 +CPU= 13.705
  14:42:23 STEP 1870 H= 467:30 +CPU= 13.815
  14:42:36 STEP 1871 H= 467:45 +CPU= 13.157
  14:42:55 STEP 1872 H= 468:00 +CPU= 18.315
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGh7zg+001872
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHh7zg+001872
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAh7zg+001872
  14:43:08 STEP 1873 H= 468:15 +CPU= 13.353
  14:43:22 STEP 1874 H= 468:30 +CPU= 13.721
  14:43:36 STEP 1875 H= 468:45 +CPU= 13.882
  14:43:50 STEP 1876 H= 469:00 +CPU= 13.819
  14:44:03 STEP 1877 H= 469:15 +CPU= 13.621
  14:44:17 STEP 1878 H= 469:30 +CPU= 13.954
  14:44:31 STEP 1879 H= 469:45 +CPU= 14.120
  14:44:45 STEP 1880 H= 470:00 +CPU= 13.776
  14:44:59 STEP 1881 H= 470:15 +CPU= 13.294
  14:45:12 STEP 1882 H= 470:30 +CPU= 13.617
  14:45:26 STEP 1883 H= 470:45 +CPU= 13.885
  14:45:43 STEP 1884 H= 471:00 +CPU= 17.219
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGh7zg+001884
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHh7zg+001884
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAh7zg+001884
  14:45:57 STEP 1885 H= 471:15 +CPU= 13.535
  14:46:11 STEP 1886 H= 471:30 +CPU= 13.849
  14:46:24 STEP 1887 H= 471:45 +CPU= 13.510
  14:46:38 STEP 1888 H= 472:00 +CPU= 13.532
  14:46:52 STEP 1889 H= 472:15 +CPU= 13.757
  14:47:06 STEP 1890 H= 472:30 +CPU= 13.954
  14:47:19 STEP 1891 H= 472:45 +CPU= 13.737
  14:47:33 STEP 1892 H= 473:00 +CPU= 14.035
  14:47:47 STEP 1893 H= 473:15 +CPU= 14.131
  14:48:01 STEP 1894 H= 473:30 +CPU= 13.537
  14:48:15 STEP 1895 H= 473:45 +CPU= 13.785
  14:48:32 STEP 1896 H= 474:00 +CPU= 17.205
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGh7zg+001896
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHh7zg+001896
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAh7zg+001896
  14:48:46 STEP 1897 H= 474:15 +CPU= 14.142
  14:49:00 STEP 1898 H= 474:30 +CPU= 13.923
  14:49:14 STEP 1899 H= 474:45 +CPU= 13.711
  14:49:27 STEP 1900 H= 475:00 +CPU= 13.570
  14:49:42 STEP 1901 H= 475:15 +CPU= 14.019
  14:49:56 STEP 1902 H= 475:30 +CPU= 14.034
  14:50:09 STEP 1903 H= 475:45 +CPU= 13.495
  14:50:23 STEP 1904 H= 476:00 +CPU= 14.153
  14:50:37 STEP 1905 H= 476:15 +CPU= 13.983
(argv0) ../../projects/climateprediction.net/oifs_43r3_bl_1.13_x86_64-pc-linux-gnu
(argv1) start_date: 2016092300
(argv2) exptid: h7zg
(argv3) unique_member_id: a0bb
(argv4) batchid: 1018
(argv5) wuid: 12288734
(argv6) fclen: 20
(argv7) app_name: oifs_43r3_bl
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/3
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_bl
version: 1.13
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288734
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/3/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/3/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_b7b0c7247a2563748eb731abf3f66db4 to: /var/lib/boinc-client/slots/3/oifs_43r3_bl_a0bb_2016092300_20_1018_12288734.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/3/oifs_43r3_bl_a0bb_2016092300_20_1018_12288734.zip
ic_ancil_file: ic_ancil_12288734
ifsdata_file: ifsdata_12288734
climate_data_file: clim_data_12288734
horiz_resolution: 159
vert_resolution: 91
grid_type: l_2
upload_interval: 96
utstep: 900
nfrres: restart dump frequency (steps) 48
Copying IC ancils from: ../../projects/climateprediction.net/jf_f7ecfd27aa1e68bbe7e8d19800e25c73 to: /var/lib/boinc-client/slots/3/ic_ancil_12288734.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/3/ic_ancil_12288734.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_7a6f82b3685548fc744f5feddee81962 to: /var/lib/boinc-client/slots/3/ifsdata/ifsdata_12288734.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/3/ifsdata/ifsdata_12288734.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/3/159l_2/clim_data_12288734.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/3/159l_2/clim_data_12288734.zip
Checking for progress XML file: /var/lib/boinc-client/slots/3/progress_file_12288734.xml
Opened progress file ok : /var/lib/boinc-client/slots/3/progress_file_12288734.xml
-- Model is restarting --
Adjusting last_iter, 1906, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/3/progress_file_12288734.xml
last_cpu_time: 36629
upload_file_number: 19
last_iter: 1873
last_upload: 1641600
model_completed: 0
total_length_of_simulation: 1728000
result_base_name: oifs_43r3_bl_a0bb_2016092300_20_1018_12288734_1_r240171188
The child process has been launched with process id: 2805
Executing the command: /var/lib/boinc-client/slots/3/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:Rig-05:1:1:2805:2805] [20240708:082515:1720452315:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2091] fp = 0x314f440
[EC_DRHOOK:Rig-05:1:1:2805:2805] [20240708:082515:1720452315:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:Rig-05:1:1:2805:2805] [20240708:082515:1720452315:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:Rig-05:1:1:2805:2805] [20240708:082515:1720452315:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:Rig-05:1:1:2805:2805] [20240708:082515:1720452315:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:Rig-05:1:1:2805:2805] [20240708:082515:1720452315:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:Rig-05:1:1:2805:2805] [20240708:082515:1720452315:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:Rig-05:1:1:2805:2805] [20240708:082515:1720452315:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:Rig-05:1:1:2805:2805] [20240708:082515:1720452315:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:Rig-05:1:1:2805:2805] [20240708:082515:1720452315:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:Rig-05:1:1:2805:2805] [20240708:082515:1720452315:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:Rig-05:1:1:2805:2805] [20240708:082515:1720452315:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:Rig-05:1:1:2805:2805] [20240708:082515:1720452315:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:Rig-05:1:1:2805:2805] [20240708:082515:1720452315:0.000] [ignore_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:Rig-05:1:1:2805:2805] [20240708:082515:1720452315:0.000] [restore_default_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:Rig-05:1:1:2805:2805] [20240708:082515:1720452315:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:Rig-05:1:1:2805:2805] [20240708:082515:1720452315:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:Rig-05:1:1:2805:2805] [20240708:082515:1720452315:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:Rig-05:1:1:2805:2805] [20240708:082515:1720452315:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x820cf0 (old at (nil))
[EC_DRHOOK:Rig-05:1:1:2805:2805] [20240708:082515:1720452315:0.000] [trapfpe_treatment@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:Rig-05:1:1:2805:2805] [20240708:082515:1720452315:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:Rig-05:1:1:2805:2805] [20240708:082515:1720452315:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:Rig-05:1:1:2805:2805] [20240708:082515:1720452315:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x820cf0 (old at (nil))
[EC_DRHOOK:Rig-05:1:1:2805:2805] [20240708:082515:1720452315:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:Rig-05:1:1:2805:2805] [20240708:082515:1720452315:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:Rig-05:1:1:2805:2805] [20240708:082515:1720452315:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:Rig-05:1:1:2805:2805] [20240708:082515:1720452315:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x820cf0 (old at (nil))
[EC_DRHOOK:Rig-05:1:1:2805:2805] [20240708:082515:1720452315:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x820cf0 (old at (nil))
[EC_DRHOOK:Rig-05:1:1:2805:2805] [20240708:082515:1720452315:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:Rig-05:1:1:2805:2805] [20240708:082515:1720452315:0.000] [catch_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  08:25:32 STEP 1872 H= 468:00 +CPU= 14.815
  08:25:46 STEP 1873 H= 468:15 +CPU= 13.539
  08:26:00 STEP 1874 H= 468:30 +CPU= 13.890
  08:26:14 STEP 1875 H= 468:45 +CPU= 14.128
  08:26:28 STEP 1876 H= 469:00 +CPU= 13.921
  08:26:42 STEP 1877 H= 469:15 +CPU= 13.647
  08:26:55 STEP 1878 H= 469:30 +CPU= 13.651
  08:27:09 STEP 1879 H= 469:45 +CPU= 13.241
  08:27:22 STEP 1880 H= 470:00 +CPU= 13.882
  08:27:36 STEP 1881 H= 470:15 +CPU= 13.787
  08:27:50 STEP 1882 H= 470:30 +CPU= 13.414
  08:28:03 STEP 1883 H= 470:45 +CPU= 13.314
  08:28:20 STEP 1884 H= 471:00 +CPU= 17.046
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGh7zg+001884
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHh7zg+001884
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAh7zg+001884
  08:28:33 STEP 1885 H= 471:15 +CPU= 13.386
  08:28:48 STEP 1886 H= 471:30 +CPU= 13.994
  08:29:01 STEP 1887 H= 471:45 +CPU= 13.755
  08:29:15 STEP 1888 H= 472:00 +CPU= 13.642
  08:29:29 STEP 1889 H= 472:15 +CPU= 13.566
  08:29:42 STEP 1890 H= 472:30 +CPU= 13.626
  08:29:56 STEP 1891 H= 472:45 +CPU= 13.712
  08:30:10 STEP 1892 H= 473:00 +CPU= 14.183
(argv0) ../../projects/climateprediction.net/oifs_43r3_bl_1.13_x86_64-pc-linux-gnu
(argv1) start_date: 2016092300
(argv2) exptid: h7zg
(argv3) unique_member_id: a0bb
(argv4) batchid: 1018
(argv5) wuid: 12288734
(argv6) fclen: 20
(argv7) app_name: oifs_43r3_bl
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/3
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_bl
version: 1.13
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288734
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/3/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/3/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_b7b0c7247a2563748eb731abf3f66db4 to: /var/lib/boinc-client/slots/3/oifs_43r3_bl_a0bb_2016092300_20_1018_12288734.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/3/oifs_43r3_bl_a0bb_2016092300_20_1018_12288734.zip
ic_ancil_file: ic_ancil_12288734
ifsdata_file: ifsdata_12288734
climate_data_file: clim_data_12288734
horiz_resolution: 159
vert_resolution: 91
grid_type: l_2
upload_interval: 96
utstep: 900
nfrres: restart dump frequency (steps) 48
Copying IC ancils from: ../../projects/climateprediction.net/jf_f7ecfd27aa1e68bbe7e8d19800e25c73 to: /var/lib/boinc-client/slots/3/ic_ancil_12288734.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/3/ic_ancil_12288734.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_7a6f82b3685548fc744f5feddee81962 to: /var/lib/boinc-client/slots/3/ifsdata/ifsdata_12288734.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/3/ifsdata/ifsdata_12288734.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/3/159l_2/clim_data_12288734.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/3/159l_2/clim_data_12288734.zip
Checking for progress XML file: /var/lib/boinc-client/slots/3/progress_file_12288734.xml
Opened progress file ok : /var/lib/boinc-client/slots/3/progress_file_12288734.xml
-- Model is restarting --
Adjusting last_iter, 1893, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/3/progress_file_12288734.xml
last_cpu_time: 36927
upload_file_number: 19
last_iter: 1873
last_upload: 1641600
model_completed: 0
total_length_of_simulation: 1728000
result_base_name: oifs_43r3_bl_a0bb_2016092300_20_1018_12288734_1_r240171188
The child process has been launched with process id: 2884
Executing the command: /var/lib/boinc-client/slots/3/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:Rig-05:1:1:2884:2884] [20240708:083110:1720452670:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2091] fp = 0x314f440
[EC_DRHOOK:Rig-05:1:1:2884:2884] [20240708:083110:1720452670:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:Rig-05:1:1:2884:2884] [20240708:083110:1720452670:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:Rig-05:1:1:2884:2884] [20240708:083110:1720452670:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:Rig-05:1:1:2884:2884] [20240708:083110:1720452670:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:Rig-05:1:1:2884:2884] [20240708:083110:1720452670:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:Rig-05:1:1:2884:2884] [20240708:083110:1720452670:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:Rig-05:1:1:2884:2884] [20240708:083110:1720452670:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:Rig-05:1:1:2884:2884] [20240708:083110:1720452670:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:Rig-05:1:1:2884:2884] [20240708:083110:1720452670:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:Rig-05:1:1:2884:2884] [20240708:083110:1720452670:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:Rig-05:1:1:2884:2884] [20240708:083110:1720452670:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:Rig-05:1:1:2884:2884] [20240708:083110:1720452670:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:Rig-05:1:1:2884:2884] [20240708:083110:1720452670:0.000] [ignore_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:Rig-05:1:1:2884:2884] [20240708:083110:1720452670:0.000] [restore_default_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:Rig-05:1:1:2884:2884] [20240708:083110:1720452670:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:Rig-05:1:1:2884:2884] [20240708:083110:1720452670:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:Rig-05:1:1:2884:2884] [20240708:083110:1720452670:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:Rig-05:1:1:2884:2884] [20240708:083110:1720452670:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x820cf0 (old at (nil))
[EC_DRHOOK:Rig-05:1:1:2884:2884] [20240708:083110:1720452670:0.000] [trapfpe_treatment@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:Rig-05:1:1:2884:2884] [20240708:083110:1720452670:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:Rig-05:1:1:2884:2884] [20240708:083110:1720452670:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:Rig-05:1:1:2884:2884] [20240708:083110:1720452670:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x820cf0 (old at (nil))
[EC_DRHOOK:Rig-05:1:1:2884:2884] [20240708:083110:1720452670:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:Rig-05:1:1:2884:2884] [20240708:083110:1720452670:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:Rig-05:1:1:2884:2884] [20240708:083110:1720452670:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:Rig-05:1:1:2884:2884] [20240708:083110:1720452670:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x820cf0 (old at (nil))
[EC_DRHOOK:Rig-05:1:1:2884:2884] [20240708:083110:1720452670:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x820cf0 (old at (nil))
[EC_DRHOOK:Rig-05:1:1:2884:2884] [20240708:083110:1720452670:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:Rig-05:1:1:2884:2884] [20240708:083110:1720452670:0.000] [catch_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  08:31:27 STEP 1872 H= 468:00 +CPU= 14.436
  08:31:41 STEP 1873 H= 468:15 +CPU= 13.713
  08:31:55 STEP 1874 H= 468:30 +CPU= 13.902
  08:32:08 STEP 1875 H= 468:45 +CPU= 13.323
  08:32:22 STEP 1876 H= 469:00 +CPU= 13.919
  08:32:36 STEP 1877 H= 469:15 +CPU= 13.859
  08:32:49 STEP 1878 H= 469:30 +CPU= 13.103
  08:33:03 STEP 1879 H= 469:45 +CPU= 13.689
  08:33:16 STEP 1880 H= 470:00 +CPU= 13.532
  08:33:29 STEP 1881 H= 470:15 +CPU= 13.262
  08:33:43 STEP 1882 H= 470:30 +CPU= 13.507
  08:33:57 STEP 1883 H= 470:45 +CPU= 13.587
  08:34:14 STEP 1884 H= 471:00 +CPU= 16.952
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGh7zg+001884
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHh7zg+001884
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAh7zg+001884
  08:34:27 STEP 1885 H= 471:15 +CPU= 13.908
  08:34:41 STEP 1886 H= 471:30 +CPU= 13.830
  08:34:55 STEP 1887 H= 471:45 +CPU= 13.575
  08:35:09 STEP 1888 H= 472:00 +CPU= 14.101
  08:35:23 STEP 1889 H= 472:15 +CPU= 13.855
  08:35:36 STEP 1890 H= 472:30 +CPU= 13.309
  08:35:50 STEP 1891 H= 472:45 +CPU= 13.817
  08:36:04 STEP 1892 H= 473:00 +CPU= 13.708
  08:36:17 STEP 1893 H= 473:15 +CPU= 13.364
  08:36:31 STEP 1894 H= 473:30 +CPU= 13.646
  08:36:45 STEP 1895 H= 473:45 +CPU= 13.681
  08:37:02 STEP 1896 H= 474:00 +CPU= 17.337
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGh7zg+001896
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHh7zg+001896
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAh7zg+001896
  08:37:16 STEP 1897 H= 474:15 +CPU= 13.779
  08:37:30 STEP 1898 H= 474:30 +CPU= 13.842
  08:37:43 STEP 1899 H= 474:45 +CPU= 13.424
  08:37:57 STEP 1900 H= 475:00 +CPU= 13.918
  08:38:10 STEP 1901 H= 475:15 +CPU= 13.325
  08:38:24 STEP 1902 H= 475:30 +CPU= 13.275
  08:38:37 STEP 1903 H= 475:45 +CPU= 13.233
  08:38:51 STEP 1904 H= 476:00 +CPU= 13.732
  08:39:04 STEP 1905 H= 476:15 +CPU= 13.552
  08:39:18 STEP 1906 H= 476:30 +CPU= 13.861
  08:39:32 STEP 1907 H= 476:45 +CPU= 13.620
  08:39:49 STEP 1908 H= 477:00 +CPU= 16.936
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGh7zg+001908
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHh7zg+001908
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAh7zg+001908
  08:40:02 STEP 1909 H= 477:15 +CPU= 13.667
  08:40:16 STEP 1910 H= 477:30 +CPU= 13.321
  08:40:29 STEP 1911 H= 477:45 +CPU= 13.333
  08:40:43 STEP 1912 H= 478:00 +CPU= 13.744
  08:40:56 STEP 1913 H= 478:15 +CPU= 13.688
  08:41:10 STEP 1914 H= 478:30 +CPU= 13.542
  08:41:24 STEP 1915 H= 478:45 +CPU= 13.779
  08:41:38 STEP 1916 H= 479:00 +CPU= 13.920
  08:41:51 STEP 1917 H= 479:15 +CPU= 13.289
  08:42:05 STEP 1918 H= 479:30 +CPU= 13.998
  08:42:19 STEP 1919 H= 479:45 +CPU= 13.875
  08:42:37 STEP 1920 H= 480:00 +CPU= 17.655
..The child process terminated with status: 0
>>> Printing last 70 lines from file: NODE.001_01

 
DDH------------------------------------------------------------------------
I                                                                         I
I TIME =  480.000H DDH * Diagnostic Var.,     AREA                GLOBAL  I
I                                                                         I
I-------------------------------------------------------------------------I
I   VARIABLE   I      Initial value         I        Current value        I
I-------------------------------------------------------------------------I
I  OMEGA (P/S) I      0.1200450697E-21      I      -0.3893251085E-05      I
ICloud fractionI      0.0000000000E+00      I       0.5437192995E-01      I
I  Relat. Hum. I      0.4088077437E+00      I       0.4369456411E+00      I
I  PASS 01     I      0.0000000000E+00      I       0.8305158676E+03      I
DDH------------------------------------------------------------------------
  
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
 NSTEP =  1920 SCAN2M_HPOS  P
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ICMGGh7zg+001920 MODE=a
 IO-STREAM CLOSED - ICMGGh7zg+001920
 MPI-TASK:   1 -    309731460 BYTES IN      7 RECORDS TRANSFERRED IN    0.0002 SECONDS******** Mbytes/s, TOTAL TIME=    0.0051( 3.9%)
 08:42:37 STEP 1920 H= 480:00 +CPU= 17.655
 END CNT3
 NSTEP =  1920 CNT0   000000000  13.001  13.001  13.024   0.998               0               0               0               0               0
IO-STREAM STATISTICS, TOTAL NO OF REC/BYTES READ           46/  1069125708 WRITTEN         1430/************
 -TASK-OPENED-OPEN-RECS IN -KBYTE IN    -RECS OUT-KBYTE OUT-WALL   -WALL IN-WALL
  OU-TOT IN -TOT OUT
   1     28    0       46      1044068     1430 ************     2.5     1.4     1.1     1.8     2.5
===-=== START OF TIMING STATISTICS ===-===
 
STATS FOR ALL TASKS
 NUM ROUTINE                                     CALLS  MEAN(ms)   MAX(ms)   FRAC(%)  UNBAL(%)
   0 CNT0     - COMPLETE EXECUTION                   1  686481.9  686481.9    100.00      0.00
   1 CNT4     - FORWARD INTEGRATION                  1  684048.1  684048.1     99.65      0.00
   8 SCAN2M - GRID-POINT DYNAMICS                   53    2008.4    2008.4     15.51      0.00
   9 SPCM     - SPECTRAL COMP.                      48     360.3     360.3      2.52      0.00
  10 SCAN2M - PHYSICS                               49    6916.1    6916.1     49.37      0.00
  11 IOPACK   - OUTPUT P.P. RESULTS                  4     387.6     387.6      0.23      0.00
  12 SPNORM   - SPECTRAL NORM COMP.                  3      22.3      22.3      0.01      0.00
  17 GRIDFPOS IN CNT4                                4      12.7      12.7      0.01      0.00
  27 RESTART FILES - WRITING                         1    1550.5    1550.5      0.23      0.00
  28 RESTART FILES - READING                         1    2164.9    2164.9      0.32      0.00
  29 SU4FPOS IN CNT4                                 4       0.0       0.0      0.00      0.00
  30 DYNFPOS IN CNT4                                 4    2917.0    2917.0      1.70      0.00
  31 POSDDH IN STEPO                                 1       0.4       0.4      0.00      0.00
  37 CPGLAG   - SL COMPUTATIONS                     49    2988.2    2988.2     21.33      0.00
  39 SU0YOMB                                         1     181.4     181.4      0.03      0.00
  51 SCAN2M   - SL COMM. PART 1                     49      16.1      16.1      0.12      0.00
  54 SPCM     - M TO S/S TO M TRANSP.               48     127.1     127.1      0.89      0.00
  55 SPCIMPF  - S TO M/M TO S TRANSP.               48      41.3      41.3      0.29      0.00
  56 SPNORM   - SPECTRAL NORM COMM.                  3       0.1       0.1      0.00      0.00
 102 LTINV_CTL   - INVERSE LEGENDRE TRANSFORM      108     216.2     216.2      3.40      0.00
 103 LTDIR_CTL   - DIRECT LEGENDRE TRANSFORM       100     117.9     117.9      1.72      0.00
 106 FTDIR_CTL   - DIRECT FOURIER TRANSFORM        100      38.3      38.3      0.56      0.00
 107 FTINV_CTL   - INVERSE FOURIER TRANSFORM       108      65.2      65.2      1.03      0.00
 140 SULEG       - COMP. OF LEGENDRE POL.            1      19.6      19.6      0.00      0.00
 152 LTINV_CTL   - M TO L TRANSPOSITION            108      25.2      25.2      0.40      0.00
 153 LTDIR_CTL   - L TO M TRANSPOSITION            100      14.7      14.7      0.21      0.00
 157 FTINV_CTL   - L TO G TRANSPOSITION            108      25.2      25.2      0.40      0.00
 158 FTDIR_CTL   - G TO L TRANSPOSITION            100      11.0      11.0      0.16      0.00
 400 GSTATS                                       6667       0.0       0.0      0.00      0.00
 401 GSTATS HOOK                                  6149       0.0       0.0      0.00      0.00
TOTAL MEASURED IMBALANCE =       0.0 SECONDS,  0.0 PERCENT
TOTAL WALLCLOCK TIME      686.5 CPU TIME     685.4 VECTOR TIME      685.4


===-=== END   OF TIMING STATISTICS ===-===
FORECAST DAYS PER DAY     2526.1
  *** END CNT0 *** 
------------------------------------------------
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGh7zg+001920
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHh7zg+001920
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAh7zg+001920
Adding to the zip: /var/lib/boinc-client/slots/3/NODE.001_01
Adding to the zip: /var/lib/boinc-client/slots/3/ifs.stat
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288734/ICMSHh7zg+001908
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288734/ICMSHh7zg+001860
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288734/ICMUAh7zg+001872
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288734/ICMSHh7zg+001884
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288734/ICMSHh7zg+001872
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288734/ICMSHh7zg+001896
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288734/ICMGGh7zg+001896
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288734/ICMGGh7zg+001860
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288734/ICMGGh7zg+001872
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288734/ICMSHh7zg+001920
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288734/ICMUAh7zg+001824
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288734/ICMUAh7zg+001884
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288734/ICMUAh7zg+001896
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288734/ICMSHh7zg+001848
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288734/ICMUAh7zg+001836
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288734/ICMGGh7zg+001884
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288734/ICMUAh7zg+001908
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288734/ICMGGh7zg+001848
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288734/ICMGGh7zg+001824
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288734/ICMSHh7zg+001836
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288734/ICMGGh7zg+001920
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288734/ICMUAh7zg+001920
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288734/ICMUAh7zg+001860
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288734/ICMUAh7zg+001848
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288734/ICMGGh7zg+001908
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288734/ICMSHh7zg+001824
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288734/ICMGGh7zg+001836
Zipping up the final file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a0bb_2016092300_20_1018_12288734_1_r240171188_19.zip
Uploading the final file: upload_file_19.zip
Uploading trickle at timestep: 1727100
08:46:05 (2775): called boinc_finish(0)

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
17 Jul 2024 15:10:38 1541000 22449782 oifs_43r3_bl_a0bb_2016092300_20_1018_12288734_1 1,727,100 37,612 0.0218


©2024 cpdn.org