climateprediction.net home page
Task 22444687

Task 22444687

Name oifs_43r3_bl_a24q_2016092300_20_1018_12291089_0
Workunit 12291089
Created 13 Jun 2024, 0:02:22 UTC
Sent 13 Jun 2024, 18:12:09 UTC
Report deadline 12 Aug 2024, 18:12:09 UTC
Received 18 Jun 2024, 5:50:24 UTC
Server state Over
Outcome Success
Client state Done
Exit status 0 (0x00000000)
Computer ID 1469580
Run time 12 hours 57 min 20 sec
CPU time 12 hours 41 min 31 sec
Validate state Valid
Credit 2,635.55
Device peak FLOPS 5.26 GFLOPS
Application version OpenIFS 43r3 Baroclinic Lifecycle v1.13
x86_64-pc-linux-gnu
Peak working set size 5,003.20 MB
Peak swap size 5,981.13 MB
Peak disk usage 1,379.22 MB
Stderr
<core_client_version>7.18.1</core_client_version>
<![CDATA[
<stderr_txt>
port/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:server242:1:1:25603:25603] [20240618:044143:1718678503:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x820cf0 (old at (nil))
[EC_DRHOOK:server242:1:1:25603:25603] [20240618:044143:1718678503:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x820cf0 (old at (nil))
[EC_DRHOOK:server242:1:1:25603:25603] [20240618:044143:1718678503:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:server242:1:1:25603:25603] [20240618:044143:1718678503:0.001] [catch_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMGGh7zg+001452
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMSHh7zg+001452
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMUAh7zg+001452
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMGGh7zg+001464
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMSHh7zg+001464
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMUAh7zg+001464
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMGGh7zg+001476
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMSHh7zg+001476
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMUAh7zg+001476
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMGGh7zg+001488
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMSHh7zg+001488
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMUAh7zg+001488
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMGGh7zg+001500
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMSHh7zg+001500
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMUAh7zg+001500
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMGGh7zg+001512
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMSHh7zg+001512
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMUAh7zg+001512
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMGGh7zg+001524
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMSHh7zg+001524
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMUAh7zg+001524
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a24q_2016092300_20_1018_12291089_0_r934871342_15.zip
Uploading the intermediate file: upload_file_15.zip
  04:42:10 STEP 1536 H= 384:00 +CPU= 23.498
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMGGh7zg+001536
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMSHh7zg+001536
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMUAh7zg+001536
  04:42:33 STEP 1537 H= 384:15 +CPU= 23.079
  04:42:56 STEP 1538 H= 384:30 +CPU= 22.454
  04:43:20 STEP 1539 H= 384:45 +CPU= 24.099
  04:43:43 STEP 1540 H= 385:00 +CPU= 22.808
  04:44:06 STEP 1541 H= 385:15 +CPU= 23.019
  04:44:28 STEP 1542 H= 385:30 +CPU= 22.806
  04:44:51 STEP 1543 H= 385:45 +CPU= 22.886
  04:45:14 STEP 1544 H= 386:00 +CPU= 22.778
  04:45:38 STEP 1545 H= 386:15 +CPU= 23.751
  04:46:02 STEP 1546 H= 386:30 +CPU= 24.014
  04:46:25 STEP 1547 H= 386:45 +CPU= 23.242
  04:46:53 STEP 1548 H= 387:00 +CPU= 27.826
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMGGh7zg+001548
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMSHh7zg+001548
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMUAh7zg+001548
  04:47:17 STEP 1549 H= 387:15 +CPU= 23.516
  04:47:40 STEP 1550 H= 387:30 +CPU= 23.195
  04:48:03 STEP 1551 H= 387:45 +CPU= 22.437
  04:48:26 STEP 1552 H= 388:00 +CPU= 23.260
  04:48:49 STEP 1553 H= 388:15 +CPU= 22.797
  04:49:13 STEP 1554 H= 388:30 +CPU= 24.397
  04:49:36 STEP 1555 H= 388:45 +CPU= 22.522
  04:49:58 STEP 1556 H= 389:00 +CPU= 22.599
  04:50:24 STEP 1557 H= 389:15 +CPU= 25.812
  04:50:49 STEP 1558 H= 389:30 +CPU= 24.334
  04:51:11 STEP 1559 H= 389:45 +CPU= 22.123
  04:51:39 STEP 1560 H= 390:00 +CPU= 27.682
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMGGh7zg+001560
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMSHh7zg+001560
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMUAh7zg+001560
  04:52:03 STEP 1561 H= 390:15 +CPU= 24.788
  04:52:28 STEP 1562 H= 390:30 +CPU= 23.636
  04:52:50 STEP 1563 H= 390:45 +CPU= 22.080
  04:53:13 STEP 1564 H= 391:00 +CPU= 23.426
  04:53:37 STEP 1565 H= 391:15 +CPU= 23.524
  04:53:59 STEP 1566 H= 391:30 +CPU= 22.373
  04:54:21 STEP 1567 H= 391:45 +CPU= 21.789
  04:54:45 STEP 1568 H= 392:00 +CPU= 24.228
  04:55:10 STEP 1569 H= 392:15 +CPU= 25.197
  04:55:34 STEP 1570 H= 392:30 +CPU= 23.335
  04:55:56 STEP 1571 H= 392:45 +CPU= 21.978
  04:56:25 STEP 1572 H= 393:00 +CPU= 29.532
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMGGh7zg+001572
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMSHh7zg+001572
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMUAh7zg+001572
  04:56:49 STEP 1573 H= 393:15 +CPU= 23.431
  04:57:11 STEP 1574 H= 393:30 +CPU= 22.430
  04:57:33 STEP 1575 H= 393:45 +CPU= 21.853
  04:57:57 STEP 1576 H= 394:00 +CPU= 23.265
  04:58:21 STEP 1577 H= 394:15 +CPU= 24.197
  04:58:43 STEP 1578 H= 394:30 +CPU= 22.497
  04:59:06 STEP 1579 H= 394:45 +CPU= 22.949
  04:59:30 STEP 1580 H= 395:00 +CPU= 23.900
  04:59:52 STEP 1581 H= 395:15 +CPU= 21.947
  05:00:16 STEP 1582 H= 395:30 +CPU= 23.246
  05:00:42 STEP 1583 H= 395:45 +CPU= 25.970
  05:01:16 STEP 1584 H= 396:00 +CPU= 33.376
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMGGh7zg+001584
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMSHh7zg+001584
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMUAh7zg+001584
  05:01:39 STEP 1585 H= 396:15 +CPU= 22.231
  05:02:02 STEP 1586 H= 396:30 +CPU= 23.589
  05:02:25 STEP 1587 H= 396:45 +CPU= 23.250
  05:02:48 STEP 1588 H= 397:00 +CPU= 22.539
  05:03:11 STEP 1589 H= 397:15 +CPU= 22.827
  05:03:36 STEP 1590 H= 397:30 +CPU= 25.233
  05:04:00 STEP 1591 H= 397:45 +CPU= 23.643
  05:04:23 STEP 1592 H= 398:00 +CPU= 22.539
  05:04:45 STEP 1593 H= 398:15 +CPU= 22.533
  05:05:10 STEP 1594 H= 398:30 +CPU= 24.807
  05:05:34 STEP 1595 H= 398:45 +CPU= 23.943
  05:06:01 STEP 1596 H= 399:00 +CPU= 26.760
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMGGh7zg+001596
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMSHh7zg+001596
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMUAh7zg+001596
  05:06:22 STEP 1597 H= 399:15 +CPU= 21.096
  05:06:46 STEP 1598 H= 399:30 +CPU= 24.033
  05:07:09 STEP 1599 H= 399:45 +CPU= 22.765
  05:07:32 STEP 1600 H= 400:00 +CPU= 22.805
  05:07:55 STEP 1601 H= 400:15 +CPU= 23.112
  05:08:22 STEP 1602 H= 400:30 +CPU= 25.414
  05:08:45 STEP 1603 H= 400:45 +CPU= 22.202
  05:09:07 STEP 1604 H= 401:00 +CPU= 22.746
  05:09:31 STEP 1605 H= 401:15 +CPU= 23.733
  05:09:55 STEP 1606 H= 401:30 +CPU= 23.725
  05:10:18 STEP 1607 H= 401:45 +CPU= 22.586
  05:10:50 STEP 1608 H= 402:00 +CPU= 28.778
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMGGh7zg+001608
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMSHh7zg+001608
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMUAh7zg+001608
  05:11:15 STEP 1609 H= 402:15 +CPU= 25.735
  05:11:38 STEP 1610 H= 402:30 +CPU= 22.804
  05:12:00 STEP 1611 H= 402:45 +CPU= 21.997
  05:12:23 STEP 1612 H= 403:00 +CPU= 23.261
  05:12:48 STEP 1613 H= 403:15 +CPU= 24.429
  05:13:10 STEP 1614 H= 403:30 +CPU= 22.086
  05:13:32 STEP 1615 H= 403:45 +CPU= 21.587
  05:13:56 STEP 1616 H= 404:00 +CPU= 24.521
  05:14:21 STEP 1617 H= 404:15 +CPU= 24.745
  05:14:43 STEP 1618 H= 404:30 +CPU= 22.001
  05:15:05 STEP 1619 H= 404:45 +CPU= 22.127
  05:15:36 STEP 1620 H= 405:00 +CPU= 31.131
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMGGh7zg+001620
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMSHh7zg+001620
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMUAh7zg+001620
  05:16:00 STEP 1621 H= 405:15 +CPU= 23.211
  05:16:23 STEP 1622 H= 405:30 +CPU= 22.999
  05:16:45 STEP 1623 H= 405:45 +CPU= 22.078
  05:17:08 STEP 1624 H= 406:00 +CPU= 22.914
  05:17:31 STEP 1625 H= 406:15 +CPU= 22.979
  05:17:54 STEP 1626 H= 406:30 +CPU= 23.434
  05:18:17 STEP 1627 H= 406:45 +CPU= 22.443
  05:18:40 STEP 1628 H= 407:00 +CPU= 22.687
  05:19:03 STEP 1629 H= 407:15 +CPU= 23.017
  05:19:28 STEP 1630 H= 407:30 +CPU= 24.808
  05:19:50 STEP 1631 H= 407:45 +CPU= 21.923
  05:20:24 STEP 1632 H= 408:00 +CPU= 32.477
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMGGh7zg+001632
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMSHh7zg+001632
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMUAh7zg+001632
  05:20:50 STEP 1633 H= 408:15 +CPU= 26.308
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMGGh7zg+001548
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMSHh7zg+001548
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMUAh7zg+001548
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMGGh7zg+001560
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMSHh7zg+001560
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMUAh7zg+001560
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMGGh7zg+001572
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMSHh7zg+001572
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMUAh7zg+001572
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMGGh7zg+001584
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMSHh7zg+001584
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMUAh7zg+001584
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMGGh7zg+001596
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMSHh7zg+001596
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMUAh7zg+001596
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMGGh7zg+001608
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMSHh7zg+001608
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMUAh7zg+001608
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMGGh7zg+001620
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMSHh7zg+001620
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMUAh7zg+001620
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMGGh7zg+001632
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMSHh7zg+001632
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMUAh7zg+001632
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a24q_2016092300_20_1018_12291089_0_r934871342_16.zip
Uploading the intermediate file: upload_file_16.zip
  05:21:15 STEP 1634 H= 408:30 +CPU= 24.204
  05:21:37 STEP 1635 H= 408:45 +CPU= 22.453
  05:22:01 STEP 1636 H= 409:00 +CPU= 23.482
  05:22:27 STEP 1637 H= 409:15 +CPU= 25.701
  05:22:49 STEP 1638 H= 409:30 +CPU= 22.256
  05:23:11 STEP 1639 H= 409:45 +CPU= 22.295
  05:23:36 STEP 1640 H= 410:00 +CPU= 25.283
  05:24:01 STEP 1641 H= 410:15 +CPU= 24.040
  05:24:23 STEP 1642 H= 410:30 +CPU= 22.040
  05:24:45 STEP 1643 H= 410:45 +CPU= 22.124
  05:25:17 STEP 1644 H= 411:00 +CPU= 31.564
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMGGh7zg+001644
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMSHh7zg+001644
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMUAh7zg+001644
  05:25:40 STEP 1645 H= 411:15 +CPU= 23.306
  05:26:02 STEP 1646 H= 411:30 +CPU= 21.926
  05:26:24 STEP 1647 H= 411:45 +CPU= 22.111
  05:26:50 STEP 1648 H= 412:00 +CPU= 25.508
  05:27:12 STEP 1649 H= 412:15 +CPU= 22.562
  05:27:35 STEP 1650 H= 412:30 +CPU= 22.167
  05:27:57 STEP 1651 H= 412:45 +CPU= 22.421
  05:28:23 STEP 1652 H= 413:00 +CPU= 25.784
  05:28:45 STEP 1653 H= 413:15 +CPU= 21.674
  05:29:07 STEP 1654 H= 413:30 +CPU= 22.257
  05:29:31 STEP 1655 H= 413:45 +CPU= 23.980
  05:30:02 STEP 1656 H= 414:00 +CPU= 30.182
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMGGh7zg+001656
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMSHh7zg+001656
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMUAh7zg+001656
  05:30:25 STEP 1657 H= 414:15 +CPU= 22.866
  05:30:48 STEP 1658 H= 414:30 +CPU= 23.585
  05:31:12 STEP 1659 H= 414:45 +CPU= 23.366
  05:31:36 STEP 1660 H= 415:00 +CPU= 24.734
  05:31:59 STEP 1661 H= 415:15 +CPU= 22.583
  05:32:23 STEP 1662 H= 415:30 +CPU= 23.573
  05:32:46 STEP 1663 H= 415:45 +CPU= 23.032
  05:33:09 STEP 1664 H= 416:00 +CPU= 22.908
  05:33:31 STEP 1665 H= 416:15 +CPU= 22.788
  05:33:55 STEP 1666 H= 416:30 +CPU= 23.228
  05:34:17 STEP 1667 H= 416:45 +CPU= 22.295
  05:34:48 STEP 1668 H= 417:00 +CPU= 27.023
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMGGh7zg+001668
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMSHh7zg+001668
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMUAh7zg+001668
  05:35:12 STEP 1669 H= 417:15 +CPU= 24.432
  05:35:39 STEP 1670 H= 417:30 +CPU= 26.222
  05:36:01 STEP 1671 H= 417:45 +CPU= 22.302
  05:36:23 STEP 1672 H= 418:00 +CPU= 22.193
  05:36:46 STEP 1673 H= 418:15 +CPU= 23.118
  05:37:18 STEP 1674 H= 418:30 +CPU= 25.759
  05:37:41 STEP 1675 H= 418:45 +CPU= 22.528
  05:38:04 STEP 1676 H= 419:00 +CPU= 23.196
  05:38:28 STEP 1677 H= 419:15 +CPU= 23.579
  05:38:53 STEP 1678 H= 419:30 +CPU= 24.689
  05:39:16 STEP 1679 H= 419:45 +CPU= 23.863
  05:40:02 STEP 1680 H= 420:00 +CPU= 34.833
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMGGh7zg+001680
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMSHh7zg+001680
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMUAh7zg+001680
  05:40:27 STEP 1681 H= 420:15 +CPU= 24.483
  05:40:49 STEP 1682 H= 420:30 +CPU= 22.482
  05:41:13 STEP 1683 H= 420:45 +CPU= 23.586
  05:41:38 STEP 1684 H= 421:00 +CPU= 24.707
  05:42:00 STEP 1685 H= 421:15 +CPU= 22.904
  05:42:23 STEP 1686 H= 421:30 +CPU= 22.114
  05:42:46 STEP 1687 H= 421:45 +CPU= 23.020
  05:43:11 STEP 1688 H= 422:00 +CPU= 25.357
  05:43:34 STEP 1689 H= 422:15 +CPU= 22.894
  05:43:56 STEP 1690 H= 422:30 +CPU= 21.705
  05:44:18 STEP 1691 H= 422:45 +CPU= 22.600
  05:44:49 STEP 1692 H= 423:00 +CPU= 30.166
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMGGh7zg+001692
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMSHh7zg+001692
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMUAh7zg+001692
  05:45:12 STEP 1693 H= 423:15 +CPU= 23.074
  05:45:34 STEP 1694 H= 423:30 +CPU= 22.252
  05:45:58 STEP 1695 H= 423:45 +CPU= 23.428
  05:46:22 STEP 1696 H= 424:00 +CPU= 24.235
  05:46:46 STEP 1697 H= 424:15 +CPU= 24.079
  05:47:09 STEP 1698 H= 424:30 +CPU= 22.477
  05:47:32 STEP 1699 H= 424:45 +CPU= 22.792
  05:47:55 STEP 1700 H= 425:00 +CPU= 23.541
  05:48:19 STEP 1701 H= 425:15 +CPU= 23.445
  05:48:41 STEP 1702 H= 425:30 +CPU= 22.742
  05:49:04 STEP 1703 H= 425:45 +CPU= 22.331
  05:49:35 STEP 1704 H= 426:00 +CPU= 31.651
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMGGh7zg+001704
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMSHh7zg+001704
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMUAh7zg+001704
  05:49:58 STEP 1705 H= 426:15 +CPU= 22.459
  05:50:21 STEP 1706 H= 426:30 +CPU= 23.074
  05:50:44 STEP 1707 H= 426:45 +CPU= 22.854
  05:51:07 STEP 1708 H= 427:00 +CPU= 23.088
  05:51:30 STEP 1709 H= 427:15 +CPU= 22.436
  05:51:52 STEP 1710 H= 427:30 +CPU= 22.171
  05:52:15 STEP 1711 H= 427:45 +CPU= 23.486
  05:52:39 STEP 1712 H= 428:00 +CPU= 23.810
  05:53:01 STEP 1713 H= 428:15 +CPU= 21.870
  05:53:23 STEP 1714 H= 428:30 +CPU= 22.235
  05:53:47 STEP 1715 H= 428:45 +CPU= 23.234
  05:54:15 STEP 1716 H= 429:00 +CPU= 28.527
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMGGh7zg+001716
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMSHh7zg+001716
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMUAh7zg+001716
  05:54:37 STEP 1717 H= 429:15 +CPU= 21.514
  05:54:59 STEP 1718 H= 429:30 +CPU= 22.079
  05:55:22 STEP 1719 H= 429:45 +CPU= 23.340
  05:55:47 STEP 1720 H= 430:00 +CPU= 24.392
  05:56:09 STEP 1721 H= 430:15 +CPU= 22.235
  05:56:31 STEP 1722 H= 430:30 +CPU= 22.426
  05:56:54 STEP 1723 H= 430:45 +CPU= 22.632
  05:57:18 STEP 1724 H= 431:00 +CPU= 23.820
  05:57:40 STEP 1725 H= 431:15 +CPU= 22.231
  05:58:02 STEP 1726 H= 431:30 +CPU= 22.080
  05:58:25 STEP 1727 H= 431:45 +CPU= 22.894
Quit request received from BOINC client, ending the child process
>>> Printing last 70 lines from file: NODE.001_01

  DATE= 2016   10    9
 NORMS AT NSTEP CNT4 1728
 SPECTRAL NORMS -   LOG(PREHYDS)    0.115262794496759E+02
 LEV     VORTICITY             DIVERGENCE           TEMPERATURE           KINETIC ENERGY    
 AVE 0.200970345654421E-04 0.508620993330194E-05 0.200571506753830E+03 0.136077315756747E+03
 GPNORM  HUMIDITY            AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.158737313808625E-02 0.999999999999995E-08 0.185609161637056E-01
 GPNORM  LIQUID WATER        AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.189726499806212E-05 0.000000000000000E+00 0.702126714131181E-03
 GPNORM  ICE WATER           AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.491583525511561E-06 0.000000000000000E+00 0.312344826879632E-03
 GPNORM SNOW                 AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.949318502413943E-06 0.000000000000000E+00 0.747017188896297E-03
 GPNORM RAIN                 AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.908342898312118E-07 0.000000000000000E+00 0.269544230738689E-03
 GPNORM  CLOUD FRACTION      AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.191422182026661E+00 0.000000000000000E+00 0.100000000000000E+01
 GPNORM CLW                  AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.189807951188231E-05 0.000000000000000E+00 0.701438355356645E-03
 GPNORM CIW                  AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.490443730647636E-06 0.000000000000000E+00 0.315924254140726E-03
 GPNORM CC                   AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.191401678273321E+00 0.000000000000000E+00 0.100000000000000E+01
 GPNORM RFL                  AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.408369887755768E-06 0.000000000000000E+00 0.122871860317810E-02
 GPNORM SFL                  AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.107331091650871E-05 0.000000000000000E+00 0.709598675927958E-03
 GPNORM CRFL                 AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.375118170965434E-06 0.000000000000000E+00 0.157069600383503E-02
 GPNORM CSFL                 AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.108279891889860E-06 0.000000000000000E+00 0.124525128051130E-02
 GPNORM PFRC                 AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.893567719346749E-01 0.000000000000000E+00 0.100000000000000E+01
 NSTEP =  1728 STEPO   F00000000
 CREATE RESTART FILE ./srf00180000.0001                                                                                                                                                                                                                                              
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ./srf00180000.0001 MODE=w
 IO-STREAM CLOSED - ./srf00180000.0001
 MPI-TASK:   1 - ************ BYTES IN     46 RECORDS TRANSFERRED IN    3.9860 SECONDS******** Mbytes/s, TOTAL TIME=    4.6805(85.2%)
 CREATE RESTART CONTROL FILE ./rcf                                                                                                                   
 WRRESF: DELETED PREVIOUS RESTART FILE OK: ./srf00171200.0001
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
 NSTEP =  1728 SCAN2M_HPOS  P
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ICMGGh7zg+001728 MODE=w
 IO-STREAM CLOSED - ICMGGh7zg+001728
 MPI-TASK:   1 -    310826308 BYTES IN      6 RECORDS TRANSFERRED IN    0.0007 SECONDS444037.6 Mbytes/s, TOTAL TIME=    0.0107( 6.5%)
 NSTEP =  1728 STEPO   0AA000000
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ./ICMSHh7zg+001728 MODE=w
 IO-STREAM CLOSED - ./ICMSHh7zg+001728
 MPI-TASK:   1 -    707399250 BYTES IN    202 RECORDS TRANSFERRED IN    0.5719 SECONDS  1236.9 Mbytes/s, TOTAL TIME=    0.8138(70.3%)
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ./ICMUAh7zg+001728 MODE=w
 IO-STREAM CLOSED - ./ICMUAh7zg+001728
 MPI-TASK:   1 -   4973220928 BYTES IN    125 RECORDS TRANSFERRED IN    0.0106 SECONDS469171.8 Mbytes/s, TOTAL TIME=    3.1893( 0.3%)
 STEPO_FPOS WAS : B00000F00E
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ./ICMUAh7zg+001728 MODE=a
 IO-STREAM CLOSED - ./ICMUAh7zg+001728
 MPI-TASK:   1 -           10 BYTES IN      2 RECORDS TRANSFERRED IN    0.0003 SECONDS     0.0 Mbytes/s, TOTAL TIME=    0.0034( 8.8%)
 STEPO_FPOS WAS : V00000000Y
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ./ICMUAh7zg+001728 MODE=a
 IO-STREAM CLOSED - ./ICMUAh7zg+001728
 MPI-TASK:   1 -    302246579 BYTES IN      4 RECORDS TRANSFERRED IN    0.0006 SECONDS503744.3 Mbytes/s, TOTAL TIME=    0.0062( 9.7%)
 STEPO_FPOS WAS : T00000000M
 NSTEP =  1728 STEPO   0AAA00AAA
------------------------------------------------
CNT0 not found; string returned was: 'STEPO'
>>> Printing last 8 lines from file: ifs.stat

 05:58:02 0AAA00AAA STEPO     1727    22.080   22.080   22.084     75:40     76:19 0.51185972818928E-05       2GB       0MB
 05:58:25 F00000000 STEPO     1728    23.032   23.032   23.044     76:03     76:42 0.50862099333019E-05       2GB       0MB
 05:58:30 0AA000000 STEPO     1728     3.817    3.817    5.034     76:06     76:47 0.50862099333019E-05       2GB       0MB
 05:58:32 FULLPOS-B DYNFPOS   1728     1.338    1.338    1.339     76:08     76:48 0.50862099333019E-05       2GB       0MB
 05:58:38 FULLPOS-V DYNFPOS   1728     2.886    2.886    6.465     76:11     76:55 0.50862099333019E-05       2GB       0MB
 05:58:39 FULLPOS-T DYNFPOS   1728     0.495    0.495    0.495     76:11     76:55 0.50862099333019E-05       2GB       0MB
 05:58:39 0AAA00AAA STEPO     1728     0.529    0.529    0.531     76:12     76:56 0.50862099333019E-05       2GB       0MB
------------------------------------------------
>>> Printing last 11 lines from file: rcf

 CSTEP   = "    1728",
 CTIME   = "00180000      ",
 NSTEPLPP        =         201,         202,          49,      228226,      228227, 2*0,        1716, 2*0,
 IPRGPNSRES      =           1,
 IPRGPEWRES      =           1,
 IPRTRWRES       =           1,
 IPRTRVRES       =           1,
 GMASS0  =  0.000000000000000E+000,
 GMASSI  =  0.000000000000000E+000
 /
------------------------------------------------
>>> Printing last 8 lines from file: /var/lib/boinc-client/slots/15/progress_file_12291089.xml

<running_values>
  <last_cpu_time>41255.810000</last_cpu_time>
  <upload_file_number>17</upload_file_number>
  <last_iter>1728</last_iter>
  <last_upload>1469700</last_upload>
  <model_completed>0</model_completed>
</running_values>
------------------------------------------------
..Failed, model did not complete successfully
(argv0) ../../projects/climateprediction.net/oifs_43r3_bl_1.13_x86_64-pc-linux-gnu
(argv1) start_date: 2016092300
(argv2) exptid: h7zg
(argv3) unique_member_id: a24q
(argv4) batchid: 1018
(argv5) wuid: 12291089
(argv6) fclen: 20
(argv7) app_name: oifs_43r3_bl
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/15
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_bl
version: 1.13
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/15/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/15/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_94906353b2336031055cc8e903887ab6 to: /var/lib/boinc-client/slots/15/oifs_43r3_bl_a24q_2016092300_20_1018_12291089.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/15/oifs_43r3_bl_a24q_2016092300_20_1018_12291089.zip
ic_ancil_file: ic_ancil_12291089
ifsdata_file: ifsdata_12291089
climate_data_file: clim_data_12291089
horiz_resolution: 159
vert_resolution: 91
grid_type: l_2
upload_interval: 96
utstep: 900
nfrres: restart dump frequency (steps) 48
Copying IC ancils from: ../../projects/climateprediction.net/jf_f7ecfd27aa1e68bbe7e8d19800e25c73 to: /var/lib/boinc-client/slots/15/ic_ancil_12291089.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/15/ic_ancil_12291089.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_03ff56090999cbfd513d63a5ebf054f1 to: /var/lib/boinc-client/slots/15/ifsdata/ifsdata_12291089.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/15/ifsdata/ifsdata_12291089.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/15/159l_2/clim_data_12291089.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/15/159l_2/clim_data_12291089.zip
Checking for progress XML file: /var/lib/boinc-client/slots/15/progress_file_12291089.xml
Opened progress file ok : /var/lib/boinc-client/slots/15/progress_file_12291089.xml
-- Model is restarting --
Adjusting last_iter, 1728, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/15/progress_file_12291089.xml
last_cpu_time: 41255
upload_file_number: 17
last_iter: 1729
last_upload: 1469700
model_completed: 0
total_length_of_simulation: 1728000
result_base_name: oifs_43r3_bl_a24q_2016092300_20_1018_12291089_0_r934871342
The child process has been launched with process id: 27103
Executing the command: /var/lib/boinc-client/slots/15/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:server242:1:1:27103:27103] [20240618:055949:1718683189:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2091] fp = 0x314f440
[EC_DRHOOK:server242:1:1:27103:27103] [20240618:055949:1718683189:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:server242:1:1:27103:27103] [20240618:055949:1718683189:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:server242:1:1:27103:27103] [20240618:055949:1718683189:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:server242:1:1:27103:27103] [20240618:055949:1718683189:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:server242:1:1:27103:27103] [20240618:055949:1718683189:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:server242:1:1:27103:27103] [20240618:055949:1718683189:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:server242:1:1:27103:27103] [20240618:055949:1718683189:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:server242:1:1:27103:27103] [20240618:055949:1718683189:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:server242:1:1:27103:27103] [20240618:055949:1718683189:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:server242:1:1:27103:27103] [20240618:055949:1718683189:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:server242:1:1:27103:27103] [20240618:055949:1718683189:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:server242:1:1:27103:27103] [20240618:055949:1718683189:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:server242:1:1:27103:27103] [20240618:055949:1718683189:0.000] [ignore_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:server242:1:1:27103:27103] [20240618:055949:1718683189:0.000] [restore_default_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:server242:1:1:27103:27103] [20240618:055949:1718683189:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:server242:1:1:27103:27103] [20240618:055949:1718683189:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:server242:1:1:27103:27103] [20240618:055949:1718683189:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:server242:1:1:27103:27103] [20240618:055949:1718683189:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x820cf0 (old at (nil))
[EC_DRHOOK:server242:1:1:27103:27103] [20240618:055949:1718683189:0.000] [trapfpe_treatment@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:server242:1:1:27103:27103] [20240618:055949:1718683189:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:server242:1:1:27103:27103] [20240618:055949:1718683189:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:server242:1:1:27103:27103] [20240618:055949:1718683189:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x820cf0 (old at (nil))
[EC_DRHOOK:server242:1:1:27103:27103] [20240618:055949:1718683189:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:server242:1:1:27103:27103] [20240618:055949:1718683189:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:server242:1:1:27103:27103] [20240618:055949:1718683189:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:server242:1:1:27103:27103] [20240618:055949:1718683189:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x820cf0 (old at (nil))
[EC_DRHOOK:server242:1:1:27103:27103] [20240618:055949:1718683189:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x820cf0 (old at (nil))
[EC_DRHOOK:server242:1:1:27103:27103] [20240618:055949:1718683189:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:server242:1:1:27103:27103] [20240618:055949:1718683189:0.000] [catch_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMGGh7zg+001644
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMSHh7zg+001644
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMUAh7zg+001644
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMGGh7zg+001656
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMSHh7zg+001656
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMUAh7zg+001656
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMGGh7zg+001668
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMSHh7zg+001668
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMUAh7zg+001668
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMGGh7zg+001680
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMSHh7zg+001680
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMUAh7zg+001680
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMGGh7zg+001692
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMSHh7zg+001692
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMUAh7zg+001692
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMGGh7zg+001704
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMSHh7zg+001704
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMUAh7zg+001704
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMGGh7zg+001716
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMSHh7zg+001716
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMUAh7zg+001716
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a24q_2016092300_20_1018_12291089_0_r934871342_17.zip
Uploading the intermediate file: upload_file_17.zip
  06:00:19 STEP 1728 H= 432:00 +CPU= 26.560
  06:00:43 STEP 1729 H= 432:15 +CPU= 23.021
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMGGh7zg+001728
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMSHh7zg+001728
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMUAh7zg+001728
  06:01:05 STEP 1730 H= 432:30 +CPU= 22.386
  06:01:28 STEP 1731 H= 432:45 +CPU= 22.852
  06:01:51 STEP 1732 H= 433:00 +CPU= 23.317
  06:02:13 STEP 1733 H= 433:15 +CPU= 22.008
  06:02:35 STEP 1734 H= 433:30 +CPU= 21.907
  06:02:58 STEP 1735 H= 433:45 +CPU= 23.006
  06:03:22 STEP 1736 H= 434:00 +CPU= 23.798
  06:03:44 STEP 1737 H= 434:15 +CPU= 21.788
  06:04:06 STEP 1738 H= 434:30 +CPU= 21.954
  06:04:30 STEP 1739 H= 434:45 +CPU= 23.984
  06:05:07 STEP 1740 H= 435:00 +CPU= 29.195
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMGGh7zg+001740
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMSHh7zg+001740
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMUAh7zg+001740
  06:05:32 STEP 1741 H= 435:15 +CPU= 24.902
  06:05:54 STEP 1742 H= 435:30 +CPU= 22.338
  06:06:16 STEP 1743 H= 435:45 +CPU= 21.972
  06:06:38 STEP 1744 H= 436:00 +CPU= 22.116
  06:07:01 STEP 1745 H= 436:15 +CPU= 22.130
  06:07:24 STEP 1746 H= 436:30 +CPU= 23.018
  06:07:46 STEP 1747 H= 436:45 +CPU= 22.782
  06:08:09 STEP 1748 H= 437:00 +CPU= 22.794
  06:08:32 STEP 1749 H= 437:15 +CPU= 22.422
  06:08:55 STEP 1750 H= 437:30 +CPU= 23.183
  06:09:18 STEP 1751 H= 437:45 +CPU= 23.236
  06:09:46 STEP 1752 H= 438:00 +CPU= 27.601
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMGGh7zg+001752
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMSHh7zg+001752
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMUAh7zg+001752
  06:10:09 STEP 1753 H= 438:15 +CPU= 22.825
  06:10:33 STEP 1754 H= 438:30 +CPU= 23.657
  06:10:55 STEP 1755 H= 438:45 +CPU= 21.932
  06:11:18 STEP 1756 H= 439:00 +CPU= 23.551
  06:11:41 STEP 1757 H= 439:15 +CPU= 22.905
  06:12:04 STEP 1758 H= 439:30 +CPU= 22.883
  06:12:27 STEP 1759 H= 439:45 +CPU= 22.535
  06:12:49 STEP 1760 H= 440:00 +CPU= 22.705
  06:13:12 STEP 1761 H= 440:15 +CPU= 22.655
  06:13:34 STEP 1762 H= 440:30 +CPU= 22.363
  06:13:57 STEP 1763 H= 440:45 +CPU= 22.396
  06:14:26 STEP 1764 H= 441:00 +CPU= 29.094
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMGGh7zg+001764
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMSHh7zg+001764
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMUAh7zg+001764
  06:14:48 STEP 1765 H= 441:15 +CPU= 22.280
  06:15:11 STEP 1766 H= 441:30 +CPU= 22.615
  06:15:34 STEP 1767 H= 441:45 +CPU= 22.995
  06:15:57 STEP 1768 H= 442:00 +CPU= 22.563
  06:16:19 STEP 1769 H= 442:15 +CPU= 22.255
  06:16:41 STEP 1770 H= 442:30 +CPU= 22.713
  06:17:05 STEP 1771 H= 442:45 +CPU= 22.959
  06:17:27 STEP 1772 H= 443:00 +CPU= 22.967
  06:17:50 STEP 1773 H= 443:15 +CPU= 22.227
  06:18:13 STEP 1774 H= 443:30 +CPU= 23.261
  06:18:35 STEP 1775 H= 443:45 +CPU= 22.032
  06:19:07 STEP 1776 H= 444:00 +CPU= 31.308
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMGGh7zg+001776
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMSHh7zg+001776
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMUAh7zg+001776
  06:19:30 STEP 1777 H= 444:15 +CPU= 23.457
  06:19:52 STEP 1778 H= 444:30 +CPU= 22.310
  06:20:15 STEP 1779 H= 444:45 +CPU= 22.966
  06:20:39 STEP 1780 H= 445:00 +CPU= 23.590
  06:21:02 STEP 1781 H= 445:15 +CPU= 23.166
  06:21:25 STEP 1782 H= 445:30 +CPU= 22.298
  06:21:47 STEP 1783 H= 445:45 +CPU= 22.164
  06:22:10 STEP 1784 H= 446:00 +CPU= 23.285
  06:22:32 STEP 1785 H= 446:15 +CPU= 21.632
  06:22:54 STEP 1786 H= 446:30 +CPU= 21.945
  06:23:17 STEP 1787 H= 446:45 +CPU= 22.873
  06:23:45 STEP 1788 H= 447:00 +CPU= 28.198
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMGGh7zg+001788
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMSHh7zg+001788
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMUAh7zg+001788
  06:24:07 STEP 1789 H= 447:15 +CPU= 22.294
  06:24:30 STEP 1790 H= 447:30 +CPU= 22.302
  06:24:51 STEP 1791 H= 447:45 +CPU= 21.860
  06:25:14 STEP 1792 H= 448:00 +CPU= 22.941
  06:25:37 STEP 1793 H= 448:15 +CPU= 22.670
  06:25:59 STEP 1794 H= 448:30 +CPU= 22.062
  06:26:22 STEP 1795 H= 448:45 +CPU= 22.440
  06:26:45 STEP 1796 H= 449:00 +CPU= 23.046
  06:27:07 STEP 1797 H= 449:15 +CPU= 22.633
  06:27:30 STEP 1798 H= 449:30 +CPU= 22.285
  06:27:52 STEP 1799 H= 449:45 +CPU= 22.251
  06:28:20 STEP 1800 H= 450:00 +CPU= 28.156
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMGGh7zg+001800
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMSHh7zg+001800
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMUAh7zg+001800
  06:28:42 STEP 1801 H= 450:15 +CPU= 22.179
  06:29:05 STEP 1802 H= 450:30 +CPU= 22.524
  06:29:28 STEP 1803 H= 450:45 +CPU= 22.652
  06:29:50 STEP 1804 H= 451:00 +CPU= 22.445
  06:30:11 STEP 1805 H= 451:15 +CPU= 20.928
  06:30:32 STEP 1806 H= 451:30 +CPU= 21.431
  06:30:54 STEP 1807 H= 451:45 +CPU= 21.385
  06:31:17 STEP 1808 H= 452:00 +CPU= 23.517
  06:31:40 STEP 1809 H= 452:15 +CPU= 22.167
  06:32:02 STEP 1810 H= 452:30 +CPU= 22.352
  06:32:24 STEP 1811 H= 452:45 +CPU= 22.122
  06:32:52 STEP 1812 H= 453:00 +CPU= 27.863
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMGGh7zg+001812
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMSHh7zg+001812
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMUAh7zg+001812
  06:33:14 STEP 1813 H= 453:15 +CPU= 22.434
  06:33:37 STEP 1814 H= 453:30 +CPU= 22.532
  06:33:59 STEP 1815 H= 453:45 +CPU= 22.015
  06:34:22 STEP 1816 H= 454:00 +CPU= 22.834
  06:34:44 STEP 1817 H= 454:15 +CPU= 22.020
  06:35:06 STEP 1818 H= 454:30 +CPU= 22.487
  06:35:28 STEP 1819 H= 454:45 +CPU= 21.395
  06:35:49 STEP 1820 H= 455:00 +CPU= 21.620
  06:36:10 STEP 1821 H= 455:15 +CPU= 20.948
  06:36:33 STEP 1822 H= 455:30 +CPU= 22.420
  06:36:55 STEP 1823 H= 455:45 +CPU= 22.354
  06:37:27 STEP 1824 H= 456:00 +CPU= 31.823
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMGGh7zg+001824
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMSHh7zg+001824
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMUAh7zg+001824
  06:37:50 STEP 1825 H= 456:15 +CPU= 22.339
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMGGh7zg+001740
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMSHh7zg+001740
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMUAh7zg+001740
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMGGh7zg+001752
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMSHh7zg+001752
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMUAh7zg+001752
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMGGh7zg+001764
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMSHh7zg+001764
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMUAh7zg+001764
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMGGh7zg+001776
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMSHh7zg+001776
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMUAh7zg+001776
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMGGh7zg+001788
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMSHh7zg+001788
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMUAh7zg+001788
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMGGh7zg+001800
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMSHh7zg+001800
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMUAh7zg+001800
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMGGh7zg+001812
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMSHh7zg+001812
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMUAh7zg+001812
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMGGh7zg+001824
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMSHh7zg+001824
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMUAh7zg+001824
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a24q_2016092300_20_1018_12291089_0_r934871342_18.zip
Uploading the intermediate file: upload_file_18.zip
  06:38:11 STEP 1826 H= 456:30 +CPU= 21.812
  06:38:34 STEP 1827 H= 456:45 +CPU= 22.132
  06:38:56 STEP 1828 H= 457:00 +CPU= 22.745
  06:39:19 STEP 1829 H= 457:15 +CPU= 22.223
  06:39:40 STEP 1830 H= 457:30 +CPU= 21.969
  06:40:03 STEP 1831 H= 457:45 +CPU= 22.339
  06:40:25 STEP 1832 H= 458:00 +CPU= 22.266
  06:40:46 STEP 1833 H= 458:15 +CPU= 20.399
  06:41:07 STEP 1834 H= 458:30 +CPU= 21.171
  06:41:29 STEP 1835 H= 458:45 +CPU= 22.151
  06:41:57 STEP 1836 H= 459:00 +CPU= 27.815
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMGGh7zg+001836
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMSHh7zg+001836
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMUAh7zg+001836
  06:42:19 STEP 1837 H= 459:15 +CPU= 22.384
  06:42:42 STEP 1838 H= 459:30 +CPU= 22.322
  06:43:04 STEP 1839 H= 459:45 +CPU= 22.131
  06:43:26 STEP 1840 H= 460:00 +CPU= 22.555
  06:43:49 STEP 1841 H= 460:15 +CPU= 22.307
  06:44:11 STEP 1842 H= 460:30 +CPU= 22.208
  06:44:34 STEP 1843 H= 460:45 +CPU= 23.050
  06:44:57 STEP 1844 H= 461:00 +CPU= 22.863
  06:45:19 STEP 1845 H= 461:15 +CPU= 22.630
  06:45:42 STEP 1846 H= 461:30 +CPU= 22.223
  06:46:04 STEP 1847 H= 461:45 +CPU= 22.174
  06:46:32 STEP 1848 H= 462:00 +CPU= 28.218
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMGGh7zg+001848
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMSHh7zg+001848
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMUAh7zg+001848
  06:46:53 STEP 1849 H= 462:15 +CPU= 21.068
  06:47:14 STEP 1850 H= 462:30 +CPU= 20.494
  06:47:35 STEP 1851 H= 462:45 +CPU= 20.916
  06:47:57 STEP 1852 H= 463:00 +CPU= 22.590
  06:48:20 STEP 1853 H= 463:15 +CPU= 22.570
  06:48:42 STEP 1854 H= 463:30 +CPU= 22.562
  06:49:05 STEP 1855 H= 463:45 +CPU= 22.491
  06:49:28 STEP 1856 H= 464:00 +CPU= 23.151
  06:49:51 STEP 1857 H= 464:15 +CPU= 22.494
  06:50:13 STEP 1858 H= 464:30 +CPU= 22.714
  06:50:36 STEP 1859 H= 464:45 +CPU= 22.415
  06:51:04 STEP 1860 H= 465:00 +CPU= 27.704
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMGGh7zg+001860
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMSHh7zg+001860
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMUAh7zg+001860
  06:51:26 STEP 1861 H= 465:15 +CPU= 22.148
  06:51:48 STEP 1862 H= 465:30 +CPU= 21.900
  06:52:10 STEP 1863 H= 465:45 +CPU= 22.461
  06:52:33 STEP 1864 H= 466:00 +CPU= 22.489
  06:52:55 STEP 1865 H= 466:15 +CPU= 22.475
  06:53:17 STEP 1866 H= 466:30 +CPU= 22.199
  06:53:39 STEP 1867 H= 466:45 +CPU= 21.999
  06:54:02 STEP 1868 H= 467:00 +CPU= 22.826
  06:54:25 STEP 1869 H= 467:15 +CPU= 22.473
  06:54:47 STEP 1870 H= 467:30 +CPU= 22.257
  06:55:09 STEP 1871 H= 467:45 +CPU= 22.384
  06:55:42 STEP 1872 H= 468:00 +CPU= 32.347
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMGGh7zg+001872
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMSHh7zg+001872
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMUAh7zg+001872
  06:56:04 STEP 1873 H= 468:15 +CPU= 21.989
  06:56:26 STEP 1874 H= 468:30 +CPU= 22.205
  06:56:49 STEP 1875 H= 468:45 +CPU= 22.446
  06:57:12 STEP 1876 H= 469:00 +CPU= 22.760
  06:57:34 STEP 1877 H= 469:15 +CPU= 22.182
  06:57:56 STEP 1878 H= 469:30 +CPU= 22.207
  06:58:19 STEP 1879 H= 469:45 +CPU= 22.515
  06:58:41 STEP 1880 H= 470:00 +CPU= 22.716
  06:59:03 STEP 1881 H= 470:15 +CPU= 22.072
  06:59:26 STEP 1882 H= 470:30 +CPU= 22.272
  06:59:48 STEP 1883 H= 470:45 +CPU= 21.989
  07:00:16 STEP 1884 H= 471:00 +CPU= 28.180
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMGGh7zg+001884
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMSHh7zg+001884
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMUAh7zg+001884
  07:00:39 STEP 1885 H= 471:15 +CPU= 22.772
  07:01:01 STEP 1886 H= 471:30 +CPU= 22.518
  07:01:24 STEP 1887 H= 471:45 +CPU= 22.649
  07:01:47 STEP 1888 H= 472:00 +CPU= 22.823
  07:02:09 STEP 1889 H= 472:15 +CPU= 22.054
  07:02:31 STEP 1890 H= 472:30 +CPU= 22.276
  07:02:53 STEP 1891 H= 472:45 +CPU= 22.047
  07:03:15 STEP 1892 H= 473:00 +CPU= 21.366
  07:03:35 STEP 1893 H= 473:15 +CPU= 20.725
  07:03:57 STEP 1894 H= 473:30 +CPU= 21.064
  07:04:19 STEP 1895 H= 473:45 +CPU= 22.505
  07:04:48 STEP 1896 H= 474:00 +CPU= 28.316
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMGGh7zg+001896
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMSHh7zg+001896
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMUAh7zg+001896
  07:05:10 STEP 1897 H= 474:15 +CPU= 22.326
  07:05:33 STEP 1898 H= 474:30 +CPU= 22.515
  07:05:55 STEP 1899 H= 474:45 +CPU= 22.124
  07:06:18 STEP 1900 H= 475:00 +CPU= 22.831
  07:06:40 STEP 1901 H= 475:15 +CPU= 22.406
  07:07:03 STEP 1902 H= 475:30 +CPU= 22.402
  07:07:25 STEP 1903 H= 475:45 +CPU= 22.408
  07:07:48 STEP 1904 H= 476:00 +CPU= 23.138
  07:08:11 STEP 1905 H= 476:15 +CPU= 22.309
  07:08:33 STEP 1906 H= 476:30 +CPU= 22.280
  07:08:55 STEP 1907 H= 476:45 +CPU= 22.320
  07:09:23 STEP 1908 H= 477:00 +CPU= 28.029
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMGGh7zg+001908
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMSHh7zg+001908
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMUAh7zg+001908
  07:09:45 STEP 1909 H= 477:15 +CPU= 22.121
  07:10:08 STEP 1910 H= 477:30 +CPU= 22.923
  07:10:30 STEP 1911 H= 477:45 +CPU= 21.700
  07:10:53 STEP 1912 H= 478:00 +CPU= 22.590
  07:11:15 STEP 1913 H= 478:15 +CPU= 22.233
  07:11:37 STEP 1914 H= 478:30 +CPU= 22.388
  07:12:00 STEP 1915 H= 478:45 +CPU= 22.463
  07:12:23 STEP 1916 H= 479:00 +CPU= 23.229
  07:12:45 STEP 1917 H= 479:15 +CPU= 21.881
  07:13:07 STEP 1918 H= 479:30 +CPU= 22.468
  07:13:29 STEP 1919 H= 479:45 +CPU= 21.943
  07:13:59 STEP 1920 H= 480:00 +CPU= 29.898
..The child process terminated with status: 0
>>> Printing last 70 lines from file: NODE.001_01

 
DDH------------------------------------------------------------------------
I                                                                         I
I TIME =  480.000H DDH * Diagnostic Var.,     AREA                GLOBAL  I
I                                                                         I
I-------------------------------------------------------------------------I
I   VARIABLE   I      Initial value         I        Current value        I
I-------------------------------------------------------------------------I
I  OMEGA (P/S) I     -0.6295950056E-21      I      -0.1144582199E-04      I
ICloud fractionI      0.0000000000E+00      I       0.8037846506E-01      I
I  Relat. Hum. I      0.2933665832E+00      I       0.4133418956E+00      I
I  PASS 01     I      0.0000000000E+00      I       0.1049099920E+04      I
DDH------------------------------------------------------------------------
  
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
 NSTEP =  1920 SCAN2M_HPOS  P
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ICMGGh7zg+001920 MODE=a
 IO-STREAM CLOSED - ICMGGh7zg+001920
 MPI-TASK:   1 -    299038020 BYTES IN      7 RECORDS TRANSFERRED IN    0.0009 SECONDS332264.5 Mbytes/s, TOTAL TIME=    0.0106( 8.5%)
 07:13:59 STEP 1920 H= 480:00 +CPU= 29.898
 END CNT3
 NSTEP =  1920 CNT0   000000000  21.565  21.565  21.710   0.993               0               0               0               0               0
IO-STREAM STATISTICS, TOTAL NO OF REC/BYTES READ           46/  1069125708 WRITTEN         5720/************
 -TASK-OPENED-OPEN-RECS IN -KBYTE IN    -RECS OUT-KBYTE OUT-WALL   -WALL IN-WALL
  OU-TOT IN -TOT OUT
   1    103    0       46      1044068     5720 ************    17.3     1.5    15.8     2.3    25.8
===-=== START OF TIMING STATISTICS ===-===
 
STATS FOR ALL TASKS
 NUM ROUTINE                                     CALLS  MEAN(ms)   MAX(ms)   FRAC(%)  UNBAL(%)
   0 CNT0     - COMPLETE EXECUTION                   1 4450403.5 4450403.5    100.00      0.00
   1 CNT4     - FORWARD INTEGRATION                  1 4446553.0 4446553.0     99.91      0.00
   8 SCAN2M - GRID-POINT DYNAMICS                  209    3938.0    3938.0     18.49      0.00
   9 SPCM     - SPECTRAL COMP.                     192     640.3     640.3      2.76      0.00
  10 SCAN2M - PHYSICS                              193   10630.1   10630.1     46.10      0.00
  11 IOPACK   - OUTPUT P.P. RESULTS                 16     933.2     933.2      0.34      0.00
  12 SPNORM   - SPECTRAL NORM COMP.                  9      41.6      41.6      0.01      0.00
  17 GRIDFPOS IN CNT4                               16     196.0     196.0      0.07      0.00
  27 RESTART FILES - WRITING                         4    3726.8    3726.8      0.33      0.00
  28 RESTART FILES - READING                         1    3160.3    3160.3      0.07      0.00
  29 SU4FPOS IN CNT4                                16       0.1       0.1      0.00      0.00
  30 DYNFPOS IN CNT4                                16    5237.8    5237.8      1.88      0.00
  31 POSDDH IN STEPO                                 1      28.6      28.6      0.00      0.00
  37 CPGLAG   - SL COMPUTATIONS                    193    4924.8    4924.8     21.36      0.00
  39 SU0YOMB                                         1     447.0     447.0      0.01      0.00
  51 SCAN2M   - SL COMM. PART 1                    193      19.7      19.7      0.09      0.00
  54 SPCM     - M TO S/S TO M TRANSP.              192     297.7     297.7      1.28      0.00
  55 SPCIMPF  - S TO M/M TO S TRANSP.              192      47.8      47.8      0.21      0.00
  56 SPNORM   - SPECTRAL NORM COMM.                  9       0.2       0.2      0.00      0.00
 102 LTINV_CTL   - INVERSE LEGENDRE TRANSFORM      420     302.6     302.6      2.86      0.00
 103 LTDIR_CTL   - DIRECT LEGENDRE TRANSFORM       400     168.2     168.2      1.51      0.00
 106 FTDIR_CTL   - DIRECT FOURIER TRANSFORM        400      63.5      63.5      0.57      0.00
 107 FTINV_CTL   - INVERSE FOURIER TRANSFORM       420      83.1      83.1      0.78      0.00
 140 SULEG       - COMP. OF LEGENDRE POL.            1      32.9      32.9      0.00      0.00
 152 LTINV_CTL   - M TO L TRANSPOSITION            420      61.1      61.1      0.58      0.00
 153 LTDIR_CTL   - L TO M TRANSPOSITION            400      31.0      31.0      0.28      0.00
 157 FTINV_CTL   - L TO G TRANSPOSITION            420      55.8      55.8      0.53      0.00
 158 FTDIR_CTL   - G TO L TRANSPOSITION            400      32.2      32.2      0.29      0.00
 400 GSTATS                                      23311       0.0       0.0      0.00      0.00
 401 GSTATS HOOK                                 21635       0.0       0.0      0.00      0.00
TOTAL MEASURED IMBALANCE =       0.0 SECONDS,  0.0 PERCENT
TOTAL WALLCLOCK TIME     4450.4 CPU TIME    4436.2 VECTOR TIME     4436.2


===-=== END   OF TIMING STATISTICS ===-===
FORECAST DAYS PER DAY      388.6
  *** END CNT0 *** 
------------------------------------------------
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMGGh7zg+001920
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMSHh7zg+001920
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMUAh7zg+001920
Adding to the zip: /var/lib/boinc-client/slots/15/NODE.001_01
Adding to the zip: /var/lib/boinc-client/slots/15/ifs.stat
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMSHh7zg+001836
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMUAh7zg+001896
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMSHh7zg+001896
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMGGh7zg+001728
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMUAh7zg+001836
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMUAh7zg+000960
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMGGh7zg+001908
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMUAh7zg+000384
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMGGh7zg+001884
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMSHh7zg+001848
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMUAh7zg+001728
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMUAh7zg+001872
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMUAh7zg+001860
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMUAh7zg+001884
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMGGh7zg+001152
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMSHh7zg+001860
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMGGh7zg+001836
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMGGh7zg+001896
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMSHh7zg+001872
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMGGh7zg+001872
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMSHh7zg+001908
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMUAh7zg+001152
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMUAh7zg+000192
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMSHh7zg+001152
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMSHh7zg+001884
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMGGh7zg+001848
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMUAh7zg+001908
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMGGh7zg+001536
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMSHh7zg+000384
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMGGh7zg+001860
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMGGh7zg+001920
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMSHh7zg+000960
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMUAh7zg+001920
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMSHh7zg+001920
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMSHh7zg+000192
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMSHh7zg+001536
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMUAh7zg+001848
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMGGh7zg+000960
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMSHh7zg+001728
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMUAh7zg+001536
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMGGh7zg+000192
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291089/ICMGGh7zg+000384
Zipping up the final file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a24q_2016092300_20_1018_12291089_0_r934871342_19.zip
Uploading the final file: upload_file_19.zip
Uploading trickle at timestep: 1727100
07:17:45 (27100): called boinc_finish(0)

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
18 Jun 2024 05:51:34 1469580 22444687 oifs_43r3_bl_a24q_2016092300_20_1018_12291089_0 1,727,100 45,691 0.0265


©2024 climateprediction.net