climateprediction.net home page
Task 22442610

Task 22442610

Name oifs_43r3_bl_a0jo_2016092300_20_1018_12289035_0
Workunit 12289035
Created 12 Jun 2024, 20:59:55 UTC
Sent 13 Jun 2024, 12:10:32 UTC
Report deadline 12 Aug 2024, 12:10:32 UTC
Received 14 Jun 2024, 3:36:52 UTC
Server state Over
Outcome Success
Client state Done
Exit status 0 (0x00000000)
Computer ID 1537687
Run time 13 hours 52 min 50 sec
CPU time 13 hours 44 min 45 sec
Validate state Valid
Credit 2,638.30
Device peak FLOPS 7.20 GFLOPS
Application version OpenIFS 43r3 Baroclinic Lifecycle v1.13
x86_64-pc-linux-gnu
Peak working set size 5,025.09 MB
Peak swap size 5,981.12 MB
Peak disk usage 1,355.75 MB
Stderr
<core_client_version>7.18.1</core_client_version>
<![CDATA[
<stderr_txt>
nIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:box.zenlime.org:1:1:827245:827245] [20240614:001733:1718324253:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:box.zenlime.org:1:1:827245:827245] [20240614:001733:1718324253:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:box.zenlime.org:1:1:827245:827245] [20240614:001733:1718324253:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:box.zenlime.org:1:1:827245:827245] [20240614:001733:1718324253:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:box.zenlime.org:1:1:827245:827245] [20240614:001733:1718324253:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:box.zenlime.org:1:1:827245:827245] [20240614:001733:1718324253:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:box.zenlime.org:1:1:827245:827245] [20240614:001733:1718324253:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:box.zenlime.org:1:1:827245:827245] [20240614:001733:1718324253:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:box.zenlime.org:1:1:827245:827245] [20240614:001733:1718324253:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:box.zenlime.org:1:1:827245:827245] [20240614:001733:1718324253:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:box.zenlime.org:1:1:827245:827245] [20240614:001733:1718324253:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:box.zenlime.org:1:1:827245:827245] [20240614:001733:1718324253:0.000] [ignore_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:box.zenlime.org:1:1:827245:827245] [20240614:001733:1718324253:0.000] [restore_default_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:box.zenlime.org:1:1:827245:827245] [20240614:001733:1718324253:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:box.zenlime.org:1:1:827245:827245] [20240614:001733:1718324253:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:box.zenlime.org:1:1:827245:827245] [20240614:001733:1718324253:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:box.zenlime.org:1:1:827245:827245] [20240614:001733:1718324253:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x820cf0 (old at (nil))
[EC_DRHOOK:box.zenlime.org:1:1:827245:827245] [20240614:001733:1718324253:0.000] [trapfpe_treatment@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:box.zenlime.org:1:1:827245:827245] [20240614:001733:1718324253:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:box.zenlime.org:1:1:827245:827245] [20240614:001733:1718324253:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:box.zenlime.org:1:1:827245:827245] [20240614:001733:1718324253:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x820cf0 (old at (nil))
[EC_DRHOOK:box.zenlime.org:1:1:827245:827245] [20240614:001733:1718324253:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:box.zenlime.org:1:1:827245:827245] [20240614:001733:1718324253:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:box.zenlime.org:1:1:827245:827245] [20240614:001733:1718324253:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:box.zenlime.org:1:1:827245:827245] [20240614:001733:1718324253:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x820cf0 (old at (nil))
[EC_DRHOOK:box.zenlime.org:1:1:827245:827245] [20240614:001733:1718324253:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x820cf0 (old at (nil))
[EC_DRHOOK:box.zenlime.org:1:1:827245:827245] [20240614:001733:1718324253:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:box.zenlime.org:1:1:827245:827245] [20240614:001733:1718324253:0.000] [catch_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289035/ICMGGh7zg+001548
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289035/ICMSHh7zg+001548
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289035/ICMUAh7zg+001548
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289035/ICMGGh7zg+001560
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289035/ICMSHh7zg+001560
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289035/ICMUAh7zg+001560
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289035/ICMGGh7zg+001572
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289035/ICMSHh7zg+001572
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289035/ICMUAh7zg+001572
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289035/ICMGGh7zg+001584
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289035/ICMSHh7zg+001584
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289035/ICMUAh7zg+001584
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289035/ICMGGh7zg+001596
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289035/ICMSHh7zg+001596
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289035/ICMUAh7zg+001596
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289035/ICMGGh7zg+001608
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289035/ICMSHh7zg+001608
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289035/ICMUAh7zg+001608
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289035/ICMGGh7zg+001620
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289035/ICMSHh7zg+001620
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289035/ICMUAh7zg+001620
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a0jo_2016092300_20_1018_12289035_0_r1895645953_16.zip
Uploading the intermediate file: upload_file_16.zip
  00:18:06 STEP 1632 H= 408:00 +CPU= 27.975
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMGGh7zg+001632
  00:18:33 STEP 1633 H= 408:15 +CPU= 27.529
  00:19:02 STEP 1634 H= 408:30 +CPU= 27.902
  00:19:30 STEP 1635 H= 408:45 +CPU= 28.288
  00:20:00 STEP 1636 H= 409:00 +CPU= 29.064
  00:20:29 STEP 1637 H= 409:15 +CPU= 28.255
  00:20:55 STEP 1638 H= 409:30 +CPU= 24.950
  00:21:16 STEP 1639 H= 409:45 +CPU= 20.952
  00:21:37 STEP 1640 H= 410:00 +CPU= 20.807
  00:22:01 STEP 1641 H= 410:15 +CPU= 24.199
  00:22:26 STEP 1642 H= 410:30 +CPU= 25.057
  00:22:52 STEP 1643 H= 410:45 +CPU= 25.758
  00:23:23 STEP 1644 H= 411:00 +CPU= 31.292
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMGGh7zg+001644
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMSHh7zg+001644
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMUAh7zg+001644
  00:23:46 STEP 1645 H= 411:15 +CPU= 22.669
  00:24:14 STEP 1646 H= 411:30 +CPU= 27.781
  00:24:43 STEP 1647 H= 411:45 +CPU= 28.375
  00:25:12 STEP 1648 H= 412:00 +CPU= 29.088
  00:25:42 STEP 1649 H= 412:15 +CPU= 28.951
  00:26:11 STEP 1650 H= 412:30 +CPU= 28.246
  00:26:40 STEP 1651 H= 412:45 +CPU= 28.231
  00:27:05 STEP 1652 H= 413:00 +CPU= 25.638
  00:27:31 STEP 1653 H= 413:15 +CPU= 25.168
  00:27:56 STEP 1654 H= 413:30 +CPU= 25.610
  00:28:23 STEP 1655 H= 413:45 +CPU= 25.784
  00:28:55 STEP 1656 H= 414:00 +CPU= 31.104
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMGGh7zg+001656
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMSHh7zg+001656
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMUAh7zg+001656
  00:29:18 STEP 1657 H= 414:15 +CPU= 23.620
  00:29:45 STEP 1658 H= 414:30 +CPU= 25.860
  00:30:14 STEP 1659 H= 414:45 +CPU= 27.964
  00:30:45 STEP 1660 H= 415:00 +CPU= 29.180
  00:31:13 STEP 1661 H= 415:15 +CPU= 28.353
  00:31:42 STEP 1662 H= 415:30 +CPU= 28.481
  00:32:10 STEP 1663 H= 415:45 +CPU= 27.874
  00:32:40 STEP 1664 H= 416:00 +CPU= 29.344
  00:33:09 STEP 1665 H= 416:15 +CPU= 28.847
  00:33:38 STEP 1666 H= 416:30 +CPU= 28.478
  00:34:06 STEP 1667 H= 416:45 +CPU= 28.000
  00:34:42 STEP 1668 H= 417:00 +CPU= 35.071
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMGGh7zg+001668
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMSHh7zg+001668
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMUAh7zg+001668
  00:35:12 STEP 1669 H= 417:15 +CPU= 29.025
  00:35:41 STEP 1670 H= 417:30 +CPU= 28.576
  00:36:09 STEP 1671 H= 417:45 +CPU= 28.127
  00:36:39 STEP 1672 H= 418:00 +CPU= 29.249
  00:37:08 STEP 1673 H= 418:15 +CPU= 28.698
  00:37:37 STEP 1674 H= 418:30 +CPU= 28.609
  00:38:06 STEP 1675 H= 418:45 +CPU= 28.328
  00:38:35 STEP 1676 H= 419:00 +CPU= 28.895
  00:38:59 STEP 1677 H= 419:15 +CPU= 23.517
  00:39:19 STEP 1678 H= 419:30 +CPU= 20.571
  00:39:39 STEP 1679 H= 419:45 +CPU= 19.382
  00:40:13 STEP 1680 H= 420:00 +CPU= 34.584
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMGGh7zg+001680
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMSHh7zg+001680
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMUAh7zg+001680
  00:40:37 STEP 1681 H= 420:15 +CPU= 23.530
  00:41:04 STEP 1682 H= 420:30 +CPU= 26.625
  00:41:28 STEP 1683 H= 420:45 +CPU= 24.336
  00:41:56 STEP 1684 H= 421:00 +CPU= 27.244
  00:42:25 STEP 1685 H= 421:15 +CPU= 28.025
  00:42:53 STEP 1686 H= 421:30 +CPU= 28.180
  00:43:22 STEP 1687 H= 421:45 +CPU= 28.204
  00:43:51 STEP 1688 H= 422:00 +CPU= 28.909
  00:44:19 STEP 1689 H= 422:15 +CPU= 28.294
  00:44:48 STEP 1690 H= 422:30 +CPU= 28.723
  00:45:16 STEP 1691 H= 422:45 +CPU= 28.211
  00:45:52 STEP 1692 H= 423:00 +CPU= 34.898
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMGGh7zg+001692
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMSHh7zg+001692
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMUAh7zg+001692
  00:46:20 STEP 1693 H= 423:15 +CPU= 27.681
  00:46:49 STEP 1694 H= 423:30 +CPU= 28.357
  00:47:18 STEP 1695 H= 423:45 +CPU= 28.267
  00:47:49 STEP 1696 H= 424:00 +CPU= 29.504
  00:48:18 STEP 1697 H= 424:15 +CPU= 28.666
  00:48:47 STEP 1698 H= 424:30 +CPU= 28.815
  00:49:15 STEP 1699 H= 424:45 +CPU= 28.388
  00:49:44 STEP 1700 H= 425:00 +CPU= 28.652
  00:50:12 STEP 1701 H= 425:15 +CPU= 28.210
  00:50:37 STEP 1702 H= 425:30 +CPU= 24.384
  00:50:57 STEP 1703 H= 425:45 +CPU= 20.148
  00:51:24 STEP 1704 H= 426:00 +CPU= 26.486
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMGGh7zg+001704
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMSHh7zg+001704
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMUAh7zg+001704
  00:51:48 STEP 1705 H= 426:15 +CPU= 24.327
  00:52:13 STEP 1706 H= 426:30 +CPU= 24.683
  00:52:39 STEP 1707 H= 426:45 +CPU= 25.794
  00:53:05 STEP 1708 H= 427:00 +CPU= 25.781
  00:53:29 STEP 1709 H= 427:15 +CPU= 24.775
  00:53:59 STEP 1710 H= 427:30 +CPU= 27.923
  00:54:28 STEP 1711 H= 427:45 +CPU= 29.089
  00:54:58 STEP 1712 H= 428:00 +CPU= 29.441
  00:55:26 STEP 1713 H= 428:15 +CPU= 28.656
  00:55:55 STEP 1714 H= 428:30 +CPU= 28.129
  00:56:24 STEP 1715 H= 428:45 +CPU= 28.746
  00:57:00 STEP 1716 H= 429:00 +CPU= 35.609
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMGGh7zg+001716
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMSHh7zg+001716
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMUAh7zg+001716
  00:57:30 STEP 1717 H= 429:15 +CPU= 28.284
  00:57:58 STEP 1718 H= 429:30 +CPU= 28.540
  00:58:27 STEP 1719 H= 429:45 +CPU= 28.547
  00:58:57 STEP 1720 H= 430:00 +CPU= 29.476
  00:59:26 STEP 1721 H= 430:15 +CPU= 28.589
  00:59:54 STEP 1722 H= 430:30 +CPU= 28.574
  01:00:23 STEP 1723 H= 430:45 +CPU= 28.616
  01:00:53 STEP 1724 H= 431:00 +CPU= 29.082
  01:01:21 STEP 1725 H= 431:15 +CPU= 28.319
  01:01:50 STEP 1726 H= 431:30 +CPU= 28.413
  01:02:18 STEP 1727 H= 431:45 +CPU= 28.624
  01:02:54 STEP 1728 H= 432:00 +CPU= 35.332
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMGGh7zg+001728
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMSHh7zg+001728
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMUAh7zg+001728
  01:03:15 STEP 1729 H= 432:15 +CPU= 21.362
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289035/ICMGGh7zg+001644
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289035/ICMSHh7zg+001644
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289035/ICMUAh7zg+001644
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289035/ICMGGh7zg+001656
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289035/ICMSHh7zg+001656
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289035/ICMUAh7zg+001656
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289035/ICMGGh7zg+001668
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289035/ICMSHh7zg+001668
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289035/ICMUAh7zg+001668
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289035/ICMGGh7zg+001680
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289035/ICMSHh7zg+001680
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289035/ICMUAh7zg+001680
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289035/ICMGGh7zg+001692
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289035/ICMSHh7zg+001692
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289035/ICMUAh7zg+001692
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289035/ICMGGh7zg+001704
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289035/ICMSHh7zg+001704
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289035/ICMUAh7zg+001704
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289035/ICMGGh7zg+001716
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289035/ICMSHh7zg+001716
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289035/ICMUAh7zg+001716
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289035/ICMGGh7zg+001728
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289035/ICMSHh7zg+001728
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289035/ICMUAh7zg+001728
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a0jo_2016092300_20_1018_12289035_0_r1895645953_17.zip
Uploading the intermediate file: upload_file_17.zip
  01:03:35 STEP 1730 H= 432:30 +CPU= 20.231
  01:04:00 STEP 1731 H= 432:45 +CPU= 24.144
  01:04:24 STEP 1732 H= 433:00 +CPU= 24.130
  01:04:49 STEP 1733 H= 433:15 +CPU= 24.526
  01:05:14 STEP 1734 H= 433:30 +CPU= 24.637
  01:05:40 STEP 1735 H= 433:45 +CPU= 26.254
  01:06:08 STEP 1736 H= 434:00 +CPU= 27.643
  01:06:36 STEP 1737 H= 434:15 +CPU= 27.670
  01:07:05 STEP 1738 H= 434:30 +CPU= 28.403
  01:07:35 STEP 1739 H= 434:45 +CPU= 28.292
  01:08:11 STEP 1740 H= 435:00 +CPU= 35.017
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMGGh7zg+001740
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMSHh7zg+001740
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMUAh7zg+001740
  01:08:39 STEP 1741 H= 435:15 +CPU= 27.731
  01:09:01 STEP 1742 H= 435:30 +CPU= 22.108
  01:09:21 STEP 1743 H= 435:45 +CPU= 20.382
  01:09:45 STEP 1744 H= 436:00 +CPU= 23.866
  01:10:10 STEP 1745 H= 436:15 +CPU= 25.185
  01:10:35 STEP 1746 H= 436:30 +CPU= 24.284
  01:11:01 STEP 1747 H= 436:45 +CPU= 26.019
  01:11:26 STEP 1748 H= 437:00 +CPU= 25.608
  01:11:51 STEP 1749 H= 437:15 +CPU= 24.565
  01:12:19 STEP 1750 H= 437:30 +CPU= 27.909
  01:12:48 STEP 1751 H= 437:45 +CPU= 28.529
  01:13:24 STEP 1752 H= 438:00 +CPU= 35.478
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMGGh7zg+001752
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMSHh7zg+001752
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMUAh7zg+001752
  01:13:52 STEP 1753 H= 438:15 +CPU= 27.384
  01:14:20 STEP 1754 H= 438:30 +CPU= 28.222
  01:14:47 STEP 1755 H= 438:45 +CPU= 26.338
  01:15:07 STEP 1756 H= 439:00 +CPU= 20.032
  01:15:28 STEP 1757 H= 439:15 +CPU= 20.255
  01:15:52 STEP 1758 H= 439:30 +CPU= 23.953
  01:16:16 STEP 1759 H= 439:45 +CPU= 24.055
  01:16:41 STEP 1760 H= 440:00 +CPU= 25.519
  01:17:06 STEP 1761 H= 440:15 +CPU= 24.916
  01:17:33 STEP 1762 H= 440:30 +CPU= 26.217
  01:17:59 STEP 1763 H= 440:45 +CPU= 25.946
  01:18:35 STEP 1764 H= 441:00 +CPU= 34.866
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMGGh7zg+001764
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMSHh7zg+001764
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMUAh7zg+001764
  01:19:04 STEP 1765 H= 441:15 +CPU= 28.426
  01:19:33 STEP 1766 H= 441:30 +CPU= 28.404
  01:20:02 STEP 1767 H= 441:45 +CPU= 28.248
  01:20:31 STEP 1768 H= 442:00 +CPU= 28.695
  01:20:56 STEP 1769 H= 442:15 +CPU= 24.402
  01:21:14 STEP 1770 H= 442:30 +CPU= 17.974
  01:21:31 STEP 1771 H= 442:45 +CPU= 17.519
  01:21:49 STEP 1772 H= 443:00 +CPU= 18.238
  01:22:15 STEP 1773 H= 443:15 +CPU= 25.510
  01:22:40 STEP 1774 H= 443:30 +CPU= 25.002
  01:23:05 STEP 1775 H= 443:45 +CPU= 24.613
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_bl_1.13_x86_64-pc-linux-gnu
(argv1) start_date: 2016092300
(argv2) exptid: h7zg
(argv3) unique_member_id: a0jo
(argv4) batchid: 1018
(argv5) wuid: 12289035
(argv6) fclen: 20
(argv7) app_name: oifs_43r3_bl
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/16
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_bl
version: 1.13
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289035
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/16/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/16/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_10aed612e9e726533dfaf58c02abf5fa to: /var/lib/boinc-client/slots/16/oifs_43r3_bl_a0jo_2016092300_20_1018_12289035.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/16/oifs_43r3_bl_a0jo_2016092300_20_1018_12289035.zip
ic_ancil_file: ic_ancil_12289035
ifsdata_file: ifsdata_12289035
climate_data_file: clim_data_12289035
horiz_resolution: 159
vert_resolution: 91
grid_type: l_2
upload_interval: 96
utstep: 900
nfrres: restart dump frequency (steps) 48
Copying IC ancils from: ../../projects/climateprediction.net/jf_f7ecfd27aa1e68bbe7e8d19800e25c73 to: /var/lib/boinc-client/slots/16/ic_ancil_12289035.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/16/ic_ancil_12289035.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_5fe2e2b063c91c30297ff53cb25f8fd9 to: /var/lib/boinc-client/slots/16/ifsdata/ifsdata_12289035.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/16/ifsdata/ifsdata_12289035.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/16/159l_2/clim_data_12289035.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/16/159l_2/clim_data_12289035.zip
Checking for progress XML file: /var/lib/boinc-client/slots/16/progress_file_12289035.xml
Opened progress file ok : /var/lib/boinc-client/slots/16/progress_file_12289035.xml
-- Model is restarting --
Adjusting last_iter, 1776, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/16/progress_file_12289035.xml
last_cpu_time: 45571
upload_file_number: 18
last_iter: 1777
last_upload: 1556100
model_completed: 0
total_length_of_simulation: 1728000
result_base_name: oifs_43r3_bl_a0jo_2016092300_20_1018_12289035_0_r1895645953
The child process has been launched with process id: 831882
Executing the command: /var/lib/boinc-client/slots/16/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:box.zenlime.org:1:1:831882:831882] [20240614:012738:1718328458:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2091] fp = 0x314f440
[EC_DRHOOK:box.zenlime.org:1:1:831882:831882] [20240614:012738:1718328458:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:box.zenlime.org:1:1:831882:831882] [20240614:012738:1718328458:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:box.zenlime.org:1:1:831882:831882] [20240614:012738:1718328458:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:box.zenlime.org:1:1:831882:831882] [20240614:012738:1718328458:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:box.zenlime.org:1:1:831882:831882] [20240614:012738:1718328458:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:box.zenlime.org:1:1:831882:831882] [20240614:012738:1718328458:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:box.zenlime.org:1:1:831882:831882] [20240614:012738:1718328458:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:box.zenlime.org:1:1:831882:831882] [20240614:012738:1718328458:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:box.zenlime.org:1:1:831882:831882] [20240614:012738:1718328458:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:box.zenlime.org:1:1:831882:831882] [20240614:012738:1718328458:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:box.zenlime.org:1:1:831882:831882] [20240614:012738:1718328458:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:box.zenlime.org:1:1:831882:831882] [20240614:012738:1718328458:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:box.zenlime.org:1:1:831882:831882] [20240614:012738:1718328458:0.000] [ignore_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:box.zenlime.org:1:1:831882:831882] [20240614:012738:1718328458:0.000] [restore_default_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:box.zenlime.org:1:1:831882:831882] [20240614:012738:1718328458:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:box.zenlime.org:1:1:831882:831882] [20240614:012738:1718328458:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:box.zenlime.org:1:1:831882:831882] [20240614:012738:1718328458:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:box.zenlime.org:1:1:831882:831882] [20240614:012738:1718328458:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x820cf0 (old at (nil))
[EC_DRHOOK:box.zenlime.org:1:1:831882:831882] [20240614:012738:1718328458:0.000] [trapfpe_treatment@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:box.zenlime.org:1:1:831882:831882] [20240614:012738:1718328458:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:box.zenlime.org:1:1:831882:831882] [20240614:012738:1718328458:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:box.zenlime.org:1:1:831882:831882] [20240614:012738:1718328458:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x820cf0 (old at (nil))
[EC_DRHOOK:box.zenlime.org:1:1:831882:831882] [20240614:012738:1718328458:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:box.zenlime.org:1:1:831882:831882] [20240614:012738:1718328458:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:box.zenlime.org:1:1:831882:831882] [20240614:012738:1718328458:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:box.zenlime.org:1:1:831882:831882] [20240614:012738:1718328458:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x820cf0 (old at (nil))
[EC_DRHOOK:box.zenlime.org:1:1:831882:831882] [20240614:012738:1718328458:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x820cf0 (old at (nil))
[EC_DRHOOK:box.zenlime.org:1:1:831882:831882] [20240614:012738:1718328458:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:box.zenlime.org:1:1:831882:831882] [20240614:012738:1718328458:0.000] [catch_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  01:28:09 STEP 1776 H= 444:00 +CPU= 27.309
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMGGh7zg+001776
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMSHh7zg+001776
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMUAh7zg+001776
  01:28:32 STEP 1777 H= 444:15 +CPU= 23.708
  01:28:57 STEP 1778 H= 444:30 +CPU= 24.618
  01:29:23 STEP 1779 H= 444:45 +CPU= 26.245
  01:29:52 STEP 1780 H= 445:00 +CPU= 27.755
  01:30:21 STEP 1781 H= 445:15 +CPU= 28.334
  01:30:49 STEP 1782 H= 445:30 +CPU= 27.446
  01:31:18 STEP 1783 H= 445:45 +CPU= 28.095
  01:31:46 STEP 1784 H= 446:00 +CPU= 28.353
  01:32:15 STEP 1785 H= 446:15 +CPU= 27.787
  01:32:44 STEP 1786 H= 446:30 +CPU= 28.698
  01:33:13 STEP 1787 H= 446:45 +CPU= 28.310
  01:33:47 STEP 1788 H= 447:00 +CPU= 33.623
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMGGh7zg+001788
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMSHh7zg+001788
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMUAh7zg+001788
  01:34:10 STEP 1789 H= 447:15 +CPU= 23.020
  01:34:34 STEP 1790 H= 447:30 +CPU= 23.317
  01:34:59 STEP 1791 H= 447:45 +CPU= 25.073
  01:35:24 STEP 1792 H= 448:00 +CPU= 25.379
  01:35:49 STEP 1793 H= 448:15 +CPU= 24.758
  01:36:16 STEP 1794 H= 448:30 +CPU= 26.402
  01:36:40 STEP 1795 H= 448:45 +CPU= 23.899
  01:37:10 STEP 1796 H= 449:00 +CPU= 28.640
  01:37:41 STEP 1797 H= 449:15 +CPU= 29.017
  01:38:10 STEP 1798 H= 449:30 +CPU= 28.400
  01:38:38 STEP 1799 H= 449:45 +CPU= 28.296
  01:39:14 STEP 1800 H= 450:00 +CPU= 35.892
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMGGh7zg+001800
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMSHh7zg+001800
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMUAh7zg+001800
  01:39:43 STEP 1801 H= 450:15 +CPU= 28.579
  01:40:10 STEP 1802 H= 450:30 +CPU= 26.562
  01:40:28 STEP 1803 H= 450:45 +CPU= 18.153
  01:40:48 STEP 1804 H= 451:00 +CPU= 19.757
  01:41:08 STEP 1805 H= 451:15 +CPU= 20.266
  01:41:34 STEP 1806 H= 451:30 +CPU= 25.463
  01:41:58 STEP 1807 H= 451:45 +CPU= 24.012
  01:42:22 STEP 1808 H= 452:00 +CPU= 24.596
  01:42:48 STEP 1809 H= 452:15 +CPU= 25.369
  01:43:13 STEP 1810 H= 452:30 +CPU= 24.873
  01:43:41 STEP 1811 H= 452:45 +CPU= 28.304
  01:44:17 STEP 1812 H= 453:00 +CPU= 34.903
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMGGh7zg+001812
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMSHh7zg+001812
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMUAh7zg+001812
  01:44:47 STEP 1813 H= 453:15 +CPU= 28.230
  01:45:16 STEP 1814 H= 453:30 +CPU= 28.454
  01:45:44 STEP 1815 H= 453:45 +CPU= 27.790
  01:46:11 STEP 1816 H= 454:00 +CPU= 26.508
  01:46:33 STEP 1817 H= 454:15 +CPU= 21.806
  01:46:58 STEP 1818 H= 454:30 +CPU= 25.166
  01:47:25 STEP 1819 H= 454:45 +CPU= 26.794
  01:47:51 STEP 1820 H= 455:00 +CPU= 26.000
  01:48:14 STEP 1821 H= 455:15 +CPU= 23.038
  01:48:40 STEP 1822 H= 455:30 +CPU= 25.608
  01:49:07 STEP 1823 H= 455:45 +CPU= 26.993
  01:49:46 STEP 1824 H= 456:00 +CPU= 38.213
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMGGh7zg+001824
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMSHh7zg+001824
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMUAh7zg+001824
  01:50:15 STEP 1825 H= 456:15 +CPU= 28.508
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289035/ICMGGh7zg+001740
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289035/ICMSHh7zg+001740
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289035/ICMUAh7zg+001740
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289035/ICMGGh7zg+001752
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289035/ICMSHh7zg+001752
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289035/ICMUAh7zg+001752
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289035/ICMGGh7zg+001764
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289035/ICMSHh7zg+001764
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289035/ICMUAh7zg+001764
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289035/ICMGGh7zg+001776
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289035/ICMSHh7zg+001776
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289035/ICMUAh7zg+001776
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289035/ICMGGh7zg+001788
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289035/ICMSHh7zg+001788
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289035/ICMUAh7zg+001788
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289035/ICMGGh7zg+001800
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289035/ICMSHh7zg+001800
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289035/ICMUAh7zg+001800
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289035/ICMGGh7zg+001812
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289035/ICMSHh7zg+001812
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289035/ICMUAh7zg+001812
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289035/ICMGGh7zg+001824
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289035/ICMSHh7zg+001824
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289035/ICMUAh7zg+001824
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a0jo_2016092300_20_1018_12289035_0_r1895645953_18.zip
Uploading the intermediate file: upload_file_18.zip
  01:50:45 STEP 1826 H= 456:30 +CPU= 28.415
  01:51:14 STEP 1827 H= 456:45 +CPU= 29.033
  01:51:44 STEP 1828 H= 457:00 +CPU= 29.508
  01:52:13 STEP 1829 H= 457:15 +CPU= 28.716
  01:52:41 STEP 1830 H= 457:30 +CPU= 27.426
  01:53:10 STEP 1831 H= 457:45 +CPU= 28.294
  01:53:38 STEP 1832 H= 458:00 +CPU= 28.646
  01:54:07 STEP 1833 H= 458:15 +CPU= 28.779
  01:54:37 STEP 1834 H= 458:30 +CPU= 29.022
  01:55:05 STEP 1835 H= 458:45 +CPU= 28.349
  01:55:41 STEP 1836 H= 459:00 +CPU= 35.043
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMGGh7zg+001836
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMSHh7zg+001836
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMUAh7zg+001836
  01:56:10 STEP 1837 H= 459:15 +CPU= 28.527
  01:56:38 STEP 1838 H= 459:30 +CPU= 28.725
  01:57:07 STEP 1839 H= 459:45 +CPU= 28.112
  01:57:28 STEP 1840 H= 460:00 +CPU= 21.156
  01:57:49 STEP 1841 H= 460:15 +CPU= 20.995
  01:58:12 STEP 1842 H= 460:30 +CPU= 22.079
  01:58:38 STEP 1843 H= 460:45 +CPU= 25.368
  01:59:05 STEP 1844 H= 461:00 +CPU= 26.737
  01:59:30 STEP 1845 H= 461:15 +CPU= 25.525
  01:59:53 STEP 1846 H= 461:30 +CPU= 22.488
  02:00:19 STEP 1847 H= 461:45 +CPU= 25.221
  02:00:54 STEP 1848 H= 462:00 +CPU= 35.074
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMGGh7zg+001848
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMSHh7zg+001848
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMUAh7zg+001848
  02:01:24 STEP 1849 H= 462:15 +CPU= 29.007
  02:01:53 STEP 1850 H= 462:30 +CPU= 28.989
  02:02:23 STEP 1851 H= 462:45 +CPU= 28.947
  02:02:52 STEP 1852 H= 463:00 +CPU= 29.273
  02:03:21 STEP 1853 H= 463:15 +CPU= 28.900
  02:03:49 STEP 1854 H= 463:30 +CPU= 27.861
  02:04:18 STEP 1855 H= 463:45 +CPU= 28.180
  02:04:47 STEP 1856 H= 464:00 +CPU= 28.960
  02:05:16 STEP 1857 H= 464:15 +CPU= 28.472
  02:05:45 STEP 1858 H= 464:30 +CPU= 28.500
  02:06:14 STEP 1859 H= 464:45 +CPU= 28.144
  02:06:50 STEP 1860 H= 465:00 +CPU= 34.913
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMGGh7zg+001860
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMSHh7zg+001860
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMUAh7zg+001860
  02:07:19 STEP 1861 H= 465:15 +CPU= 28.178
  02:07:48 STEP 1862 H= 465:30 +CPU= 28.806
  02:08:17 STEP 1863 H= 465:45 +CPU= 28.676
  02:08:43 STEP 1864 H= 466:00 +CPU= 25.450
  02:09:04 STEP 1865 H= 466:15 +CPU= 21.105
  02:09:27 STEP 1866 H= 466:30 +CPU= 22.755
  02:09:52 STEP 1867 H= 466:45 +CPU= 25.292
  02:10:17 STEP 1868 H= 467:00 +CPU= 24.544
  02:10:42 STEP 1869 H= 467:15 +CPU= 25.361
  02:11:09 STEP 1870 H= 467:30 +CPU= 25.410
  02:11:34 STEP 1871 H= 467:45 +CPU= 25.065
  02:12:13 STEP 1872 H= 468:00 +CPU= 37.628
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMGGh7zg+001872
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMSHh7zg+001872
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMUAh7zg+001872
  02:12:42 STEP 1873 H= 468:15 +CPU= 28.547
  02:13:10 STEP 1874 H= 468:30 +CPU= 28.472
  02:13:40 STEP 1875 H= 468:45 +CPU= 29.187
  02:14:09 STEP 1876 H= 469:00 +CPU= 29.122
  02:14:38 STEP 1877 H= 469:15 +CPU= 28.377
  02:15:00 STEP 1878 H= 469:30 +CPU= 22.191
  02:15:20 STEP 1879 H= 469:45 +CPU= 20.521
  02:15:40 STEP 1880 H= 470:00 +CPU= 19.223
  02:16:05 STEP 1881 H= 470:15 +CPU= 25.455
  02:16:30 STEP 1882 H= 470:30 +CPU= 24.912
  02:16:54 STEP 1883 H= 470:45 +CPU= 24.024
  02:17:23 STEP 1884 H= 471:00 +CPU= 29.104
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMGGh7zg+001884
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMSHh7zg+001884
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMUAh7zg+001884
  02:17:48 STEP 1885 H= 471:15 +CPU= 24.482
  02:18:17 STEP 1886 H= 471:30 +CPU= 28.333
  02:18:45 STEP 1887 H= 471:45 +CPU= 28.340
  02:19:15 STEP 1888 H= 472:00 +CPU= 29.017
  02:19:43 STEP 1889 H= 472:15 +CPU= 28.178
  02:20:11 STEP 1890 H= 472:30 +CPU= 28.302
  02:20:41 STEP 1891 H= 472:45 +CPU= 27.907
  02:21:06 STEP 1892 H= 473:00 +CPU= 24.966
  02:21:27 STEP 1893 H= 473:15 +CPU= 20.514
  02:21:48 STEP 1894 H= 473:30 +CPU= 20.796
  02:22:13 STEP 1895 H= 473:45 +CPU= 25.040
  02:22:43 STEP 1896 H= 474:00 +CPU= 29.927
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMGGh7zg+001896
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMSHh7zg+001896
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMUAh7zg+001896
  02:23:09 STEP 1897 H= 474:15 +CPU= 26.300
  02:23:34 STEP 1898 H= 474:30 +CPU= 25.171
  02:23:58 STEP 1899 H= 474:45 +CPU= 23.588
  02:24:25 STEP 1900 H= 475:00 +CPU= 27.463
  02:24:54 STEP 1901 H= 475:15 +CPU= 28.475
  02:25:24 STEP 1902 H= 475:30 +CPU= 28.902
  02:25:53 STEP 1903 H= 475:45 +CPU= 28.736
  02:26:22 STEP 1904 H= 476:00 +CPU= 28.858
  02:26:51 STEP 1905 H= 476:15 +CPU= 28.939
  02:27:19 STEP 1906 H= 476:30 +CPU= 27.571
  02:27:40 STEP 1907 H= 476:45 +CPU= 20.892
  02:28:06 STEP 1908 H= 477:00 +CPU= 25.495
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMGGh7zg+001908
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMSHh7zg+001908
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMUAh7zg+001908
  02:28:30 STEP 1909 H= 477:15 +CPU= 24.259
  02:28:54 STEP 1910 H= 477:30 +CPU= 24.438
  02:29:19 STEP 1911 H= 477:45 +CPU= 24.627
  02:29:45 STEP 1912 H= 478:00 +CPU= 25.384
  02:30:09 STEP 1913 H= 478:15 +CPU= 24.415
  02:30:38 STEP 1914 H= 478:30 +CPU= 27.943
  02:31:07 STEP 1915 H= 478:45 +CPU= 28.874
  02:31:36 STEP 1916 H= 479:00 +CPU= 28.878
  02:32:05 STEP 1917 H= 479:15 +CPU= 28.208
  02:32:34 STEP 1918 H= 479:30 +CPU= 28.555
  02:33:01 STEP 1919 H= 479:45 +CPU= 26.957
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_bl_1.13_x86_64-pc-linux-gnu
(argv1) start_date: 2016092300
(argv2) exptid: h7zg
(argv3) unique_member_id: a0jo
(argv4) batchid: 1018
(argv5) wuid: 12289035
(argv6) fclen: 20
(argv7) app_name: oifs_43r3_bl
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/16
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_bl
version: 1.13
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289035
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/16/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/16/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_10aed612e9e726533dfaf58c02abf5fa to: /var/lib/boinc-client/slots/16/oifs_43r3_bl_a0jo_2016092300_20_1018_12289035.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/16/oifs_43r3_bl_a0jo_2016092300_20_1018_12289035.zip
ic_ancil_file: ic_ancil_12289035
ifsdata_file: ifsdata_12289035
climate_data_file: clim_data_12289035
horiz_resolution: 159
vert_resolution: 91
grid_type: l_2
upload_interval: 96
utstep: 900
nfrres: restart dump frequency (steps) 48
Copying IC ancils from: ../../projects/climateprediction.net/jf_f7ecfd27aa1e68bbe7e8d19800e25c73 to: /var/lib/boinc-client/slots/16/ic_ancil_12289035.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/16/ic_ancil_12289035.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_5fe2e2b063c91c30297ff53cb25f8fd9 to: /var/lib/boinc-client/slots/16/ifsdata/ifsdata_12289035.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/16/ifsdata/ifsdata_12289035.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/16/159l_2/clim_data_12289035.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/16/159l_2/clim_data_12289035.zip
Checking for progress XML file: /var/lib/boinc-client/slots/16/progress_file_12289035.xml
Opened progress file ok : /var/lib/boinc-client/slots/16/progress_file_12289035.xml
-- Model is restarting --
Adjusting last_iter, 1920, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/16/progress_file_12289035.xml
last_cpu_time: 49458
upload_file_number: 19
last_iter: 1921
last_upload: 1642500
model_completed: 0
total_length_of_simulation: 1728000
result_base_name: oifs_43r3_bl_a0jo_2016092300_20_1018_12289035_0_r1895645953
The child process has been launched with process id: 834693
Executing the command: /var/lib/boinc-client/slots/16/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:box.zenlime.org:1:1:834693:834693] [20240614:023415:1718332455:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2091] fp = 0x314f440
[EC_DRHOOK:box.zenlime.org:1:1:834693:834693] [20240614:023415:1718332455:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:box.zenlime.org:1:1:834693:834693] [20240614:023415:1718332455:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:box.zenlime.org:1:1:834693:834693] [20240614:023415:1718332455:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:box.zenlime.org:1:1:834693:834693] [20240614:023415:1718332455:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:box.zenlime.org:1:1:834693:834693] [20240614:023415:1718332455:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:box.zenlime.org:1:1:834693:834693] [20240614:023415:1718332455:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:box.zenlime.org:1:1:834693:834693] [20240614:023415:1718332455:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:box.zenlime.org:1:1:834693:834693] [20240614:023415:1718332455:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:box.zenlime.org:1:1:834693:834693] [20240614:023415:1718332455:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:box.zenlime.org:1:1:834693:834693] [20240614:023415:1718332455:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:box.zenlime.org:1:1:834693:834693] [20240614:023415:1718332455:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:box.zenlime.org:1:1:834693:834693] [20240614:023415:1718332455:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:box.zenlime.org:1:1:834693:834693] [20240614:023415:1718332455:0.000] [ignore_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:box.zenlime.org:1:1:834693:834693] [20240614:023415:1718332455:0.000] [restore_default_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:box.zenlime.org:1:1:834693:834693] [20240614:023415:1718332455:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:box.zenlime.org:1:1:834693:834693] [20240614:023415:1718332455:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:box.zenlime.org:1:1:834693:834693] [20240614:023415:1718332455:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:box.zenlime.org:1:1:834693:834693] [20240614:023415:1718332455:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x820cf0 (old at (nil))
[EC_DRHOOK:box.zenlime.org:1:1:834693:834693] [20240614:023415:1718332455:0.000] [trapfpe_treatment@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:box.zenlime.org:1:1:834693:834693] [20240614:023415:1718332455:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:box.zenlime.org:1:1:834693:834693] [20240614:023415:1718332455:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:box.zenlime.org:1:1:834693:834693] [20240614:023415:1718332455:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x820cf0 (old at (nil))
[EC_DRHOOK:box.zenlime.org:1:1:834693:834693] [20240614:023415:1718332455:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:box.zenlime.org:1:1:834693:834693] [20240614:023415:1718332455:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:box.zenlime.org:1:1:834693:834693] [20240614:023415:1718332455:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:box.zenlime.org:1:1:834693:834693] [20240614:023415:1718332455:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x820cf0 (old at (nil))
[EC_DRHOOK:box.zenlime.org:1:1:834693:834693] [20240614:023415:1718332455:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x820cf0 (old at (nil))
[EC_DRHOOK:box.zenlime.org:1:1:834693:834693] [20240614:023415:1718332455:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:box.zenlime.org:1:1:834693:834693] [20240614:023415:1718332455:0.000] [catch_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  02:34:43 STEP 1920 H= 480:00 +CPU= 24.247
..The child process terminated with status: 0
>>> Printing last 70 lines from file: NODE.001_01

I      I Hor. Flux of PEII     -0.00000 I              I              I              I               I
I      I Vert.Flux of PEII      0.00000 I              I              I              I               I
I      I Work Pres.G.F  II              I      0.00926 I              I              I               I
I      I Conv. Ad.-Int. II     -0.00925 I              I              I              I               I
DDH--------------------------------------------------------------------------------------------------I
I DYN- Horizont. Flux   II      0.00090 I     -0.00007 I      0.01275 I      0.00005 I      0.00004  I
I    I Vertical Flux    II      0.00000 I      0.00000 I      0.00000 I      0.00000 I      0.00000  I
DDH--------------------------------------------------------------------------------------------------I
  
DDH----------------------------------------------------------------------------------------------------------------------------------
  I   - Mass balance    (10**(-6)  KG/M**2) 
  
  I INITIAL      I HORIZ. FLUX  I DYN. FLUX    I DYN. FLUX    I PHYS. FLUX   I PHYS. FLUX   I CURRENT      I I I RESIDUAL     I
  I MASS         I              I TOA          I SURFACE      I TOA          I SURFACE      I MASS         I I I              I
  ---------------------------------------------------------------------------------------------------------- I ----------------
  -              -              -              -              -              -              -              - I -              -
  - 10341776742. I          -0. .           0. .           0. .           0. .           0. I 10341808115. - I -       31372. -
  -              -              -              -              -              -              -              - I -              -
  ---------------------------------------------------------------------------------------------------------- I ----------------
 
DDH------------------------------------------------------------------------
I                                                                         I
I TIME =  480.000H DDH * Diagnostic Var.,     AREA                GLOBAL  I
I                                                                         I
I-------------------------------------------------------------------------I
I   VARIABLE   I      Initial value         I        Current value        I
I-------------------------------------------------------------------------I
I  OMEGA (P/S) I     -0.4063959055E-21      I       0.9433247012E-07      I
ICloud fractionI      0.0000000000E+00      I       0.1831165054E-02      I
I  Relat. Hum. I      0.3075287418E+00      I       0.3252756746E+00      I
I  PASS 01     I      0.0000000000E+00      I       0.4070895136E+02      I
DDH------------------------------------------------------------------------
  
 02:34:43 STEP 1920 H= 480:00 +CPU= 24.247
 END CNT3
 NSTEP =  1920 CNT0   000000000  24.172  24.172  24.377   0.992               0               0               0               0               0
IO-STREAM STATISTICS, TOTAL NO OF REC/BYTES READ           46/  1069125708 WRITTEN            0/           0
 -TASK-OPENED-OPEN-RECS IN -KBYTE IN    -RECS OUT-KBYTE OUT-WALL   -WALL IN-WALL
  OU-TOT IN -TOT OUT
   1      3    0       46      1044068        0            0     1.0     1.0     0.0     1.6     0.0
===-=== START OF TIMING STATISTICS ===-===
 
STATS FOR ALL TASKS
 NUM ROUTINE                                     CALLS  MEAN(ms)   MAX(ms)   FRAC(%)  UNBAL(%)
   0 CNT0     - COMPLETE EXECUTION                   1   27262.2   27262.2    100.00      0.00
   1 CNT4     - FORWARD INTEGRATION                  1   24450.9   24450.9     89.69      0.00
   8 SCAN2M - GRID-POINT DYNAMICS                    1    4724.6    4724.6     17.33      0.00
  10 SCAN2M - PHYSICS                                1   12469.2   12469.2     45.74      0.00
  12 SPNORM   - SPECTRAL NORM COMP.                  1      41.4      41.4      0.15      0.00
  28 RESTART FILES - READING                         1    2332.2    2332.2      8.55      0.00
  31 POSDDH IN STEPO                                 1       0.6       0.6      0.00      0.00
  37 CPGLAG   - SL COMPUTATIONS                      1    6311.4    6311.4     23.15      0.00
  39 SU0YOMB                                         1     310.9     310.9      1.14      0.00
  51 SCAN2M   - SL COMM. PART 1                      1      26.3      26.3      0.10      0.00
  56 SPNORM   - SPECTRAL NORM COMM.                  1       0.2       0.2      0.00      0.00
 102 LTINV_CTL   - INVERSE LEGENDRE TRANSFORM        4      94.3      94.3      1.38      0.00
 107 FTINV_CTL   - INVERSE FOURIER TRANSFORM         4      37.4      37.4      0.55      0.00
 140 SULEG       - COMP. OF LEGENDRE POL.            1      35.4      35.4      0.13      0.00
 152 LTINV_CTL   - M TO L TRANSPOSITION              4      15.8      15.8      0.23      0.00
 157 FTINV_CTL   - L TO G TRANSPOSITION              4      39.9      39.9      0.59      0.00
 400 GSTATS                                       1119       0.0       0.0      0.00      0.00
 401 GSTATS HOOK                                   987       0.0       0.0      0.00      0.00
TOTAL MEASURED IMBALANCE =       0.0 SECONDS,  0.0 PERCENT
TOTAL WALLCLOCK TIME       27.3 CPU TIME      27.1 VECTOR TIME       27.1


===-=== END   OF TIMING STATISTICS ===-===
FORECAST DAYS PER DAY    70672.2
  *** END CNT0 *** 
------------------------------------------------
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMGGh7zg+001920
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMSHh7zg+001920
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMUAh7zg+001920
Adding to the zip: /var/lib/boinc-client/slots/16/NODE.001_01
Adding to the zip: /var/lib/boinc-client/slots/16/ifs.stat
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289035/ICMGGh7zg+001884
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289035/ICMSHh7zg+001920
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289035/ICMGGh7zg+001632
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289035/ICMSHh7zg+001884
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289035/ICMUAh7zg+001884
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289035/ICMUAh7zg+001872
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289035/ICMGGh7zg+001896
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289035/ICMUAh7zg+001896
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289035/ICMUAh7zg+001908
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289035/ICMUAh7zg+001860
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289035/ICMGGh7zg+001908
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289035/ICMSHh7zg+001908
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289035/ICMUAh7zg+001836
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289035/ICMSHh7zg+001344
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289035/ICMUAh7zg+001344
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289035/ICMSHh7zg+001848
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289035/ICMGGh7zg+001848
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289035/ICMGGh7zg+001920
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289035/ICMGGh7zg+001872
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289035/ICMSHh7zg+001860
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289035/ICMSHh7zg+001872
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289035/ICMSHh7zg+001896
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289035/ICMUAh7zg+001920
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289035/ICMGGh7zg+001344
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289035/ICMGGh7zg+001836
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289035/ICMGGh7zg+001860
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289035/ICMSHh7zg+001836
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289035/ICMUAh7zg+001848
Zipping up the final file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a0jo_2016092300_20_1018_12289035_0_r1895645953_19.zip
Uploading the final file: upload_file_19.zip
Uploading trickle at timestep: 1728900
02:38:10 (834690): called boinc_finish(0)

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
14 Jun 2024 02:36:59 1537687 22442610 oifs_43r3_bl_a0jo_2016092300_20_1018_12289035_0 1,728,900 49,485 0.0286


©2024 cpdn.org