climateprediction.net home page
Task 22293210

Task 22293210

Name oifs_43r3_ps_0951_2020050100_123_989_12206595_0
Workunit 12206595
Created 25 Dec 2022, 3:18:52 UTC
Sent 4 Jan 2023, 12:54:19 UTC
Report deadline 3 Feb 2023, 12:54:19 UTC
Received 9 Jan 2023, 15:35:48 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 196 (0x000000C4) EXIT_DISK_LIMIT_EXCEEDED
Computer ID 1536947
Run time 19 hours 13 min 54 sec
CPU time 17 hours 57 min 10 sec
Validate state Invalid
Credit 8,830.08
Device peak FLOPS 1.00 GFLOPS
Application version OpenIFS 43r3 Perturbed Surface v1.05
x86_64-pc-linux-gnu
Peak working set size 4,799.02 MB
Peak swap size 4,980.82 MB
Peak disk usage 7,826.15 MB
Stderr
<core_client_version>7.18.1</core_client_version>
<![CDATA[
<message>
Disk usage limit exceeded</message>
<stderr_txt>
BOINC client, suspending the child process
Resuming the child process
  15:01:07 STEP 1683 H=1683:00 +CPU= 36.938
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:01:34 STEP 1684 H=1684:00 +CPU= 25.170
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:02:03 STEP 1685 H=1685:00 +CPU= 24.500
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:02:43 STEP 1686 H=1686:00 +CPU= 35.950
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:03:10 STEP 1687 H=1687:00 +CPU= 25.308
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:03:38 STEP 1688 H=1688:00 +CPU= 25.641
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:04:23 STEP 1689 H=1689:00 +CPU= 39.669
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:04:50 STEP 1690 H=1690:00 +CPU= 24.553
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:05:14 STEP 1691 H=1691:00 +CPU= 21.521
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:05:59 STEP 1692 H=1692:00 +CPU= 40.972
Moving to projects directory: /root/slots/1/ICMGGhq0f+001692
Moving to projects directory: /root/slots/1/ICMSHhq0f+001692
Moving to projects directory: /root/slots/1/ICMUAhq0f+001692
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:06:25 STEP 1693 H=1693:00 +CPU= 22.251
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:06:52 STEP 1694 H=1694:00 +CPU= 24.871
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:07:36 STEP 1695 H=1695:00 +CPU= 39.896
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:08:06 STEP 1696 H=1696:00 +CPU= 26.003
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:08:33 STEP 1697 H=1697:00 +CPU= 24.711
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:09:18 STEP 1698 H=1698:00 +CPU= 41.429
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:09:47 STEP 1699 H=1699:00 +CPU= 25.265
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:10:13 STEP 1700 H=1700:00 +CPU= 23.531
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:10:51 STEP 1701 H=1701:00 +CPU= 36.344
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:11:20 STEP 1702 H=1702:00 +CPU= 24.774
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:11:48 STEP 1703 H=1703:00 +CPU= 25.667
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:12:37 STEP 1704 H=1704:00 +CPU= 44.845
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /root/slots/1/ICMGGhq0f+001704
Moving to projects directory: /root/slots/1/ICMSHhq0f+001704
Moving to projects directory: /root/slots/1/ICMUAhq0f+001704
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12206595/ICMGGhq0f+001680
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12206595/ICMSHhq0f+001680
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12206595/ICMUAhq0f+001680
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12206595/ICMGGhq0f+001692
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12206595/ICMSHhq0f+001692
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12206595/ICMUAhq0f+001692
Zipping up the intermediate file: /root/projects/climateprediction.net/oifs_43r3_ps_0951_2020050100_123_989_12206595_0_r1690545410_70.zip
Uploading the intermediate file: upload_file_70.zip
  15:13:03 STEP 1705 H=1705:00 +CPU= 23.667
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:13:29 STEP 1706 H=1706:00 +CPU= 24.165
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:14:11 STEP 1707 H=1707:00 +CPU= 38.070
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:14:38 STEP 1708 H=1708:00 +CPU= 24.956
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:15:07 STEP 1709 H=1709:00 +CPU= 24.852
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:15:49 STEP 1710 H=1710:00 +CPU= 38.067
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:16:15 STEP 1711 H=1711:00 +CPU= 24.057
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:16:43 STEP 1712 H=1712:00 +CPU= 23.761
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:17:25 STEP 1713 H=1713:00 +CPU= 37.704
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:17:53 STEP 1714 H=1714:00 +CPU= 25.482
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:18:21 STEP 1715 H=1715:00 +CPU= 24.888
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:19:08 STEP 1716 H=1716:00 +CPU= 42.230
Moving to projects directory: /root/slots/1/ICMGGhq0f+001716
Moving to projects directory: /root/slots/1/ICMSHhq0f+001716
Moving to projects directory: /root/slots/1/ICMUAhq0f+001716
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:19:39 STEP 1717 H=1717:00 +CPU= 26.199
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:20:09 STEP 1718 H=1718:00 +CPU= 25.550
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:20:52 STEP 1719 H=1719:00 +CPU= 37.416
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:21:22 STEP 1720 H=1720:00 +CPU= 24.511
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:21:47 STEP 1721 H=1721:00 +CPU= 22.998
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:22:29 STEP 1722 H=1722:00 +CPU= 36.832
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:22:56 STEP 1723 H=1723:00 +CPU= 24.548
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:23:24 STEP 1724 H=1724:00 +CPU= 23.435
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:24:06 STEP 1725 H=1725:00 +CPU= 36.492
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:24:33 STEP 1726 H=1726:00 +CPU= 24.396
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:25:02 STEP 1727 H=1727:00 +CPU= 24.217
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:25:49 STEP 1728 H=1728:00 +CPU= 42.015
Moving to projects directory: /root/slots/1/ICMGGhq0f+001728
Moving to projects directory: /root/slots/1/ICMSHhq0f+001728
Moving to projects directory: /root/slots/1/ICMUAhq0f+001728
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12206595/ICMGGhq0f+001704
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12206595/ICMSHhq0f+001704
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12206595/ICMUAhq0f+001704
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12206595/ICMGGhq0f+001716
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12206595/ICMSHhq0f+001716
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12206595/ICMUAhq0f+001716
Zipping up the intermediate file: /root/projects/climateprediction.net/oifs_43r3_ps_0951_2020050100_123_989_12206595_0_r1690545410_71.zip
Uploading the intermediate file: upload_file_71.zip
  15:26:13 STEP 1729 H=1729:00 +CPU= 22.856
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:26:41 STEP 1730 H=1730:00 +CPU= 23.710
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:27:22 STEP 1731 H=1731:00 +CPU= 35.874
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:27:49 STEP 1732 H=1732:00 +CPU= 24.337
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:28:14 STEP 1733 H=1733:00 +CPU= 23.017
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:28:56 STEP 1734 H=1734:00 +CPU= 36.439
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:29:24 STEP 1735 H=1735:00 +CPU= 23.234
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:29:50 STEP 1736 H=1736:00 +CPU= 23.882
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:30:31 STEP 1737 H=1737:00 +CPU= 35.903
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:30:57 STEP 1738 H=1738:00 +CPU= 23.697
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:31:24 STEP 1739 H=1739:00 +CPU= 22.573
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:32:08 STEP 1740 H=1740:00 +CPU= 39.622
Moving to projects directory: /root/slots/1/ICMGGhq0f+001740
Moving to projects directory: /root/slots/1/ICMSHhq0f+001740
Moving to projects directory: /root/slots/1/ICMUAhq0f+001740
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:32:33 STEP 1741 H=1741:00 +CPU= 22.535
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:32:59 STEP 1742 H=1742:00 +CPU= 23.620
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:33:40 STEP 1743 H=1743:00 +CPU= 34.939
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:34:06 STEP 1744 H=1744:00 +CPU= 23.162
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:34:31 STEP 1745 H=1745:00 +CPU= 22.920
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:35:13 STEP 1746 H=1746:00 +CPU= 36.424
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:35:38 STEP 1747 H=1747:00 +CPU= 23.091
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:36:06 STEP 1748 H=1748:00 +CPU= 23.184
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:36:45 STEP 1749 H=1749:00 +CPU= 34.969
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:37:12 STEP 1750 H=1750:00 +CPU= 24.075
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:37:38 STEP 1751 H=1751:00 +CPU= 23.572
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:38:25 STEP 1752 H=1752:00 +CPU= 41.034
Moving to projects directory: /root/slots/1/ICMGGhq0f+001752
Moving to projects directory: /root/slots/1/ICMSHhq0f+001752
Moving to projects directory: /root/slots/1/ICMUAhq0f+001752
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12206595/ICMGGhq0f+001728
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12206595/ICMSHhq0f+001728
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12206595/ICMUAhq0f+001728
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12206595/ICMGGhq0f+001740
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12206595/ICMSHhq0f+001740
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12206595/ICMUAhq0f+001740
Zipping up the intermediate file: /root/projects/climateprediction.net/oifs_43r3_ps_0951_2020050100_123_989_12206595_0_r1690545410_72.zip
Uploading the intermediate file: upload_file_72.zip
  15:38:48 STEP 1753 H=1753:00 +CPU= 22.141
Uploading trickle at timestep: 6307200
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:39:14 STEP 1754 H=1754:00 +CPU= 23.708
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:39:54 STEP 1755 H=1755:00 +CPU= 35.856
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:40:22 STEP 1756 H=1756:00 +CPU= 23.610
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:40:47 STEP 1757 H=1757:00 +CPU= 22.675
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:41:27 STEP 1758 H=1758:00 +CPU= 34.883
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:41:53 STEP 1759 H=1759:00 +CPU= 23.465
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:42:22 STEP 1760 H=1760:00 +CPU= 23.831
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:43:02 STEP 1761 H=1761:00 +CPU= 35.319
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:43:28 STEP 1762 H=1762:00 +CPU= 23.630
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:43:53 STEP 1763 H=1763:00 +CPU= 22.562
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:44:36 STEP 1764 H=1764:00 +CPU= 38.478
Moving to projects directory: /root/slots/1/ICMGGhq0f+001764
Moving to projects directory: /root/slots/1/ICMSHhq0f+001764
Moving to projects directory: /root/slots/1/ICMUAhq0f+001764
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:45:03 STEP 1765 H=1765:00 +CPU= 22.806
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:45:29 STEP 1766 H=1766:00 +CPU= 22.980
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:46:10 STEP 1767 H=1767:00 +CPU= 36.056
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:46:36 STEP 1768 H=1768:00 +CPU= 23.940
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:47:04 STEP 1769 H=1769:00 +CPU= 22.974
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:47:45 STEP 1770 H=1770:00 +CPU= 36.514
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:48:10 STEP 1771 H=1771:00 +CPU= 22.740
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:49:14 STEP 1772 H=1772:00 +CPU= 24.649
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:51:26 STEP 1773 H=1773:00 +CPU= 38.735
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:51:57 STEP 1774 H=1774:00 +CPU= 25.854
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:52:26 STEP 1775 H=1775:00 +CPU= 26.851
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:53:07 STEP 1776 H=1776:00 +CPU= 36.335
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /root/slots/1/ICMGGhq0f+001776
Moving to projects directory: /root/slots/1/ICMSHhq0f+001776
Moving to projects directory: /root/slots/1/ICMUAhq0f+001776
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12206595/ICMGGhq0f+001752
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12206595/ICMSHhq0f+001752
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12206595/ICMUAhq0f+001752
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12206595/ICMGGhq0f+001764
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12206595/ICMSHhq0f+001764
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12206595/ICMUAhq0f+001764
Zipping up the intermediate file: /root/projects/climateprediction.net/oifs_43r3_ps_0951_2020050100_123_989_12206595_0_r1690545410_73.zip
Uploading the intermediate file: upload_file_73.zip
  15:53:27 STEP 1777 H=1777:00 +CPU= 18.687
  15:53:48 STEP 1778 H=1778:00 +CPU= 19.967
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:54:24 STEP 1779 H=1779:00 +CPU= 31.392
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:54:48 STEP 1780 H=1780:00 +CPU= 21.284
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:55:14 STEP 1781 H=1781:00 +CPU= 21.085
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:56:21 STEP 1782 H=1782:00 +CPU= 37.055
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:56:48 STEP 1783 H=1783:00 +CPU= 24.169
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:57:15 STEP 1784 H=1784:00 +CPU= 24.775
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:57:55 STEP 1785 H=1785:00 +CPU= 37.421
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:58:24 STEP 1786 H=1786:00 +CPU= 24.758
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:58:51 STEP 1787 H=1787:00 +CPU= 24.067
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:59:42 STEP 1788 H=1788:00 +CPU= 43.588
Moving to projects directory: /root/slots/1/ICMGGhq0f+001788
Moving to projects directory: /root/slots/1/ICMSHhq0f+001788
Moving to projects directory: /root/slots/1/ICMUAhq0f+001788
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:00:11 STEP 1789 H=1789:00 +CPU= 25.215
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:00:44 STEP 1790 H=1790:00 +CPU= 26.259
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:01:27 STEP 1791 H=1791:00 +CPU= 38.624
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:01:54 STEP 1792 H=1792:00 +CPU= 24.386
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:02:18 STEP 1793 H=1793:00 +CPU= 22.610
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:03:03 STEP 1794 H=1794:00 +CPU= 39.019
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:03:30 STEP 1795 H=1795:00 +CPU= 24.721
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:04:01 STEP 1796 H=1796:00 +CPU= 26.911
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:04:38 STEP 1797 H=1797:00 +CPU= 34.792
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:05:04 STEP 1798 H=1798:00 +CPU= 21.780
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:05:31 STEP 1799 H=1799:00 +CPU= 25.763
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:06:21 STEP 1800 H=1800:00 +CPU= 43.345
Moving to projects directory: /root/slots/1/ICMGGhq0f+001800
Moving to projects directory: /root/slots/1/ICMSHhq0f+001800
Moving to projects directory: /root/slots/1/ICMUAhq0f+001800
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12206595/ICMGGhq0f+001776
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12206595/ICMSHhq0f+001776
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12206595/ICMUAhq0f+001776
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12206595/ICMGGhq0f+001788
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12206595/ICMSHhq0f+001788
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12206595/ICMUAhq0f+001788
Zipping up the intermediate file: /root/projects/climateprediction.net/oifs_43r3_ps_0951_2020050100_123_989_12206595_0_r1690545410_74.zip
Uploading the intermediate file: upload_file_74.zip
  16:06:42 STEP 1801 H=1801:00 +CPU= 21.225
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:07:08 STEP 1802 H=1802:00 +CPU= 23.447
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:07:50 STEP 1803 H=1803:00 +CPU= 37.608
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:08:18 STEP 1804 H=1804:00 +CPU= 26.281
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:08:45 STEP 1805 H=1805:00 +CPU= 23.068
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:09:27 STEP 1806 H=1806:00 +CPU= 38.042
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:09:53 STEP 1807 H=1807:00 +CPU= 23.965
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:10:23 STEP 1808 H=1808:00 +CPU= 25.931
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:11:05 STEP 1809 H=1809:00 +CPU= 37.407
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:11:34 STEP 1810 H=1810:00 +CPU= 27.473
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:12:01 STEP 1811 H=1811:00 +CPU= 22.920
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:12:44 STEP 1812 H=1812:00 +CPU= 38.647
Moving to projects directory: /root/slots/1/ICMGGhq0f+001812
Moving to projects directory: /root/slots/1/ICMSHhq0f+001812
Moving to projects directory: /root/slots/1/ICMUAhq0f+001812
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:13:06 STEP 1813 H=1813:00 +CPU= 19.778
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:13:28 STEP 1814 H=1814:00 +CPU= 20.534
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:14:02 STEP 1815 H=1815:00 +CPU= 30.117
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:14:27 STEP 1816 H=1816:00 +CPU= 22.439
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:14:49 STEP 1817 H=1817:00 +CPU= 20.403
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:15:25 STEP 1818 H=1818:00 +CPU= 31.783
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:15:47 STEP 1819 H=1819:00 +CPU= 19.908
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:16:10 STEP 1820 H=1820:00 +CPU= 20.409
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:16:46 STEP 1821 H=1821:00 +CPU= 32.279
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:17:09 STEP 1822 H=1822:00 +CPU= 21.096
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:17:33 STEP 1823 H=1823:00 +CPU= 22.300
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:18:12 STEP 1824 H=1824:00 +CPU= 34.883
Moving to projects directory: /root/slots/1/ICMGGhq0f+001824
Moving to projects directory: /root/slots/1/ICMSHhq0f+001824
Moving to projects directory: /root/slots/1/ICMUAhq0f+001824
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12206595/ICMGGhq0f+001800
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12206595/ICMSHhq0f+001800
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12206595/ICMUAhq0f+001800
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12206595/ICMGGhq0f+001812
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12206595/ICMSHhq0f+001812
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12206595/ICMUAhq0f+001812
Zipping up the intermediate file: /root/projects/climateprediction.net/oifs_43r3_ps_0951_2020050100_123_989_12206595_0_r1690545410_75.zip
Uploading the intermediate file: upload_file_75.zip
  16:18:32 STEP 1825 H=1825:00 +CPU= 20.166
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:18:56 STEP 1826 H=1826:00 +CPU= 21.173
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:19:31 STEP 1827 H=1827:00 +CPU= 30.844
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:19:55 STEP 1828 H=1828:00 +CPU= 22.075
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:20:17 STEP 1829 H=1829:00 +CPU= 20.461
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:20:56 STEP 1830 H=1830:00 +CPU= 34.831
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:21:19 STEP 1831 H=1831:00 +CPU= 20.572
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:21:43 STEP 1832 H=1832:00 +CPU= 20.257
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:22:17 STEP 1833 H=1833:00 +CPU= 31.491
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:22:43 STEP 1834 H=1834:00 +CPU= 22.411
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:23:06 STEP 1835 H=1835:00 +CPU= 21.121
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:23:44 STEP 1836 H=1836:00 +CPU= 33.783
Moving to projects directory: /root/slots/1/ICMGGhq0f+001836
Moving to projects directory: /root/slots/1/ICMSHhq0f+001836
Moving to projects directory: /root/slots/1/ICMUAhq0f+001836
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:24:07 STEP 1837 H=1837:00 +CPU= 20.568
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:24:28 STEP 1838 H=1838:00 +CPU= 19.818
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:25:06 STEP 1839 H=1839:00 +CPU= 33.465
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:25:29 STEP 1840 H=1840:00 +CPU= 21.448
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:25:53 STEP 1841 H=1841:00 +CPU= 21.643
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:26:29 STEP 1842 H=1842:00 +CPU= 32.071
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:26:50 STEP 1843 H=1843:00 +CPU= 19.145
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:27:11 STEP 1844 H=1844:00 +CPU= 18.680
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:27:48 STEP 1845 H=1845:00 +CPU= 32.505
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:28:12 STEP 1846 H=1846:00 +CPU= 22.165
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:28:36 STEP 1847 H=1847:00 +CPU= 22.148
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:29:17 STEP 1848 H=1848:00 +CPU= 36.576
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /root/slots/1/ICMGGhq0f+001848
Moving to projects directory: /root/slots/1/ICMSHhq0f+001848
Moving to projects directory: /root/slots/1/ICMUAhq0f+001848
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12206595/ICMGGhq0f+001824
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12206595/ICMSHhq0f+001824
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12206595/ICMUAhq0f+001824
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12206595/ICMGGhq0f+001836
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12206595/ICMSHhq0f+001836
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12206595/ICMUAhq0f+001836
Zipping up the intermediate file: /root/projects/climateprediction.net/oifs_43r3_ps_0951_2020050100_123_989_12206595_0_r1690545410_76.zip
Uploading the intermediate file: upload_file_76.zip
  16:29:39 STEP 1849 H=1849:00 +CPU= 20.122
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:30:01 STEP 1850 H=1850:00 +CPU= 19.902
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:30:36 STEP 1851 H=1851:00 +CPU= 32.842
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:30:57 STEP 1852 H=1852:00 +CPU= 19.130
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:31:18 STEP 1853 H=1853:00 +CPU= 19.480
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:31:55 STEP 1854 H=1854:00 +CPU= 33.019
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:32:21 STEP 1855 H=1855:00 +CPU= 21.869
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:32:44 STEP 1856 H=1856:00 +CPU= 20.623
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:33:18 STEP 1857 H=1857:00 +CPU= 32.173
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:33:43 STEP 1858 H=1858:00 +CPU= 20.351
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:34:08 STEP 1859 H=1859:00 +CPU= 21.185
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:35:15 STEP 1860 H=1860:00 +CPU= 38.744
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /root/slots/1/ICMGGhq0f+001860
Moving to projects directory: /root/slots/1/ICMSHhq0f+001860
Moving to projects directory: /root/slots/1/ICMUAhq0f+001860
  16:35:39 STEP 1861 H=1861:00 +CPU= 21.221
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:36:04 STEP 1862 H=1862:00 +CPU= 21.583
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:36:38 STEP 1863 H=1863:00 +CPU= 31.601
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:37:03 STEP 1864 H=1864:00 +CPU= 19.895
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:37:23 STEP 1865 H=1865:00 +CPU= 18.305
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:37:58 STEP 1866 H=1866:00 +CPU= 32.776
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Quit request received from the BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 2020050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0951
(argv4) batchid: 989
(argv5) wuid: 12206595
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /root/slots/1
Project directory is: /root/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /root/projects/climateprediction.net/oifs_43r3_ps_12206595
..mkdir for temp folder for results failed
Copying: /root/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /root/slots/1/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /root/slots/1/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_afc71b691a5c5260b8b8002ca8706ef5 to: /root/slots/1/oifs_43r3_ps_0951_2020050100_123_989_12206595.zip
Unzipping the namelist zip file: /root/slots/1/oifs_43r3_ps_0951_2020050100_123_989_12206595.zip
ic_ancil_file: ic_ancil_12206595
ifsdata_file: ifsdata_12206595
climate_data_file: clim_data_12206595
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_26bf94ed8288231480f75d7501c6435c to: /root/slots/1/ic_ancil_12206595.zip
Unzipping the IC ancils zip file: /root/slots/1/ic_ancil_12206595.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_b575e596ddb5bf01aab521127ce16899 to: /root/slots/1/ifsdata/ifsdata_12206595.zip
Unzipping the ifsdata_zip file: /root/slots/1/ifsdata/ifsdata_12206595.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /root/slots/1/159l_2/clim_data_12206595.zip
Unzipping the climate data zip file: /root/slots/1/159l_2/clim_data_12206595.zip
Checking for progress XML file: /root/slots/1/progress_file_12206595.xml
Opened progress file ok : /root/slots/1/progress_file_12206595.xml
-- Model is restarting --
Adjusting last_iter, 1867, to previous model restart step.
Creating progress file: /root/slots/1/progress_file_12206595.xml
last_cpu_time: 63837
upload_file_number: 77
last_iter: 1847
last_upload: 6652800
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0951_2020050100_123_989_12206595_0_r1690545410
The child process has been launched with process id: 7671
Executing the command: /root/slots/1/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:52-dienstlich:1:1:7671:7671] [20230109:161343:1673277223:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:52-dienstlich:1:1:7671:7671] [20230109:161343:1673277223:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:52-dienstlich:1:1:7671:7671] [20230109:161343:1673277223:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:52-dienstlich:1:1:7671:7671] [20230109:161343:1673277223:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:52-dienstlich:1:1:7671:7671] [20230109:161343:1673277223:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:52-dienstlich:1:1:7671:7671] [20230109:161343:1673277223:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:52-dienstlich:1:1:7671:7671] [20230109:161343:1673277223:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:52-dienstlich:1:1:7671:7671] [20230109:161343:1673277223:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:52-dienstlich:1:1:7671:7671] [20230109:161343:1673277223:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:52-dienstlich:1:1:7671:7671] [20230109:161343:1673277223:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:52-dienstlich:1:1:7671:7671] [20230109:161343:1673277223:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:52-dienstlich:1:1:7671:7671] [20230109:161343:1673277223:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:52-dienstlich:1:1:7671:7671] [20230109:161343:1673277223:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:52-dienstlich:1:1:7671:7671] [20230109:161343:1673277223:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:52-dienstlich:1:1:7671:7671] [20230109:161343:1673277223:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:52-dienstlich:1:1:7671:7671] [20230109:161343:1673277223:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:52-dienstlich:1:1:7671:7671] [20230109:161343:1673277223:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:52-dienstlich:1:1:7671:7671] [20230109:161343:1673277223:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:52-dienstlich:1:1:7671:7671] [20230109:161343:1673277223:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:52-dienstlich:1:1:7671:7671] [20230109:161343:1673277223:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:52-dienstlich:1:1:7671:7671] [20230109:161343:1673277223:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:52-dienstlich:1:1:7671:7671] [20230109:161343:1673277223:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:52-dienstlich:1:1:7671:7671] [20230109:161343:1673277223:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:52-dienstlich:1:1:7671:7671] [20230109:161343:1673277223:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:52-dienstlich:1:1:7671:7671] [20230109:161343:1673277223:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:52-dienstlich:1:1:7671:7671] [20230109:161343:1673277223:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:52-dienstlich:1:1:7671:7671] [20230109:161343:1673277223:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:52-dienstlich:1:1:7671:7671] [20230109:161343:1673277223:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:52-dienstlich:1:1:7671:7671] [20230109:161343:1673277223:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:52-dienstlich:1:1:7671:7671] [20230109:161343:1673277223:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:14:28 STEP 1848 H=1848:00 +CPU= 36.085
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:14:55 STEP 1849 H=1849:00 +CPU= 22.861
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:15:23 STEP 1850 H=1850:00 +CPU= 22.871
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:16:02 STEP 1851 H=1851:00 +CPU= 33.679
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:16:27 STEP 1852 H=1852:00 +CPU= 21.789
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:17:16 STEP 1853 H=1853:00 +CPU= 22.742
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:18:03 STEP 1854 H=1854:00 +CPU= 37.934
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:18:43 STEP 1855 H=1855:00 +CPU= 23.737
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:19:49 STEP 1856 H=1856:00 +CPU= 26.037
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:20:44 STEP 1857 H=1857:00 +CPU= 35.260
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:21:12 STEP 1858 H=1858:00 +CPU= 23.124
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:21:37 STEP 1859 H=1859:00 +CPU= 21.936
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:22:19 STEP 1860 H=1860:00 +CPU= 36.948
Moving to projects directory: /root/slots/1/ICMGGhq0f+001860
Moving to projects directory: /root/slots/1/ICMSHhq0f+001860
Moving to projects directory: /root/slots/1/ICMUAhq0f+001860
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:22:59 STEP 1861 H=1861:00 +CPU= 23.100
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:23:42 STEP 1862 H=1862:00 +CPU= 24.158
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:24:24 STEP 1863 H=1863:00 +CPU= 36.268
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:25:06 STEP 1864 H=1864:00 +CPU= 24.076
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:25:49 STEP 1865 H=1865:00 +CPU= 25.008
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:26:50 STEP 1866 H=1866:00 +CPU= 40.671
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:27:34 STEP 1867 H=1867:00 +CPU= 25.543
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:28:26 STEP 1868 H=1868:00 +CPU= 26.437
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:29:38 STEP 1869 H=1869:00 +CPU= 39.962
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:30:32 STEP 1870 H=1870:00 +CPU= 26.076
  16:30:59 STEP 1871 H=1871:00 +CPU= 25.301
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:32:17 STEP 1872 H=1872:00 +CPU= 45.893
Moving to projects directory: /root/slots/1/ICMGGhq0f+001872
Moving to projects directory: /root/slots/1/ICMSHhq0f+001872
Moving to projects directory: /root/slots/1/ICMUAhq0f+001872
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12206595/ICMGGhq0f+001848
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12206595/ICMSHhq0f+001848
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12206595/ICMUAhq0f+001848
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12206595/ICMGGhq0f+001860
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12206595/ICMSHhq0f+001860
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12206595/ICMUAhq0f+001860
Zipping up the intermediate file: /root/projects/climateprediction.net/oifs_43r3_ps_0951_2020050100_123_989_12206595_0_r1690545410_77.zip
Uploading the intermediate file: upload_file_77.zip
  16:32:38 STEP 1873 H=1873:00 +CPU= 19.715
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:33:18 STEP 1874 H=1874:00 +CPU= 24.659
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Abort request received from BOINC client, ending the child process

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 May 2023 16:12:50 1536947 22293210 oifs_43r3_ps_0951_2020050100_123_989_12206595_0 6,307,200 60,795 0.0096
09 May 2023 16:12:50 1536947 22293210 oifs_43r3_ps_0951_2020050100_123_989_12206595_0 6,307,200 60,795 0.0096
09 May 2023 16:12:44 1536947 22293210 oifs_43r3_ps_0951_2020050100_123_989_12206595_0 5,184,000 51,156 0.0099
09 May 2023 16:09:21 1536947 22293210 oifs_43r3_ps_0951_2020050100_123_989_12206595_0 3,715,200 37,701 0.0101
09 May 2023 16:09:10 1536947 22293210 oifs_43r3_ps_0951_2020050100_123_989_12206595_0 2,851,200 28,669 0.0101
09 May 2023 16:08:57 1536947 22293210 oifs_43r3_ps_0951_2020050100_123_989_12206595_0 1,814,400 17,727 0.0098


©2024 cpdn.org