climateprediction.net home page
Task 22301142

Task 22301142

Name oifs_43r3_ps_0747_1986050100_123_955_12172391_2
Workunit 12172391
Created 21 Jan 2023, 14:19:35 UTC
Sent 21 Jan 2023, 14:31:36 UTC
Report deadline 22 Mar 2023, 14:31:36 UTC
Received 22 Jan 2023, 13:24:06 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 196 (0x000000C4) EXIT_DISK_LIMIT_EXCEEDED
Computer ID 1491072
Run time 21 hours 22 min 17 sec
CPU time 20 hours 55 min 34 sec
Validate state Invalid
Credit 3,749.76
Device peak FLOPS 4.70 GFLOPS
Application version OpenIFS 43r3 Perturbed Surface v1.05
x86_64-pc-linux-gnu
Peak working set size 4,409.98 MB
Peak swap size 4,978.08 MB
Peak disk usage 7,189.15 MB
Stderr
<core_client_version>7.16.16</core_client_version>
<![CDATA[
<message>
Disk usage limit exceeded</message>
<stderr_txt>
03:39 STEP  711 H= 711:00 +CPU= 83.947
  04:04:34 STEP  712 H= 712:00 +CPU= 53.898
  04:05:29 STEP  713 H= 713:00 +CPU= 53.352
  04:06:56 STEP  714 H= 714:00 +CPU= 86.124
  04:07:50 STEP  715 H= 715:00 +CPU= 52.593
  04:08:45 STEP  716 H= 716:00 +CPU= 54.484
  04:10:10 STEP  717 H= 717:00 +CPU= 83.691
  04:11:06 STEP  718 H= 718:00 +CPU= 54.880
  04:12:02 STEP  719 H= 719:00 +CPU= 55.267
  04:13:43 STEP  720 H= 720:00 +CPU= 98.373
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+000720
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+000720
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+000720
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMGGhq0f+000696
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMSHhq0f+000696
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMUAhq0f+000696
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMGGhq0f+000708
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMSHhq0f+000708
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMUAhq0f+000708
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0747_1986050100_123_955_12172391_2_r1416727145_29.zip
Uploading the intermediate file: upload_file_29.zip
  04:14:36 STEP  721 H= 721:00 +CPU= 52.224
  04:15:32 STEP  722 H= 722:00 +CPU= 54.526
  04:16:56 STEP  723 H= 723:00 +CPU= 83.584
  04:17:52 STEP  724 H= 724:00 +CPU= 54.184
  04:18:45 STEP  725 H= 725:00 +CPU= 52.865
  04:20:12 STEP  726 H= 726:00 +CPU= 85.536
  04:21:06 STEP  727 H= 727:00 +CPU= 52.937
  04:22:01 STEP  728 H= 728:00 +CPU= 54.206
  04:23:27 STEP  729 H= 729:00 +CPU= 84.299
  04:24:23 STEP  730 H= 730:00 +CPU= 54.976
  04:25:16 STEP  731 H= 731:00 +CPU= 52.252
  04:26:52 STEP  732 H= 732:00 +CPU= 94.126
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+000732
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+000732
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+000732
  04:27:46 STEP  733 H= 733:00 +CPU= 53.170
  04:28:41 STEP  734 H= 734:00 +CPU= 53.971
  04:30:07 STEP  735 H= 735:00 +CPU= 84.043
  04:31:02 STEP  736 H= 736:00 +CPU= 54.438
  04:31:55 STEP  737 H= 737:00 +CPU= 52.041
  04:33:23 STEP  738 H= 738:00 +CPU= 86.493
  04:34:17 STEP  739 H= 739:00 +CPU= 53.085
  04:35:11 STEP  740 H= 740:00 +CPU= 52.837
  04:36:37 STEP  741 H= 741:00 +CPU= 85.067
  04:37:33 STEP  742 H= 742:00 +CPU= 54.637
  04:38:27 STEP  743 H= 743:00 +CPU= 53.401
  04:40:09 STEP  744 H= 744:00 +CPU= 99.287
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+000744
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+000744
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+000744
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMGGhq0f+000720
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMSHhq0f+000720
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMUAhq0f+000720
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMGGhq0f+000732
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMSHhq0f+000732
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMUAhq0f+000732
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0747_1986050100_123_955_12172391_2_r1416727145_30.zip
Uploading the intermediate file: upload_file_30.zip
Uploading trickle at timestep: 2678400
  04:41:03 STEP  745 H= 745:00 +CPU= 52.621
  04:41:57 STEP  746 H= 746:00 +CPU= 53.921
  04:43:23 STEP  747 H= 747:00 +CPU= 84.016
  04:44:19 STEP  748 H= 748:00 +CPU= 54.509
  04:45:11 STEP  749 H= 749:00 +CPU= 51.919
  04:46:40 STEP  750 H= 750:00 +CPU= 86.658
  04:47:34 STEP  751 H= 751:00 +CPU= 53.087
  04:48:28 STEP  752 H= 752:00 +CPU= 53.296
  04:49:54 STEP  753 H= 753:00 +CPU= 84.715
  04:50:49 STEP  754 H= 754:00 +CPU= 54.473
  04:51:42 STEP  755 H= 755:00 +CPU= 51.580
  04:53:17 STEP  756 H= 756:00 +CPU= 94.030
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+000756
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+000756
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+000756
  04:54:12 STEP  757 H= 757:00 +CPU= 53.422
  04:55:06 STEP  758 H= 758:00 +CPU= 53.269
  04:56:33 STEP  759 H= 759:00 +CPU= 85.182
  04:57:29 STEP  760 H= 760:00 +CPU= 54.952
  04:58:21 STEP  761 H= 761:00 +CPU= 51.632
  04:59:49 STEP  762 H= 762:00 +CPU= 86.713
  05:00:44 STEP  763 H= 763:00 +CPU= 53.318
  05:01:38 STEP  764 H= 764:00 +CPU= 52.876
  05:03:03 STEP  765 H= 765:00 +CPU= 83.896
  05:03:59 STEP  766 H= 766:00 +CPU= 55.303
  05:04:54 STEP  767 H= 767:00 +CPU= 54.013
  05:06:36 STEP  768 H= 768:00 +CPU= 99.665
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+000768
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+000768
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+000768
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMGGhq0f+000744
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMSHhq0f+000744
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMUAhq0f+000744
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMGGhq0f+000756
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMSHhq0f+000756
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMUAhq0f+000756
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0747_1986050100_123_955_12172391_2_r1416727145_31.zip
Uploading the intermediate file: upload_file_31.zip
  05:07:30 STEP  769 H= 769:00 +CPU= 52.693
  05:08:24 STEP  770 H= 770:00 +CPU= 53.247
  05:09:51 STEP  771 H= 771:00 +CPU= 85.074
  05:10:47 STEP  772 H= 772:00 +CPU= 54.993
  05:11:40 STEP  773 H= 773:00 +CPU= 51.838
  05:13:08 STEP  774 H= 774:00 +CPU= 86.410
  05:14:02 STEP  775 H= 775:00 +CPU= 53.464
  05:14:56 STEP  776 H= 776:00 +CPU= 53.064
  05:16:21 STEP  777 H= 777:00 +CPU= 83.962
  05:17:18 STEP  778 H= 778:00 +CPU= 55.224
  05:18:11 STEP  779 H= 779:00 +CPU= 52.631
  05:19:46 STEP  780 H= 780:00 +CPU= 93.362
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+000780
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+000780
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+000780
  05:20:41 STEP  781 H= 781:00 +CPU= 53.797
  05:21:36 STEP  782 H= 782:00 +CPU= 53.387
  05:23:01 STEP  783 H= 783:00 +CPU= 83.842
  05:23:57 STEP  784 H= 784:00 +CPU= 55.616
  05:24:50 STEP  785 H= 785:00 +CPU= 52.028
  05:26:17 STEP  786 H= 786:00 +CPU= 84.897
  05:27:11 STEP  787 H= 787:00 +CPU= 53.103
  05:28:06 STEP  788 H= 788:00 +CPU= 54.224
  05:29:30 STEP  789 H= 789:00 +CPU= 82.788
  05:30:26 STEP  790 H= 790:00 +CPU= 55.113
  05:31:23 STEP  791 H= 791:00 +CPU= 55.519
  05:33:02 STEP  792 H= 792:00 +CPU= 97.294
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+000792
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+000792
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+000792
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMGGhq0f+000768
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMSHhq0f+000768
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMUAhq0f+000768
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMGGhq0f+000780
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMSHhq0f+000780
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMUAhq0f+000780
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0747_1986050100_123_955_12172391_2_r1416727145_32.zip
Uploading the intermediate file: upload_file_32.zip
  05:33:57 STEP  793 H= 793:00 +CPU= 53.329
  05:34:51 STEP  794 H= 794:00 +CPU= 53.695
  05:36:16 STEP  795 H= 795:00 +CPU= 83.331
  05:37:12 STEP  796 H= 796:00 +CPU= 54.707
  05:38:06 STEP  797 H= 797:00 +CPU= 53.384
  05:39:31 STEP  798 H= 798:00 +CPU= 83.857
  05:40:26 STEP  799 H= 799:00 +CPU= 53.768
  05:41:22 STEP  800 H= 800:00 +CPU= 54.887
  05:42:45 STEP  801 H= 801:00 +CPU= 82.385
  05:43:42 STEP  802 H= 802:00 +CPU= 55.886
  05:44:36 STEP  803 H= 803:00 +CPU= 53.062
  05:46:09 STEP  804 H= 804:00 +CPU= 91.413
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+000804
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+000804
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+000804
  05:47:04 STEP  805 H= 805:00 +CPU= 53.726
  05:48:00 STEP  806 H= 806:00 +CPU= 54.613
  05:49:24 STEP  807 H= 807:00 +CPU= 82.910
  05:50:21 STEP  808 H= 808:00 +CPU= 55.996
  05:51:15 STEP  809 H= 809:00 +CPU= 53.262
  05:52:40 STEP  810 H= 810:00 +CPU= 83.642
  05:53:36 STEP  811 H= 811:00 +CPU= 54.458
  05:54:30 STEP  812 H= 812:00 +CPU= 53.708
  05:55:54 STEP  813 H= 813:00 +CPU= 82.091
  05:56:52 STEP  814 H= 814:00 +CPU= 57.179
  05:57:48 STEP  815 H= 815:00 +CPU= 54.869
  05:59:26 STEP  816 H= 816:00 +CPU= 96.129
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+000816
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+000816
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+000816
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMGGhq0f+000792
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMSHhq0f+000792
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMUAhq0f+000792
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMGGhq0f+000804
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMSHhq0f+000804
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMUAhq0f+000804
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0747_1986050100_123_955_12172391_2_r1416727145_33.zip
Uploading the intermediate file: upload_file_33.zip
  06:00:22 STEP  817 H= 817:00 +CPU= 54.139
  06:01:17 STEP  818 H= 818:00 +CPU= 54.417
  06:02:40 STEP  819 H= 819:00 +CPU= 81.854
  06:03:38 STEP  820 H= 820:00 +CPU= 56.617
  06:04:31 STEP  821 H= 821:00 +CPU= 52.148
  06:05:56 STEP  822 H= 822:00 +CPU= 83.535
  06:06:52 STEP  823 H= 823:00 +CPU= 55.506
  06:07:47 STEP  824 H= 824:00 +CPU= 54.282
  06:09:10 STEP  825 H= 825:00 +CPU= 81.404
  06:10:08 STEP  826 H= 826:00 +CPU= 57.322
  06:11:02 STEP  827 H= 827:00 +CPU= 52.510
  06:12:35 STEP  828 H= 828:00 +CPU= 91.121
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+000828
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+000828
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+000828
  06:13:31 STEP  829 H= 829:00 +CPU= 55.685
  06:14:27 STEP  830 H= 830:00 +CPU= 54.321
  06:15:50 STEP  831 H= 831:00 +CPU= 81.676
  06:16:48 STEP  832 H= 832:00 +CPU= 57.595
  06:17:42 STEP  833 H= 833:00 +CPU= 52.490
  06:19:06 STEP  834 H= 834:00 +CPU= 83.091
  06:20:02 STEP  835 H= 835:00 +CPU= 55.317
  06:20:57 STEP  836 H= 836:00 +CPU= 53.995
  06:22:19 STEP  837 H= 837:00 +CPU= 80.596
  06:23:18 STEP  838 H= 838:00 +CPU= 57.593
  06:24:14 STEP  839 H= 839:00 +CPU= 55.181
  06:25:52 STEP  840 H= 840:00 +CPU= 95.357
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+000840
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+000840
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+000840
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMGGhq0f+000816
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMSHhq0f+000816
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMUAhq0f+000816
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMGGhq0f+000828
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMSHhq0f+000828
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMUAhq0f+000828
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0747_1986050100_123_955_12172391_2_r1416727145_34.zip
Uploading the intermediate file: upload_file_34.zip
  06:26:48 STEP  841 H= 841:00 +CPU= 55.142
  06:27:43 STEP  842 H= 842:00 +CPU= 53.758
  06:29:05 STEP  843 H= 843:00 +CPU= 80.621
  06:30:03 STEP  844 H= 844:00 +CPU= 57.269
  06:30:57 STEP  845 H= 845:00 +CPU= 52.710
  06:32:20 STEP  846 H= 846:00 +CPU= 82.010
  06:33:17 STEP  847 H= 847:00 +CPU= 56.403
  06:34:13 STEP  848 H= 848:00 +CPU= 54.619
  06:35:35 STEP  849 H= 849:00 +CPU= 80.286
  06:36:34 STEP  850 H= 850:00 +CPU= 58.122
  06:37:28 STEP  851 H= 851:00 +CPU= 53.138
  06:38:53 STEP  852 H= 852:00 +CPU= 83.120
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+000852
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+000852
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+000852
  06:39:43 STEP  853 H= 853:00 +CPU= 50.011
  06:40:33 STEP  854 H= 854:00 +CPU= 48.934
  06:41:48 STEP  855 H= 855:00 +CPU= 73.866
  06:42:41 STEP  856 H= 856:00 +CPU= 51.465
  06:43:29 STEP  857 H= 857:00 +CPU= 47.313
  06:44:45 STEP  858 H= 858:00 +CPU= 74.854
  06:45:35 STEP  859 H= 859:00 +CPU= 49.709
  06:46:25 STEP  860 H= 860:00 +CPU= 48.910
  06:47:40 STEP  861 H= 861:00 +CPU= 73.245
  06:48:32 STEP  862 H= 862:00 +CPU= 51.265
  06:49:22 STEP  863 H= 863:00 +CPU= 49.800
  06:50:55 STEP  864 H= 864:00 +CPU= 88.891
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+000864
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+000864
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+000864
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMGGhq0f+000840
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMSHhq0f+000840
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMUAhq0f+000840
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMGGhq0f+000852
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMSHhq0f+000852
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMUAhq0f+000852
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0747_1986050100_123_955_12172391_2_r1416727145_35.zip
Uploading the intermediate file: upload_file_35.zip
  06:51:49 STEP  865 H= 865:00 +CPU= 51.770
  06:52:37 STEP  866 H= 866:00 +CPU= 47.493
  06:53:53 STEP  867 H= 867:00 +CPU= 75.254
  06:54:45 STEP  868 H= 868:00 +CPU= 48.849
  06:55:32 STEP  869 H= 869:00 +CPU= 46.032
  06:56:45 STEP  870 H= 870:00 +CPU= 72.544
  06:57:33 STEP  871 H= 871:00 +CPU= 46.943
  06:58:21 STEP  872 H= 872:00 +CPU= 47.451
  06:59:33 STEP  873 H= 873:00 +CPU= 71.339
  07:00:21 STEP  874 H= 874:00 +CPU= 47.937
  07:01:09 STEP  875 H= 875:00 +CPU= 47.303
  07:02:34 STEP  876 H= 876:00 +CPU= 83.249
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+000876
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+000876
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+000876
  07:03:22 STEP  877 H= 877:00 +CPU= 47.817
  07:04:10 STEP  878 H= 878:00 +CPU= 47.587
  07:05:24 STEP  879 H= 879:00 +CPU= 73.213
  07:06:12 STEP  880 H= 880:00 +CPU= 47.247
  07:06:59 STEP  881 H= 881:00 +CPU= 46.431
  07:08:14 STEP  882 H= 882:00 +CPU= 74.211
  07:09:02 STEP  883 H= 883:00 +CPU= 47.091
  07:09:50 STEP  884 H= 884:00 +CPU= 47.988
  07:11:04 STEP  885 H= 885:00 +CPU= 73.394
  07:11:52 STEP  886 H= 886:00 +CPU= 47.721
  07:12:41 STEP  887 H= 887:00 +CPU= 48.454
  07:14:06 STEP  888 H= 888:00 +CPU= 81.867
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+000888
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+000888
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+000888
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMGGhq0f+000864
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMSHhq0f+000864
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMUAhq0f+000864
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMGGhq0f+000876
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMSHhq0f+000876
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMUAhq0f+000876
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0747_1986050100_123_955_12172391_2_r1416727145_36.zip
Uploading the intermediate file: upload_file_36.zip
  07:14:59 STEP  889 H= 889:00 +CPU= 48.367
  07:15:48 STEP  890 H= 890:00 +CPU= 48.405
  07:16:58 STEP  891 H= 891:00 +CPU= 69.517
  07:17:48 STEP  892 H= 892:00 +CPU= 49.276
  07:18:36 STEP  893 H= 893:00 +CPU= 47.520
  07:19:48 STEP  894 H= 894:00 +CPU= 71.769
  07:20:36 STEP  895 H= 895:00 +CPU= 47.035
  07:21:25 STEP  896 H= 896:00 +CPU= 49.300
  07:22:41 STEP  897 H= 897:00 +CPU= 72.482
  07:23:30 STEP  898 H= 898:00 +CPU= 48.724
  07:24:23 STEP  899 H= 899:00 +CPU= 51.654
Quit request received from BOINC client, ending the child process
..Failed, model did not complete successfully
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 1986050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0747
(argv4) batchid: 955
(argv5) wuid: 12172391
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/5
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/5/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/5/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_42544a327b86f80adf57eeff90e512e6 to: /var/lib/boinc-client/slots/5/oifs_43r3_ps_0747_1986050100_123_955_12172391.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/5/oifs_43r3_ps_0747_1986050100_123_955_12172391.zip
ic_ancil_file: ic_ancil_12172391
ifsdata_file: ifsdata_12172391
climate_data_file: clim_data_12172391
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_601ffbc9f2c2e84909ebb0f2c84571aa to: /var/lib/boinc-client/slots/5/ic_ancil_12172391.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/5/ic_ancil_12172391.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_3f2d2d4be85e88ea0b43c4051e9eeff6 to: /var/lib/boinc-client/slots/5/ifsdata/ifsdata_12172391.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/5/ifsdata/ifsdata_12172391.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/5/159l_2/clim_data_12172391.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/5/159l_2/clim_data_12172391.zip
Checking for progress XML file: /var/lib/boinc-client/slots/5/progress_file_12172391.xml
Opened progress file ok : /var/lib/boinc-client/slots/5/progress_file_12172391.xml
-- Model is restarting --
Adjusting last_iter, 900, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/5/progress_file_12172391.xml
last_cpu_time: 58484
upload_file_number: 37
last_iter: 887
last_upload: 3196800
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0747_1986050100_123_955_12172391_2_r1416727145
The child process has been launched with process id: 341637
Executing the command: /var/lib/boinc-client/slots/5/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:Debian17:1:1:341637:341637] [20230122:072612:1674372372:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:Debian17:1:1:341637:341637] [20230122:072612:1674372372:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:Debian17:1:1:341637:341637] [20230122:072612:1674372372:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:Debian17:1:1:341637:341637] [20230122:072612:1674372372:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:Debian17:1:1:341637:341637] [20230122:072612:1674372372:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:Debian17:1:1:341637:341637] [20230122:072612:1674372372:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:Debian17:1:1:341637:341637] [20230122:072612:1674372372:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:Debian17:1:1:341637:341637] [20230122:072612:1674372372:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:Debian17:1:1:341637:341637] [20230122:072612:1674372372:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:Debian17:1:1:341637:341637] [20230122:072612:1674372372:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:Debian17:1:1:341637:341637] [20230122:072612:1674372372:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:Debian17:1:1:341637:341637] [20230122:072612:1674372372:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:Debian17:1:1:341637:341637] [20230122:072612:1674372372:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:Debian17:1:1:341637:341637] [20230122:072612:1674372372:0.001] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:Debian17:1:1:341637:341637] [20230122:072612:1674372372:0.001] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:Debian17:1:1:341637:341637] [20230122:072612:1674372372:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Debian17:1:1:341637:341637] [20230122:072612:1674372372:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:Debian17:1:1:341637:341637] [20230122:072612:1674372372:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Debian17:1:1:341637:341637] [20230122:072612:1674372372:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:Debian17:1:1:341637:341637] [20230122:072612:1674372372:0.001] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:Debian17:1:1:341637:341637] [20230122:072612:1674372372:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Debian17:1:1:341637:341637] [20230122:072612:1674372372:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Debian17:1:1:341637:341637] [20230122:072612:1674372372:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:Debian17:1:1:341637:341637] [20230122:072612:1674372372:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Debian17:1:1:341637:341637] [20230122:072612:1674372372:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Debian17:1:1:341637:341637] [20230122:072612:1674372372:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Debian17:1:1:341637:341637] [20230122:072612:1674372372:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:Debian17:1:1:341637:341637] [20230122:072612:1674372372:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:Debian17:1:1:341637:341637] [20230122:072612:1674372372:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:Debian17:1:1:341637:341637] [20230122:072612:1674372372:0.001] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  07:28:03 STEP  888 H= 888:00 +CPU= 94.940
  07:28:53 STEP  889 H= 889:00 +CPU= 49.791
  07:29:48 STEP  890 H= 890:00 +CPU= 53.942
  07:31:12 STEP  891 H= 891:00 +CPU= 82.316
  07:32:01 STEP  892 H= 892:00 +CPU= 48.401
  07:32:49 STEP  893 H= 893:00 +CPU= 47.095
  07:34:03 STEP  894 H= 894:00 +CPU= 73.201
  07:34:48 STEP  895 H= 895:00 +CPU= 44.613
  07:35:36 STEP  896 H= 896:00 +CPU= 47.696
  07:36:49 STEP  897 H= 897:00 +CPU= 72.332
  07:37:36 STEP  898 H= 898:00 +CPU= 46.282
  07:38:22 STEP  899 H= 899:00 +CPU= 45.640
  07:39:45 STEP  900 H= 900:00 +CPU= 81.592
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+000900
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+000900
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+000900
  07:40:33 STEP  901 H= 901:00 +CPU= 47.449
  07:41:24 STEP  902 H= 902:00 +CPU= 49.813
  07:42:37 STEP  903 H= 903:00 +CPU= 72.353
  07:43:24 STEP  904 H= 904:00 +CPU= 46.400
  07:44:12 STEP  905 H= 905:00 +CPU= 46.831
  07:45:26 STEP  906 H= 906:00 +CPU= 73.507
  07:46:12 STEP  907 H= 907:00 +CPU= 45.508
  07:46:59 STEP  908 H= 908:00 +CPU= 46.948
  07:48:17 STEP  909 H= 909:00 +CPU= 75.826
  07:49:05 STEP  910 H= 910:00 +CPU= 47.821
  07:49:57 STEP  911 H= 911:00 +CPU= 50.324
  07:51:30 STEP  912 H= 912:00 +CPU= 87.354
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+000912
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+000912
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+000912
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMGGhq0f+000888
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMSHhq0f+000888
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMUAhq0f+000888
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMGGhq0f+000900
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMSHhq0f+000900
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMUAhq0f+000900
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0747_1986050100_123_955_12172391_2_r1416727145_37.zip
Uploading the intermediate file: upload_file_37.zip
  07:52:18 STEP  913 H= 913:00 +CPU= 47.270
  07:53:11 STEP  914 H= 914:00 +CPU= 51.780
  07:54:28 STEP  915 H= 915:00 +CPU= 74.309
  07:55:18 STEP  916 H= 916:00 +CPU= 49.092
  07:56:09 STEP  917 H= 917:00 +CPU= 50.449
  07:57:30 STEP  918 H= 918:00 +CPU= 79.575
  07:58:17 STEP  919 H= 919:00 +CPU= 46.166
  07:59:10 STEP  920 H= 920:00 +CPU= 51.856
  08:00:28 STEP  921 H= 921:00 +CPU= 76.411
  08:01:16 STEP  922 H= 922:00 +CPU= 47.978
  08:02:02 STEP  923 H= 923:00 +CPU= 45.426
  08:03:30 STEP  924 H= 924:00 +CPU= 85.942
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+000924
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+000924
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+000924
  08:04:20 STEP  925 H= 925:00 +CPU= 48.814
  08:05:08 STEP  926 H= 926:00 +CPU= 47.797
  08:06:26 STEP  927 H= 927:00 +CPU= 76.753
  08:07:19 STEP  928 H= 928:00 +CPU= 51.910
  08:08:05 STEP  929 H= 929:00 +CPU= 45.496
  08:09:25 STEP  930 H= 930:00 +CPU= 78.426
  08:10:16 STEP  931 H= 931:00 +CPU= 50.130
  08:11:03 STEP  932 H= 932:00 +CPU= 46.586
  08:12:21 STEP  933 H= 933:00 +CPU= 77.428
  08:13:16 STEP  934 H= 934:00 +CPU= 53.361
  08:14:03 STEP  935 H= 935:00 +CPU= 46.609
  08:15:37 STEP  936 H= 936:00 +CPU= 90.963
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+000936
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+000936
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+000936
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMGGhq0f+000912
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMSHhq0f+000912
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMUAhq0f+000912
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMGGhq0f+000924
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMSHhq0f+000924
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMUAhq0f+000924
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0747_1986050100_123_955_12172391_2_r1416727145_38.zip
Uploading the intermediate file: upload_file_38.zip
  08:16:26 STEP  937 H= 937:00 +CPU= 48.348
  08:17:14 STEP  938 H= 938:00 +CPU= 47.210
  08:18:31 STEP  939 H= 939:00 +CPU= 75.410
  08:19:19 STEP  940 H= 940:00 +CPU= 47.573
  08:20:00 STEP  941 H= 941:00 +CPU= 40.736
  08:21:14 STEP  942 H= 942:00 +CPU= 73.089
  08:22:01 STEP  943 H= 943:00 +CPU= 45.786
  08:22:44 STEP  944 H= 944:00 +CPU= 42.300
  08:23:57 STEP  945 H= 945:00 +CPU= 72.037
  08:24:46 STEP  946 H= 946:00 +CPU= 49.021
  08:25:31 STEP  947 H= 947:00 +CPU= 44.217
  08:26:47 STEP  948 H= 948:00 +CPU= 75.096
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+000948
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+000948
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+000948
  08:27:30 STEP  949 H= 949:00 +CPU= 42.318
  08:28:18 STEP  950 H= 950:00 +CPU= 47.688
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 1986050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0747
(argv4) batchid: 955
(argv5) wuid: 12172391
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/5
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/5/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/5/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_42544a327b86f80adf57eeff90e512e6 to: /var/lib/boinc-client/slots/5/oifs_43r3_ps_0747_1986050100_123_955_12172391.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/5/oifs_43r3_ps_0747_1986050100_123_955_12172391.zip
ic_ancil_file: ic_ancil_12172391
ifsdata_file: ifsdata_12172391
climate_data_file: clim_data_12172391
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_601ffbc9f2c2e84909ebb0f2c84571aa to: /var/lib/boinc-client/slots/5/ic_ancil_12172391.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/5/ic_ancil_12172391.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_3f2d2d4be85e88ea0b43c4051e9eeff6 to: /var/lib/boinc-client/slots/5/ifsdata/ifsdata_12172391.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/5/ifsdata/ifsdata_12172391.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/5/159l_2/clim_data_12172391.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/5/159l_2/clim_data_12172391.zip
Checking for progress XML file: /var/lib/boinc-client/slots/5/progress_file_12172391.xml
Opened progress file ok : /var/lib/boinc-client/slots/5/progress_file_12172391.xml
-- Model is restarting --
Adjusting last_iter, 951, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/5/progress_file_12172391.xml
last_cpu_time: 62156
upload_file_number: 39
last_iter: 935
last_upload: 3369600
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0747_1986050100_123_955_12172391_2_r1416727145
The child process has been launched with process id: 341857
Executing the command: /var/lib/boinc-client/slots/5/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:Debian17:1:1:341857:341857] [20230122:083955:1674376795:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:Debian17:1:1:341857:341857] [20230122:083955:1674376795:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:Debian17:1:1:341857:341857] [20230122:083955:1674376795:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:Debian17:1:1:341857:341857] [20230122:083955:1674376795:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:Debian17:1:1:341857:341857] [20230122:083955:1674376795:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:Debian17:1:1:341857:341857] [20230122:083955:1674376795:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:Debian17:1:1:341857:341857] [20230122:083955:1674376795:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:Debian17:1:1:341857:341857] [20230122:083955:1674376795:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:Debian17:1:1:341857:341857] [20230122:083955:1674376795:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:Debian17:1:1:341857:341857] [20230122:083955:1674376795:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:Debian17:1:1:341857:341857] [20230122:083955:1674376795:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:Debian17:1:1:341857:341857] [20230122:083955:1674376795:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:Debian17:1:1:341857:341857] [20230122:083955:1674376795:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:Debian17:1:1:341857:341857] [20230122:083955:1674376795:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:Debian17:1:1:341857:341857] [20230122:083955:1674376795:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:Debian17:1:1:341857:341857] [20230122:083955:1674376795:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Debian17:1:1:341857:341857] [20230122:083955:1674376795:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:Debian17:1:1:341857:341857] [20230122:083955:1674376795:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Debian17:1:1:341857:341857] [20230122:083955:1674376795:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:Debian17:1:1:341857:341857] [20230122:083955:1674376795:0.001] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:Debian17:1:1:341857:341857] [20230122:083955:1674376795:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Debian17:1:1:341857:341857] [20230122:083955:1674376795:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Debian17:1:1:341857:341857] [20230122:083955:1674376795:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:Debian17:1:1:341857:341857] [20230122:083955:1674376795:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Debian17:1:1:341857:341857] [20230122:083955:1674376795:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Debian17:1:1:341857:341857] [20230122:083955:1674376795:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Debian17:1:1:341857:341857] [20230122:083955:1674376795:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:Debian17:1:1:341857:341857] [20230122:083955:1674376795:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:Debian17:1:1:341857:341857] [20230122:083955:1674376795:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:Debian17:1:1:341857:341857] [20230122:083955:1674376795:0.001] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  08:41:33 STEP  936 H= 936:00 +CPU= 84.498
  08:42:25 STEP  937 H= 937:00 +CPU= 50.668
  08:43:13 STEP  938 H= 938:00 +CPU= 48.219
  08:44:21 STEP  939 H= 939:00 +CPU= 66.816
  08:45:13 STEP  940 H= 940:00 +CPU= 51.350
  08:46:04 STEP  941 H= 941:00 +CPU= 50.389
  08:47:23 STEP  942 H= 942:00 +CPU= 77.453
  08:48:18 STEP  943 H= 943:00 +CPU= 53.953
  08:49:12 STEP  944 H= 944:00 +CPU= 53.030
  08:50:20 STEP  945 H= 945:00 +CPU= 67.060
  08:51:06 STEP  946 H= 946:00 +CPU= 44.981
  08:51:52 STEP  947 H= 947:00 +CPU= 43.881
  08:53:00 STEP  948 H= 948:00 +CPU= 67.040
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+000948
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+000948
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+000948
  08:53:40 STEP  949 H= 949:00 +CPU= 39.796
  08:54:25 STEP  950 H= 950:00 +CPU= 43.817
  08:55:26 STEP  951 H= 951:00 +CPU= 60.760
  08:56:09 STEP  952 H= 952:00 +CPU= 42.070
  08:56:53 STEP  953 H= 953:00 +CPU= 43.927
  08:57:57 STEP  954 H= 954:00 +CPU= 63.397
  08:58:38 STEP  955 H= 955:00 +CPU= 40.665
  08:59:23 STEP  956 H= 956:00 +CPU= 44.372
  09:00:28 STEP  957 H= 957:00 +CPU= 64.378
  09:01:10 STEP  958 H= 958:00 +CPU= 41.405
  09:01:55 STEP  959 H= 959:00 +CPU= 43.610
  09:03:15 STEP  960 H= 960:00 +CPU= 75.390
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+000960
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+000960
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+000960
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMGGhq0f+000936
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMSHhq0f+000936
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMUAhq0f+000936
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMGGhq0f+000948
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMSHhq0f+000948
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMUAhq0f+000948
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0747_1986050100_123_955_12172391_2_r1416727145_39.zip
Uploading the intermediate file: upload_file_39.zip
  09:03:58 STEP  961 H= 961:00 +CPU= 41.778
  09:04:48 STEP  962 H= 962:00 +CPU= 49.429
  09:05:57 STEP  963 H= 963:00 +CPU= 68.451
  09:06:43 STEP  964 H= 964:00 +CPU= 46.082
  09:07:32 STEP  965 H= 965:00 +CPU= 48.522
  09:08:44 STEP  966 H= 966:00 +CPU= 71.257
  09:09:30 STEP  967 H= 967:00 +CPU= 44.868
  09:10:19 STEP  968 H= 968:00 +CPU= 49.187
  09:11:30 STEP  969 H= 969:00 +CPU= 69.667
  09:12:18 STEP  970 H= 970:00 +CPU= 47.388
  09:13:06 STEP  971 H= 971:00 +CPU= 47.228
  09:14:23 STEP  972 H= 972:00 +CPU= 75.788
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+000972
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+000972
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+000972
  09:15:11 STEP  973 H= 973:00 +CPU= 47.370
  09:16:06 STEP  974 H= 974:00 +CPU= 54.549
  09:17:26 STEP  975 H= 975:00 +CPU= 78.058
  09:18:20 STEP  976 H= 976:00 +CPU= 53.585
  09:19:12 STEP  977 H= 977:00 +CPU= 51.099
  09:20:33 STEP  978 H= 978:00 +CPU= 79.637
  09:21:28 STEP  979 H= 979:00 +CPU= 53.924
  09:22:20 STEP  980 H= 980:00 +CPU= 51.273
  09:23:40 STEP  981 H= 981:00 +CPU= 78.374
  09:24:37 STEP  982 H= 982:00 +CPU= 55.935
  09:25:30 STEP  983 H= 983:00 +CPU= 52.000
  09:27:05 STEP  984 H= 984:00 +CPU= 92.662
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+000984
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+000984
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+000984
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMGGhq0f+000960
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMSHhq0f+000960
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMUAhq0f+000960
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMGGhq0f+000972
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMSHhq0f+000972
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMUAhq0f+000972
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0747_1986050100_123_955_12172391_2_r1416727145_40.zip
Uploading the intermediate file: upload_file_40.zip
  09:27:59 STEP  985 H= 985:00 +CPU= 53.497
  09:28:52 STEP  986 H= 986:00 +CPU= 51.650
  09:30:11 STEP  987 H= 987:00 +CPU= 78.150
  09:31:07 STEP  988 H= 988:00 +CPU= 54.996
  09:31:58 STEP  989 H= 989:00 +CPU= 49.648
  09:33:19 STEP  990 H= 990:00 +CPU= 79.711
  09:34:14 STEP  991 H= 991:00 +CPU= 54.107
  09:35:05 STEP  992 H= 992:00 +CPU= 50.565
  09:36:25 STEP  993 H= 993:00 +CPU= 78.241
  09:37:22 STEP  994 H= 994:00 +CPU= 56.094
  09:38:13 STEP  995 H= 995:00 +CPU= 49.617
  09:39:42 STEP  996 H= 996:00 +CPU= 87.864
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+000996
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+000996
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+000996
  09:40:37 STEP  997 H= 997:00 +CPU= 54.220
  09:41:29 STEP  998 H= 998:00 +CPU= 51.146
  09:42:50 STEP  999 H= 999:00 +CPU= 79.145
  09:43:45 STEP 1000 H=1000:00 +CPU= 54.875
  09:44:36 STEP 1001 H=1001:00 +CPU= 49.914
  09:45:58 STEP 1002 H=1002:00 +CPU= 80.493
  09:46:52 STEP 1003 H=1003:00 +CPU= 52.427
  09:47:45 STEP 1004 H=1004:00 +CPU= 52.687
  09:49:03 STEP 1005 H=1005:00 +CPU= 76.988
  09:49:57 STEP 1006 H=1006:00 +CPU= 52.436
  09:50:53 STEP 1007 H=1007:00 +CPU= 55.080
  09:52:25 STEP 1008 H=1008:00 +CPU= 90.423
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+001008
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+001008
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+001008
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMGGhq0f+000984
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMSHhq0f+000984
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMUAhq0f+000984
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMGGhq0f+000996
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMSHhq0f+000996
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMUAhq0f+000996
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0747_1986050100_123_955_12172391_2_r1416727145_41.zip
Uploading the intermediate file: upload_file_41.zip
  09:53:18 STEP 1009 H=1009:00 +CPU= 52.064
  09:54:13 STEP 1010 H=1010:00 +CPU= 53.645
  09:55:32 STEP 1011 H=1011:00 +CPU= 77.313
  09:56:25 STEP 1012 H=1012:00 +CPU= 52.866
  09:57:20 STEP 1013 H=1013:00 +CPU= 53.425
  09:58:39 STEP 1014 H=1014:00 +CPU= 78.225
  09:59:32 STEP 1015 H=1015:00 +CPU= 51.389
  10:00:28 STEP 1016 H=1016:00 +CPU= 55.636
  10:01:47 STEP 1017 H=1017:00 +CPU= 77.049
  10:02:41 STEP 1018 H=1018:00 +CPU= 53.456
  10:03:36 STEP 1019 H=1019:00 +CPU= 54.020
  10:05:03 STEP 1020 H=1020:00 +CPU= 85.497
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+001020
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+001020
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+001020
  10:05:56 STEP 1021 H=1021:00 +CPU= 51.777
  10:06:52 STEP 1022 H=1022:00 +CPU= 55.690
  10:08:10 STEP 1023 H=1023:00 +CPU= 76.519
  10:09:04 STEP 1024 H=1024:00 +CPU= 53.409
  10:09:59 STEP 1025 H=1025:00 +CPU= 53.351
  10:11:18 STEP 1026 H=1026:00 +CPU= 78.589
  10:12:12 STEP 1027 H=1027:00 +CPU= 52.759
  10:13:07 STEP 1028 H=1028:00 +CPU= 54.219
  10:14:25 STEP 1029 H=1029:00 +CPU= 76.671
  10:15:21 STEP 1030 H=1030:00 +CPU= 54.942
  10:16:17 STEP 1031 H=1031:00 +CPU= 54.460
  10:17:50 STEP 1032 H=1032:00 +CPU= 91.295
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+001032
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+001032
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+001032
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMGGhq0f+001008
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMSHhq0f+001008
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMUAhq0f+001008
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMGGhq0f+001020
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMSHhq0f+001020
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMUAhq0f+001020
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0747_1986050100_123_955_12172391_2_r1416727145_42.zip
Uploading the intermediate file: upload_file_42.zip
  10:18:44 STEP 1033 H=1033:00 +CPU= 52.383
  10:19:39 STEP 1034 H=1034:00 +CPU= 53.865
  10:20:56 STEP 1035 H=1035:00 +CPU= 76.282
  10:21:53 STEP 1036 H=1036:00 +CPU= 55.251
  10:22:45 STEP 1037 H=1037:00 +CPU= 52.098
  10:24:05 STEP 1038 H=1038:00 +CPU= 78.406
  10:25:00 STEP 1039 H=1039:00 +CPU= 53.516
  10:25:54 STEP 1040 H=1040:00 +CPU= 53.191
  10:27:13 STEP 1041 H=1041:00 +CPU= 77.324
  10:28:08 STEP 1042 H=1042:00 +CPU= 54.590
  10:29:01 STEP 1043 H=1043:00 +CPU= 52.124
  10:30:29 STEP 1044 H=1044:00 +CPU= 86.423
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+001044
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+001044
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+001044
  10:31:23 STEP 1045 H=1045:00 +CPU= 52.998
  10:32:18 STEP 1046 H=1046:00 +CPU= 53.805
  10:33:36 STEP 1047 H=1047:00 +CPU= 77.265
  10:34:31 STEP 1048 H=1048:00 +CPU= 53.958
  10:35:24 STEP 1049 H=1049:00 +CPU= 51.821
  10:36:43 STEP 1050 H=1050:00 +CPU= 78.116
  10:37:37 STEP 1051 H=1051:00 +CPU= 52.259
  10:38:32 STEP 1052 H=1052:00 +CPU= 54.138
  10:39:50 STEP 1053 H=1053:00 +CPU= 77.286
  10:40:43 STEP 1054 H=1054:00 +CPU= 52.327
  10:41:40 STEP 1055 H=1055:00 +CPU= 55.511
  10:43:12 STEP 1056 H=1056:00 +CPU= 90.298
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+001056
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+001056
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+001056
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMGGhq0f+001032
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMSHhq0f+001032
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMUAhq0f+001032
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMGGhq0f+001044
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMSHhq0f+001044
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMUAhq0f+001044
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0747_1986050100_123_955_12172391_2_r1416727145_43.zip
Uploading the intermediate file: upload_file_43.zip
  10:44:06 STEP 1057 H=1057:00 +CPU= 52.218
  10:45:01 STEP 1058 H=1058:00 +CPU= 53.996
  10:46:19 STEP 1059 H=1059:00 +CPU= 77.274
  10:47:11 STEP 1060 H=1060:00 +CPU= 51.201
  10:48:06 STEP 1061 H=1061:00 +CPU= 53.996
  10:49:27 STEP 1062 H=1062:00 +CPU= 79.850
  10:50:17 STEP 1063 H=1063:00 +CPU= 49.540
  10:51:13 STEP 1064 H=1064:00 +CPU= 54.857
  10:52:32 STEP 1065 H=1065:00 +CPU= 77.587
  10:53:25 STEP 1066 H=1066:00 +CPU= 52.358
  10:54:19 STEP 1067 H=1067:00 +CPU= 52.287
  10:55:47 STEP 1068 H=1068:00 +CPU= 87.078
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+001068
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+001068
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+001068
  10:56:39 STEP 1069 H=1069:00 +CPU= 51.142
  10:57:34 STEP 1070 H=1070:00 +CPU= 54.492
  10:58:54 STEP 1071 H=1071:00 +CPU= 78.013
  10:59:48 STEP 1072 H=1072:00 +CPU= 52.933
  11:00:42 STEP 1073 H=1073:00 +CPU= 52.819
  11:02:02 STEP 1074 H=1074:00 +CPU= 79.612
  11:02:54 STEP 1075 H=1075:00 +CPU= 51.029
  11:03:49 STEP 1076 H=1076:00 +CPU= 54.086
  11:05:08 STEP 1077 H=1077:00 +CPU= 77.589
  11:06:02 STEP 1078 H=1078:00 +CPU= 52.818
  11:06:57 STEP 1079 H=1079:00 +CPU= 54.097
  11:08:32 STEP 1080 H=1080:00 +CPU= 93.182
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+001080
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+001080
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+001080
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMGGhq0f+001056
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMSHhq0f+001056
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMUAhq0f+001056
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMGGhq0f+001068
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMSHhq0f+001068
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMUAhq0f+001068
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0747_1986050100_123_955_12172391_2_r1416727145_44.zip
Uploading the intermediate file: upload_file_44.zip
  11:09:25 STEP 1081 H=1081:00 +CPU= 51.405
  11:10:20 STEP 1082 H=1082:00 +CPU= 54.117
  11:11:39 STEP 1083 H=1083:00 +CPU= 78.488
  11:12:33 STEP 1084 H=1084:00 +CPU= 53.165
  11:13:26 STEP 1085 H=1085:00 +CPU= 52.166
  11:14:47 STEP 1086 H=1086:00 +CPU= 79.511
  11:15:41 STEP 1087 H=1087:00 +CPU= 52.590
  11:16:34 STEP 1088 H=1088:00 +CPU= 52.463
  11:17:53 STEP 1089 H=1089:00 +CPU= 77.574
  11:18:48 STEP 1090 H=1090:00 +CPU= 54.129
  11:19:39 STEP 1091 H=1091:00 +CPU= 50.429
  11:21:08 STEP 1092 H=1092:00 +CPU= 86.725
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+001092
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+001092
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+001092
  11:22:01 STEP 1093 H=1093:00 +CPU= 52.917
  11:22:54 STEP 1094 H=1094:00 +CPU= 51.996
  11:24:15 STEP 1095 H=1095:00 +CPU= 79.761
  11:25:10 STEP 1096 H=1096:00 +CPU= 53.562
  11:26:03 STEP 1097 H=1097:00 +CPU= 51.992
  11:27:25 STEP 1098 H=1098:00 +CPU= 80.690
  11:28:18 STEP 1099 H=1099:00 +CPU= 51.869
  11:29:12 STEP 1100 H=1100:00 +CPU= 52.908
  11:30:33 STEP 1101 H=1101:00 +CPU= 79.622
  11:31:27 STEP 1102 H=1102:00 +CPU= 53.433
  11:32:21 STEP 1103 H=1103:00 +CPU= 53.438
  11:33:58 STEP 1104 H=1104:00 +CPU= 94.726
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+001104
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+001104
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+001104
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMGGhq0f+001080
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMSHhq0f+001080
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMUAhq0f+001080
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMGGhq0f+001092
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMSHhq0f+001092
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMUAhq0f+001092
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0747_1986050100_123_955_12172391_2_r1416727145_45.zip
Uploading the intermediate file: upload_file_45.zip
  11:34:51 STEP 1105 H=1105:00 +CPU= 52.082
  11:35:45 STEP 1106 H=1106:00 +CPU= 53.280
  11:37:07 STEP 1107 H=1107:00 +CPU= 80.282
  11:38:01 STEP 1108 H=1108:00 +CPU= 53.408
  11:38:54 STEP 1109 H=1109:00 +CPU= 51.654
  11:40:17 STEP 1110 H=1110:00 +CPU= 81.923
  11:41:10 STEP 1111 H=1111:00 +CPU= 52.197
  11:42:05 STEP 1112 H=1112:00 +CPU= 53.670
  11:43:25 STEP 1113 H=1113:00 +CPU= 79.281
  11:44:20 STEP 1114 H=1114:00 +CPU= 53.396
  11:45:13 STEP 1115 H=1115:00 +CPU= 52.380
  11:46:41 STEP 1116 H=1116:00 +CPU= 87.281
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+001116
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+001116
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+001116
  11:47:34 STEP 1117 H=1117:00 +CPU= 51.603
  11:48:28 STEP 1118 H=1118:00 +CPU= 53.402
  11:49:47 STEP 1119 H=1119:00 +CPU= 77.797
  11:50:43 STEP 1120 H=1120:00 +CPU= 54.273
  11:51:35 STEP 1121 H=1121:00 +CPU= 51.908
  11:52:55 STEP 1122 H=1122:00 +CPU= 78.649
  11:53:50 STEP 1123 H=1123:00 +CPU= 53.938
  11:54:44 STEP 1124 H=1124:00 +CPU= 53.061
  11:56:02 STEP 1125 H=1125:00 +CPU= 77.194
  11:56:58 STEP 1126 H=1126:00 +CPU= 54.805
  11:57:53 STEP 1127 H=1127:00 +CPU= 53.555
  11:59:26 STEP 1128 H=1128:00 +CPU= 91.233
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+001128
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+001128
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+001128
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMGGhq0f+001104
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMSHhq0f+001104
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMUAhq0f+001104
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMGGhq0f+001116
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMSHhq0f+001116
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172391/ICMUAhq0f+001116
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0747_1986050100_123_955_12172391_2_r1416727145_46.zip
Uploading the intermediate file: upload_file_46.zip
  12:00:21 STEP 1129 H=1129:00 +CPU= 53.782
  12:01:15 STEP 1130 H=1130:00 +CPU= 53.385
  12:02:35 STEP 1131 H=1131:00 +CPU= 78.166
  12:03:30 STEP 1132 H=1132:00 +CPU= 54.679
  12:04:23 STEP 1133 H=1133:00 +CPU= 51.876
  12:05:44 STEP 1134 H=1134:00 +CPU= 80.060
  12:06:38 STEP 1135 H=1135:00 +CPU= 53.103
  12:07:32 STEP 1136 H=1136:00 +CPU= 53.182
  12:08:51 STEP 1137 H=1137:00 +CPU= 77.726
  12:09:47 STEP 1138 H=1138:00 +CPU= 54.983
  12:10:40 STEP 1139 H=1139:00 +CPU= 51.705
  12:12:09 STEP 1140 H=1140:00 +CPU= 87.376
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+001140
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+001140
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+001140
  12:13:03 STEP 1141 H=1141:00 +CPU= 53.097
  12:13:57 STEP 1142 H=1142:00 +CPU= 53.210
  12:15:16 STEP 1143 H=1143:00 +CPU= 78.111
  12:16:12 STEP 1144 H=1144:00 +CPU= 54.500
  12:17:05 STEP 1145 H=1145:00 +CPU= 52.291
  12:18:25 STEP 1146 H=1146:00 +CPU= 78.602
  12:19:19 STEP 1147 H=1147:00 +CPU= 52.819
  12:20:15 STEP 1148 H=1148:00 +CPU= 54.970
  12:21:32 STEP 1149 H=1149:00 +CPU= 76.076
  12:22:28 STEP 1150 H=1150:00 +CPU= 55.382
  12:23:24 STEP 1151 H=1151:00 +CPU= 55.289
Abort request received from BOINC client, ending the child process
..Failed, model did not complete successfully

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 May 2023 16:34:45 1491072 22301142 oifs_43r3_ps_0747_1986050100_123_955_12172391_2 2,678,400 48,783 0.0182
09 May 2023 16:34:17 1491072 22301142 oifs_43r3_ps_0747_1986050100_123_955_12172391_2 1,814,400 33,187 0.0183


©2024 cpdn.org