climateprediction.net home page
Task 22261491

Task 22261491

Name oifs_43r3_ps_0085_1991050100_123_960_12176729_0
Workunit 12176729
Created 21 Dec 2022, 20:53:13 UTC
Sent 20 Jan 2023, 20:11:36 UTC
Report deadline 21 Mar 2023, 20:11:36 UTC
Received 26 Jan 2023, 0:40:51 UTC
Server state Over
Outcome Success
Client state Done
Exit status 0 (0x00000000)
Computer ID 1538243
Run time 22 hours 24 min 41 sec
CPU time 22 hours 24 min 41 sec
Validate state Valid
Credit 14,873.04
Device peak FLOPS 5.52 GFLOPS
Application version OpenIFS 43r3 Perturbed Surface v1.05
x86_64-pc-linux-gnu
Peak working set size 4,411.84 MB
Peak swap size 4,977.99 MB
Peak disk usage 3,752.56 MB
Stderr
<core_client_version>7.20.2</core_client_version>
<![CDATA[
<stderr_txt>
nt_40000/slots/4/ICMGGhq0f+002628
Moving to projects directory: /var/lib/boinc-client_40000/slots/4/ICMSHhq0f+002628
Moving to projects directory: /var/lib/boinc-client_40000/slots/4/ICMUAhq0f+002628
  20:03:04 STEP 2629 H=2629:00 +CPU= 19.929
  20:03:29 STEP 2630 H=2630:00 +CPU= 24.067
  20:04:05 STEP 2631 H=2631:00 +CPU= 36.457
  20:04:29 STEP 2632 H=2632:00 +CPU= 23.841
  20:04:49 STEP 2633 H=2633:00 +CPU= 20.018
  20:05:28 STEP 2634 H=2634:00 +CPU= 38.712
  20:05:53 STEP 2635 H=2635:00 +CPU= 24.961
  20:06:12 STEP 2636 H=2636:00 +CPU= 19.271
  20:06:44 STEP 2637 H=2637:00 +CPU= 32.422
  20:07:07 STEP 2638 H=2638:00 +CPU= 23.239
  20:07:30 STEP 2639 H=2639:00 +CPU= 22.592
  20:08:13 STEP 2640 H=2640:00 +CPU= 43.330
Moving to projects directory: /var/lib/boinc-client_40000/slots/4/ICMGGhq0f+002640
Moving to projects directory: /var/lib/boinc-client_40000/slots/4/ICMSHhq0f+002640
Moving to projects directory: /var/lib/boinc-client_40000/slots/4/ICMUAhq0f+002640
Adding to the zip: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_12176729/ICMGGhq0f+002616
Adding to the zip: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_12176729/ICMSHhq0f+002616
Adding to the zip: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_12176729/ICMUAhq0f+002616
Adding to the zip: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_12176729/ICMGGhq0f+002628
Adding to the zip: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_12176729/ICMSHhq0f+002628
Adding to the zip: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_12176729/ICMUAhq0f+002628
Zipping up the intermediate file: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_0085_1991050100_123_960_12176729_0_r826685277_109.zip
Uploading the intermediate file: upload_file_109.zip
  20:08:33 STEP 2641 H=2641:00 +CPU= 19.948
Uploading trickle at timestep: 9504000
  20:08:55 STEP 2642 H=2642:00 +CPU= 21.520
  20:09:26 STEP 2643 H=2643:00 +CPU= 31.171
  20:09:52 STEP 2644 H=2644:00 +CPU= 25.929
  20:10:14 STEP 2645 H=2645:00 +CPU= 22.480
  20:10:46 STEP 2646 H=2646:00 +CPU= 31.162
  20:11:07 STEP 2647 H=2647:00 +CPU= 21.249
  20:11:28 STEP 2648 H=2648:00 +CPU= 21.271
  20:12:04 STEP 2649 H=2649:00 +CPU= 36.269
  20:12:30 STEP 2650 H=2650:00 +CPU= 25.143
  20:12:50 STEP 2651 H=2651:00 +CPU= 20.728
  20:13:31 STEP 2652 H=2652:00 +CPU= 41.075
Moving to projects directory: /var/lib/boinc-client_40000/slots/4/ICMGGhq0f+002652
Moving to projects directory: /var/lib/boinc-client_40000/slots/4/ICMSHhq0f+002652
Moving to projects directory: /var/lib/boinc-client_40000/slots/4/ICMUAhq0f+002652
  20:13:53 STEP 2653 H=2653:00 +CPU= 21.679
  20:14:12 STEP 2654 H=2654:00 +CPU= 18.833
  20:14:48 STEP 2655 H=2655:00 +CPU= 35.752
  20:15:13 STEP 2656 H=2656:00 +CPU= 25.341
  20:15:37 STEP 2657 H=2657:00 +CPU= 23.752
  20:16:12 STEP 2658 H=2658:00 +CPU= 35.587
  20:16:35 STEP 2659 H=2659:00 +CPU= 22.139
  20:16:58 STEP 2660 H=2660:00 +CPU= 22.959
  20:17:34 STEP 2661 H=2661:00 +CPU= 36.628
  20:17:58 STEP 2662 H=2662:00 +CPU= 23.620
  20:18:22 STEP 2663 H=2663:00 +CPU= 24.390
  20:19:04 STEP 2664 H=2664:00 +CPU= 41.884
Moving to projects directory: /var/lib/boinc-client_40000/slots/4/ICMGGhq0f+002664
Moving to projects directory: /var/lib/boinc-client_40000/slots/4/ICMSHhq0f+002664
Moving to projects directory: /var/lib/boinc-client_40000/slots/4/ICMUAhq0f+002664
Adding to the zip: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_12176729/ICMGGhq0f+002640
Adding to the zip: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_12176729/ICMSHhq0f+002640
Adding to the zip: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_12176729/ICMUAhq0f+002640
Adding to the zip: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_12176729/ICMGGhq0f+002652
Adding to the zip: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_12176729/ICMSHhq0f+002652
Adding to the zip: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_12176729/ICMUAhq0f+002652
Zipping up the intermediate file: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_0085_1991050100_123_960_12176729_0_r826685277_110.zip
Uploading the intermediate file: upload_file_110.zip
  20:19:28 STEP 2665 H=2665:00 +CPU= 23.982
  20:19:51 STEP 2666 H=2666:00 +CPU= 23.240
  20:20:26 STEP 2667 H=2667:00 +CPU= 34.524
  20:20:47 STEP 2668 H=2668:00 +CPU= 20.813
  20:21:11 STEP 2669 H=2669:00 +CPU= 24.484
  20:21:48 STEP 2670 H=2670:00 +CPU= 36.903
  20:22:09 STEP 2671 H=2671:00 +CPU= 21.098
  20:22:33 STEP 2672 H=2672:00 +CPU= 23.843
  20:23:11 STEP 2673 H=2673:00 +CPU= 37.993
  20:23:33 STEP 2674 H=2674:00 +CPU= 21.889
  20:23:54 STEP 2675 H=2675:00 +CPU= 20.822
  20:24:37 STEP 2676 H=2676:00 +CPU= 43.009
Moving to projects directory: /var/lib/boinc-client_40000/slots/4/ICMGGhq0f+002676
Moving to projects directory: /var/lib/boinc-client_40000/slots/4/ICMSHhq0f+002676
Moving to projects directory: /var/lib/boinc-client_40000/slots/4/ICMUAhq0f+002676
  20:24:59 STEP 2677 H=2677:00 +CPU= 22.268
  20:25:19 STEP 2678 H=2678:00 +CPU= 20.195
  20:25:52 STEP 2679 H=2679:00 +CPU= 32.937
  20:26:16 STEP 2680 H=2680:00 +CPU= 23.600
  20:26:39 STEP 2681 H=2681:00 +CPU= 22.873
  20:27:13 STEP 2682 H=2682:00 +CPU= 34.635
  20:27:37 STEP 2683 H=2683:00 +CPU= 23.638
  20:27:59 STEP 2684 H=2684:00 +CPU= 22.182
  20:28:35 STEP 2685 H=2685:00 +CPU= 36.141
  20:28:59 STEP 2686 H=2686:00 +CPU= 24.025
  20:29:22 STEP 2687 H=2687:00 +CPU= 23.073
  20:30:04 STEP 2688 H=2688:00 +CPU= 41.653
Moving to projects directory: /var/lib/boinc-client_40000/slots/4/ICMGGhq0f+002688
Moving to projects directory: /var/lib/boinc-client_40000/slots/4/ICMSHhq0f+002688
Moving to projects directory: /var/lib/boinc-client_40000/slots/4/ICMUAhq0f+002688
Adding to the zip: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_12176729/ICMGGhq0f+002664
Adding to the zip: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_12176729/ICMSHhq0f+002664
Adding to the zip: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_12176729/ICMUAhq0f+002664
Adding to the zip: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_12176729/ICMGGhq0f+002676
Adding to the zip: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_12176729/ICMSHhq0f+002676
Adding to the zip: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_12176729/ICMUAhq0f+002676
Zipping up the intermediate file: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_0085_1991050100_123_960_12176729_0_r826685277_111.zip
Uploading the intermediate file: upload_file_111.zip
  20:30:27 STEP 2689 H=2689:00 +CPU= 22.604
  20:30:51 STEP 2690 H=2690:00 +CPU= 24.596
  20:31:29 STEP 2691 H=2691:00 +CPU= 37.564
  20:31:52 STEP 2692 H=2692:00 +CPU= 23.096
  20:32:15 STEP 2693 H=2693:00 +CPU= 23.617
  20:32:51 STEP 2694 H=2694:00 +CPU= 35.211
  20:33:10 STEP 2695 H=2695:00 +CPU= 19.288
  20:33:31 STEP 2696 H=2696:00 +CPU= 21.222
  20:34:08 STEP 2697 H=2697:00 +CPU= 36.371
  20:34:31 STEP 2698 H=2698:00 +CPU= 23.897
  20:34:54 STEP 2699 H=2699:00 +CPU= 22.841
  20:35:32 STEP 2700 H=2700:00 +CPU= 37.432
Moving to projects directory: /var/lib/boinc-client_40000/slots/4/ICMGGhq0f+002700
Moving to projects directory: /var/lib/boinc-client_40000/slots/4/ICMSHhq0f+002700
Moving to projects directory: /var/lib/boinc-client_40000/slots/4/ICMUAhq0f+002700
  20:35:54 STEP 2701 H=2701:00 +CPU= 21.748
  20:36:16 STEP 2702 H=2702:00 +CPU= 22.642
  20:36:50 STEP 2703 H=2703:00 +CPU= 34.132
  20:37:09 STEP 2704 H=2704:00 +CPU= 18.383
  20:37:32 STEP 2705 H=2705:00 +CPU= 22.930
  20:38:10 STEP 2706 H=2706:00 +CPU= 38.182
  20:38:35 STEP 2707 H=2707:00 +CPU= 24.787
  20:38:57 STEP 2708 H=2708:00 +CPU= 22.097
  20:39:32 STEP 2709 H=2709:00 +CPU= 34.897
  20:39:58 STEP 2710 H=2710:00 +CPU= 25.995
  20:40:20 STEP 2711 H=2711:00 +CPU= 22.280
  20:41:00 STEP 2712 H=2712:00 +CPU= 40.442
Moving to projects directory: /var/lib/boinc-client_40000/slots/4/ICMGGhq0f+002712
Moving to projects directory: /var/lib/boinc-client_40000/slots/4/ICMSHhq0f+002712
Moving to projects directory: /var/lib/boinc-client_40000/slots/4/ICMUAhq0f+002712
Adding to the zip: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_12176729/ICMGGhq0f+002688
Adding to the zip: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_12176729/ICMSHhq0f+002688
Adding to the zip: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_12176729/ICMUAhq0f+002688
Adding to the zip: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_12176729/ICMGGhq0f+002700
Adding to the zip: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_12176729/ICMSHhq0f+002700
Adding to the zip: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_12176729/ICMUAhq0f+002700
Zipping up the intermediate file: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_0085_1991050100_123_960_12176729_0_r826685277_112.zip
Uploading the intermediate file: upload_file_112.zip
  20:41:22 STEP 2713 H=2713:00 +CPU= 21.961
  20:41:48 STEP 2714 H=2714:00 +CPU= 25.912
  20:42:24 STEP 2715 H=2715:00 +CPU= 36.152
  20:42:48 STEP 2716 H=2716:00 +CPU= 23.204
  20:43:08 STEP 2717 H=2717:00 +CPU= 20.414
  20:43:45 STEP 2718 H=2718:00 +CPU= 37.379
  20:44:05 STEP 2719 H=2719:00 +CPU= 19.213
  20:44:28 STEP 2720 H=2720:00 +CPU= 23.581
  20:45:03 STEP 2721 H=2721:00 +CPU= 34.816
  20:45:25 STEP 2722 H=2722:00 +CPU= 21.902
  20:45:47 STEP 2723 H=2723:00 +CPU= 21.888
  20:46:23 STEP 2724 H=2724:00 +CPU= 36.322
Moving to projects directory: /var/lib/boinc-client_40000/slots/4/ICMGGhq0f+002724
Moving to projects directory: /var/lib/boinc-client_40000/slots/4/ICMSHhq0f+002724
Moving to projects directory: /var/lib/boinc-client_40000/slots/4/ICMUAhq0f+002724
  20:46:46 STEP 2725 H=2725:00 +CPU= 23.207
  20:47:09 STEP 2726 H=2726:00 +CPU= 23.013
  20:47:44 STEP 2727 H=2727:00 +CPU= 34.931
  20:48:08 STEP 2728 H=2728:00 +CPU= 23.728
  20:48:30 STEP 2729 H=2729:00 +CPU= 21.804
  20:49:06 STEP 2730 H=2730:00 +CPU= 35.806
  20:49:24 STEP 2731 H=2731:00 +CPU= 18.723
  20:49:48 STEP 2732 H=2732:00 +CPU= 23.829
  20:50:23 STEP 2733 H=2733:00 +CPU= 34.392
  20:50:47 STEP 2734 H=2734:00 +CPU= 24.290
  20:51:11 STEP 2735 H=2735:00 +CPU= 24.218
  20:51:53 STEP 2736 H=2736:00 +CPU= 41.393
Moving to projects directory: /var/lib/boinc-client_40000/slots/4/ICMGGhq0f+002736
Moving to projects directory: /var/lib/boinc-client_40000/slots/4/ICMSHhq0f+002736
Moving to projects directory: /var/lib/boinc-client_40000/slots/4/ICMUAhq0f+002736
Adding to the zip: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_12176729/ICMGGhq0f+002712
Adding to the zip: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_12176729/ICMSHhq0f+002712
Adding to the zip: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_12176729/ICMUAhq0f+002712
Adding to the zip: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_12176729/ICMGGhq0f+002724
Adding to the zip: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_12176729/ICMSHhq0f+002724
Adding to the zip: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_12176729/ICMUAhq0f+002724
Zipping up the intermediate file: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_0085_1991050100_123_960_12176729_0_r826685277_113.zip
Uploading the intermediate file: upload_file_113.zip
  20:52:15 STEP 2737 H=2737:00 +CPU= 22.862
  20:52:39 STEP 2738 H=2738:00 +CPU= 23.668
  20:53:14 STEP 2739 H=2739:00 +CPU= 34.944
  20:53:38 STEP 2740 H=2740:00 +CPU= 23.776
  20:53:54 STEP 2741 H=2741:00 +CPU= 16.631
  20:54:25 STEP 2742 H=2742:00 +CPU= 30.230
  20:54:45 STEP 2743 H=2743:00 +CPU= 20.601
  20:55:08 STEP 2744 H=2744:00 +CPU= 22.796
  20:55:44 STEP 2745 H=2745:00 +CPU= 35.436
  20:56:06 STEP 2746 H=2746:00 +CPU= 22.027
  20:56:28 STEP 2747 H=2747:00 +CPU= 22.331
  20:57:04 STEP 2748 H=2748:00 +CPU= 35.682
Moving to projects directory: /var/lib/boinc-client_40000/slots/4/ICMGGhq0f+002748
Moving to projects directory: /var/lib/boinc-client_40000/slots/4/ICMSHhq0f+002748
Moving to projects directory: /var/lib/boinc-client_40000/slots/4/ICMUAhq0f+002748
  20:57:27 STEP 2749 H=2749:00 +CPU= 23.058
  20:57:49 STEP 2750 H=2750:00 +CPU= 22.627
  20:58:23 STEP 2751 H=2751:00 +CPU= 33.293
  20:58:48 STEP 2752 H=2752:00 +CPU= 25.383
  20:59:11 STEP 2753 H=2753:00 +CPU= 23.045
  20:59:48 STEP 2754 H=2754:00 +CPU= 37.008
  21:00:05 STEP 2755 H=2755:00 +CPU= 17.049
  21:00:26 STEP 2756 H=2756:00 +CPU= 20.821
  21:00:58 STEP 2757 H=2757:00 +CPU= 31.538
  21:01:18 STEP 2758 H=2758:00 +CPU= 20.006
  21:01:40 STEP 2759 H=2759:00 +CPU= 22.496
  21:02:19 STEP 2760 H=2760:00 +CPU= 38.927
Moving to projects directory: /var/lib/boinc-client_40000/slots/4/ICMGGhq0f+002760
Moving to projects directory: /var/lib/boinc-client_40000/slots/4/ICMSHhq0f+002760
Moving to projects directory: /var/lib/boinc-client_40000/slots/4/ICMUAhq0f+002760
Adding to the zip: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_12176729/ICMGGhq0f+002736
Adding to the zip: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_12176729/ICMSHhq0f+002736
Adding to the zip: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_12176729/ICMUAhq0f+002736
Adding to the zip: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_12176729/ICMGGhq0f+002748
Adding to the zip: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_12176729/ICMSHhq0f+002748
Adding to the zip: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_12176729/ICMUAhq0f+002748
Zipping up the intermediate file: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_0085_1991050100_123_960_12176729_0_r826685277_114.zip
Uploading the intermediate file: upload_file_114.zip
  21:02:41 STEP 2761 H=2761:00 +CPU= 21.684
  21:03:03 STEP 2762 H=2762:00 +CPU= 22.049
  21:03:40 STEP 2763 H=2763:00 +CPU= 36.882
  21:04:02 STEP 2764 H=2764:00 +CPU= 22.678
  21:04:25 STEP 2765 H=2765:00 +CPU= 22.742
  21:04:54 STEP 2766 H=2766:00 +CPU= 29.361
  21:05:17 STEP 2767 H=2767:00 +CPU= 22.893
  21:05:42 STEP 2768 H=2768:00 +CPU= 25.037
  21:06:18 STEP 2769 H=2769:00 +CPU= 35.709
  21:06:37 STEP 2770 H=2770:00 +CPU= 18.916
  21:07:02 STEP 2771 H=2771:00 +CPU= 24.510
  21:07:43 STEP 2772 H=2772:00 +CPU= 41.179
Moving to projects directory: /var/lib/boinc-client_40000/slots/4/ICMGGhq0f+002772
Moving to projects directory: /var/lib/boinc-client_40000/slots/4/ICMSHhq0f+002772
Moving to projects directory: /var/lib/boinc-client_40000/slots/4/ICMUAhq0f+002772
  21:08:02 STEP 2773 H=2773:00 +CPU= 18.978
  21:08:23 STEP 2774 H=2774:00 +CPU= 21.004
  21:08:57 STEP 2775 H=2775:00 +CPU= 34.545
  21:09:15 STEP 2776 H=2776:00 +CPU= 17.711
  21:09:38 STEP 2777 H=2777:00 +CPU= 23.462
  21:10:17 STEP 2778 H=2778:00 +CPU= 38.499
  21:10:39 STEP 2779 H=2779:00 +CPU= 22.296
  21:11:05 STEP 2780 H=2780:00 +CPU= 25.742
  21:11:45 STEP 2781 H=2781:00 +CPU= 39.669
  21:12:11 STEP 2782 H=2782:00 +CPU= 26.132
  21:12:36 STEP 2783 H=2783:00 +CPU= 25.361
  21:13:16 STEP 2784 H=2784:00 +CPU= 39.651
Moving to projects directory: /var/lib/boinc-client_40000/slots/4/ICMGGhq0f+002784
Moving to projects directory: /var/lib/boinc-client_40000/slots/4/ICMSHhq0f+002784
Moving to projects directory: /var/lib/boinc-client_40000/slots/4/ICMUAhq0f+002784
Adding to the zip: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_12176729/ICMGGhq0f+002760
Adding to the zip: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_12176729/ICMSHhq0f+002760
Adding to the zip: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_12176729/ICMUAhq0f+002760
Adding to the zip: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_12176729/ICMGGhq0f+002772
Adding to the zip: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_12176729/ICMSHhq0f+002772
Adding to the zip: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_12176729/ICMUAhq0f+002772
Zipping up the intermediate file: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_0085_1991050100_123_960_12176729_0_r826685277_115.zip
Uploading the intermediate file: upload_file_115.zip
  21:13:41 STEP 2785 H=2785:00 +CPU= 25.405
  21:14:04 STEP 2786 H=2786:00 +CPU= 22.508
  21:14:35 STEP 2787 H=2787:00 +CPU= 31.638
  21:14:58 STEP 2788 H=2788:00 +CPU= 22.825
  21:15:21 STEP 2789 H=2789:00 +CPU= 22.640
  21:15:58 STEP 2790 H=2790:00 +CPU= 37.030
  21:16:17 STEP 2791 H=2791:00 +CPU= 19.270
  21:16:35 STEP 2792 H=2792:00 +CPU= 18.293
  21:17:10 STEP 2793 H=2793:00 +CPU= 34.716
  21:17:33 STEP 2794 H=2794:00 +CPU= 22.526
  21:17:51 STEP 2795 H=2795:00 +CPU= 18.628
  21:18:27 STEP 2796 H=2796:00 +CPU= 35.407
Moving to projects directory: /var/lib/boinc-client_40000/slots/4/ICMGGhq0f+002796
Moving to projects directory: /var/lib/boinc-client_40000/slots/4/ICMSHhq0f+002796
Moving to projects directory: /var/lib/boinc-client_40000/slots/4/ICMUAhq0f+002796
  21:18:49 STEP 2797 H=2797:00 +CPU= 22.127
  21:19:11 STEP 2798 H=2798:00 +CPU= 22.103
  21:19:41 STEP 2799 H=2799:00 +CPU= 29.904
  21:20:04 STEP 2800 H=2800:00 +CPU= 22.854
  21:20:22 STEP 2801 H=2801:00 +CPU= 18.549
  21:20:56 STEP 2802 H=2802:00 +CPU= 33.797
  21:21:15 STEP 2803 H=2803:00 +CPU= 18.871
  21:21:39 STEP 2804 H=2804:00 +CPU= 23.801
  21:22:16 STEP 2805 H=2805:00 +CPU= 36.974
  21:22:40 STEP 2806 H=2806:00 +CPU= 24.353
  21:23:05 STEP 2807 H=2807:00 +CPU= 24.381
  21:23:47 STEP 2808 H=2808:00 +CPU= 42.407
Moving to projects directory: /var/lib/boinc-client_40000/slots/4/ICMGGhq0f+002808
Moving to projects directory: /var/lib/boinc-client_40000/slots/4/ICMSHhq0f+002808
Moving to projects directory: /var/lib/boinc-client_40000/slots/4/ICMUAhq0f+002808
Adding to the zip: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_12176729/ICMGGhq0f+002784
Adding to the zip: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_12176729/ICMSHhq0f+002784
Adding to the zip: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_12176729/ICMUAhq0f+002784
Adding to the zip: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_12176729/ICMGGhq0f+002796
Adding to the zip: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_12176729/ICMSHhq0f+002796
Adding to the zip: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_12176729/ICMUAhq0f+002796
Zipping up the intermediate file: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_0085_1991050100_123_960_12176729_0_r826685277_116.zip
Uploading the intermediate file: upload_file_116.zip
  21:24:10 STEP 2809 H=2809:00 +CPU= 22.866
  21:24:31 STEP 2810 H=2810:00 +CPU= 21.412
  21:25:05 STEP 2811 H=2811:00 +CPU= 33.870
  21:25:30 STEP 2812 H=2812:00 +CPU= 25.040
  21:25:54 STEP 2813 H=2813:00 +CPU= 24.137
  21:26:31 STEP 2814 H=2814:00 +CPU= 36.525
  21:26:50 STEP 2815 H=2815:00 +CPU= 18.840
  21:27:10 STEP 2816 H=2816:00 +CPU= 20.274
  21:27:46 STEP 2817 H=2817:00 +CPU= 35.735
  21:28:08 STEP 2818 H=2818:00 +CPU= 22.380
  21:28:32 STEP 2819 H=2819:00 +CPU= 23.740
  21:29:15 STEP 2820 H=2820:00 +CPU= 43.398
Moving to projects directory: /var/lib/boinc-client_40000/slots/4/ICMGGhq0f+002820
Moving to projects directory: /var/lib/boinc-client_40000/slots/4/ICMSHhq0f+002820
Moving to projects directory: /var/lib/boinc-client_40000/slots/4/ICMUAhq0f+002820
  21:29:37 STEP 2821 H=2821:00 +CPU= 21.823
  21:29:55 STEP 2822 H=2822:00 +CPU= 18.281
  21:30:24 STEP 2823 H=2823:00 +CPU= 28.672
  21:30:42 STEP 2824 H=2824:00 +CPU= 18.253
  21:31:05 STEP 2825 H=2825:00 +CPU= 22.249
  21:31:41 STEP 2826 H=2826:00 +CPU= 36.384
  21:32:02 STEP 2827 H=2827:00 +CPU= 21.067
  21:32:24 STEP 2828 H=2828:00 +CPU= 21.809
  21:33:00 STEP 2829 H=2829:00 +CPU= 35.763
  21:33:25 STEP 2830 H=2830:00 +CPU= 25.258
  21:33:47 STEP 2831 H=2831:00 +CPU= 22.299
  21:34:28 STEP 2832 H=2832:00 +CPU= 41.298
Moving to projects directory: /var/lib/boinc-client_40000/slots/4/ICMGGhq0f+002832
Moving to projects directory: /var/lib/boinc-client_40000/slots/4/ICMSHhq0f+002832
Moving to projects directory: /var/lib/boinc-client_40000/slots/4/ICMUAhq0f+002832
Adding to the zip: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_12176729/ICMGGhq0f+002808
Adding to the zip: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_12176729/ICMSHhq0f+002808
Adding to the zip: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_12176729/ICMUAhq0f+002808
Adding to the zip: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_12176729/ICMGGhq0f+002820
Adding to the zip: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_12176729/ICMSHhq0f+002820
Adding to the zip: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_12176729/ICMUAhq0f+002820
Zipping up the intermediate file: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_0085_1991050100_123_960_12176729_0_r826685277_117.zip
Uploading the intermediate file: upload_file_117.zip
  21:34:54 STEP 2833 H=2833:00 +CPU= 25.294
  21:35:16 STEP 2834 H=2834:00 +CPU= 22.144
  21:35:53 STEP 2835 H=2835:00 +CPU= 36.746
  21:36:17 STEP 2836 H=2836:00 +CPU= 24.032
  21:36:38 STEP 2837 H=2837:00 +CPU= 21.518
  21:37:16 STEP 2838 H=2838:00 +CPU= 37.996
  21:37:36 STEP 2839 H=2839:00 +CPU= 19.315
  21:37:59 STEP 2840 H=2840:00 +CPU= 23.799
  21:38:36 STEP 2841 H=2841:00 +CPU= 36.645
  21:39:02 STEP 2842 H=2842:00 +CPU= 26.038
  21:39:27 STEP 2843 H=2843:00 +CPU= 25.187
  21:40:02 STEP 2844 H=2844:00 +CPU= 34.607
Moving to projects directory: /var/lib/boinc-client_40000/slots/4/ICMGGhq0f+002844
Moving to projects directory: /var/lib/boinc-client_40000/slots/4/ICMSHhq0f+002844
Moving to projects directory: /var/lib/boinc-client_40000/slots/4/ICMUAhq0f+002844
  21:40:26 STEP 2845 H=2845:00 +CPU= 24.488
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 1991050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0085
(argv4) batchid: 960
(argv5) wuid: 12176729
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client_40000/slots/4
Project directory is: /var/lib/boinc-client_40000/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_12176729
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client_40000/slots/4/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client_40000/slots/4/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_67e954eea24220fb7d10bf54d05cbdfc to: /var/lib/boinc-client_40000/slots/4/oifs_43r3_ps_0085_1991050100_123_960_12176729.zip
Unzipping the namelist zip file: /var/lib/boinc-client_40000/slots/4/oifs_43r3_ps_0085_1991050100_123_960_12176729.zip
ic_ancil_file: ic_ancil_12176729
ifsdata_file: ifsdata_12176729
climate_data_file: clim_data_12176729
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_db87540e98a3a0efd46881405dbb7807 to: /var/lib/boinc-client_40000/slots/4/ic_ancil_12176729.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client_40000/slots/4/ic_ancil_12176729.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_0949eccd6e810ff71bfa0ab9d379e664 to: /var/lib/boinc-client_40000/slots/4/ifsdata/ifsdata_12176729.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client_40000/slots/4/ifsdata/ifsdata_12176729.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client_40000/slots/4/159l_2/clim_data_12176729.zip
Unzipping the climate data zip file: /var/lib/boinc-client_40000/slots/4/159l_2/clim_data_12176729.zip
Checking for progress XML file: /var/lib/boinc-client_40000/slots/4/progress_file_12176729.xml
Opened progress file ok : /var/lib/boinc-client_40000/slots/4/progress_file_12176729.xml
-- Model is restarting --
Adjusting last_iter, 2846, to previous model restart step.
Creating progress file: /var/lib/boinc-client_40000/slots/4/progress_file_12176729.xml
last_cpu_time: 77288
upload_file_number: 118
last_iter: 2831
last_upload: 10195200
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0085_1991050100_123_960_12176729_0_r826685277
The child process has been launched with process id: 1415
Executing the command: /var/lib/boinc-client_40000/slots/4/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:w1:1:1:1415:1415] [20230123:215456:1674510896:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:w1:1:1:1415:1415] [20230123:215456:1674510896:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:w1:1:1:1415:1415] [20230123:215456:1674510896:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:w1:1:1:1415:1415] [20230123:215456:1674510896:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:w1:1:1:1415:1415] [20230123:215456:1674510896:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:w1:1:1:1415:1415] [20230123:215456:1674510896:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:w1:1:1:1415:1415] [20230123:215456:1674510896:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:w1:1:1:1415:1415] [20230123:215456:1674510896:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:w1:1:1:1415:1415] [20230123:215456:1674510896:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:w1:1:1:1415:1415] [20230123:215456:1674510896:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:w1:1:1:1415:1415] [20230123:215456:1674510896:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:w1:1:1:1415:1415] [20230123:215456:1674510896:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:w1:1:1:1415:1415] [20230123:215456:1674510896:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:w1:1:1:1415:1415] [20230123:215456:1674510896:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:w1:1:1:1415:1415] [20230123:215456:1674510896:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:w1:1:1:1415:1415] [20230123:215456:1674510896:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:w1:1:1:1415:1415] [20230123:215456:1674510896:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:w1:1:1:1415:1415] [20230123:215456:1674510896:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:w1:1:1:1415:1415] [20230123:215456:1674510896:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:w1:1:1:1415:1415] [20230123:215456:1674510896:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:w1:1:1:1415:1415] [20230123:215456:1674510896:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:w1:1:1:1415:1415] [20230123:215456:1674510896:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:w1:1:1:1415:1415] [20230123:215456:1674510896:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:w1:1:1:1415:1415] [20230123:215456:1674510896:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:w1:1:1:1415:1415] [20230123:215456:1674510896:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:w1:1:1:1415:1415] [20230123:215456:1674510896:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:w1:1:1:1415:1415] [20230123:215456:1674510896:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:w1:1:1:1415:1415] [20230123:215456:1674510896:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:w1:1:1:1415:1415] [20230123:215456:1674510896:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:w1:1:1:1415:1415] [20230123:215456:1674510896:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  21:56:20 STEP 2832 H=2832:00 +CPU= 43.252
  21:56:43 STEP 2833 H=2833:00 +CPU= 23.083
  21:57:04 STEP 2834 H=2834:00 +CPU= 20.769
  21:57:41 STEP 2835 H=2835:00 +CPU= 37.439
  21:58:05 STEP 2836 H=2836:00 +CPU= 23.767
  21:58:27 STEP 2837 H=2837:00 +CPU= 21.619
  21:59:03 STEP 2838 H=2838:00 +CPU= 36.384
  21:59:24 STEP 2839 H=2839:00 +CPU= 21.481
  21:59:48 STEP 2840 H=2840:00 +CPU= 23.768
  22:00:20 STEP 2841 H=2841:00 +CPU= 32.266
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 1991050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0085
(argv4) batchid: 960
(argv5) wuid: 12176729
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client_40000/slots/4
Project directory is: /var/lib/boinc-client_40000/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_12176729
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client_40000/slots/4/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client_40000/slots/4/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_67e954eea24220fb7d10bf54d05cbdfc to: /var/lib/boinc-client_40000/slots/4/oifs_43r3_ps_0085_1991050100_123_960_12176729.zip
Unzipping the namelist zip file: /var/lib/boinc-client_40000/slots/4/oifs_43r3_ps_0085_1991050100_123_960_12176729.zip
ic_ancil_file: ic_ancil_12176729
ifsdata_file: ifsdata_12176729
climate_data_file: clim_data_12176729
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_db87540e98a3a0efd46881405dbb7807 to: /var/lib/boinc-client_40000/slots/4/ic_ancil_12176729.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client_40000/slots/4/ic_ancil_12176729.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_0949eccd6e810ff71bfa0ab9d379e664 to: /var/lib/boinc-client_40000/slots/4/ifsdata/ifsdata_12176729.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client_40000/slots/4/ifsdata/ifsdata_12176729.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client_40000/slots/4/159l_2/clim_data_12176729.zip
Unzipping the climate data zip file: /var/lib/boinc-client_40000/slots/4/159l_2/clim_data_12176729.zip
Checking for progress XML file: /var/lib/boinc-client_40000/slots/4/progress_file_12176729.xml
Opened progress file ok : /var/lib/boinc-client_40000/slots/4/progress_file_12176729.xml
-- Model is restarting --
Adjusting last_iter, 2842, to previous model restart step.
Creating progress file: /var/lib/boinc-client_40000/slots/4/progress_file_12176729.xml
last_cpu_time: 77590
upload_file_number: 118
last_iter: 2831
last_upload: 10195200
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0085_1991050100_123_960_12176729_0_r826685277
The child process has been launched with process id: 1406
Executing the command: /var/lib/boinc-client_40000/slots/4/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:w1:1:1:1406:1406] [20230123:221214:1674511934:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:w1:1:1:1406:1406] [20230123:221214:1674511934:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:w1:1:1:1406:1406] [20230123:221214:1674511934:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:w1:1:1:1406:1406] [20230123:221214:1674511934:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:w1:1:1:1406:1406] [20230123:221214:1674511934:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:w1:1:1:1406:1406] [20230123:221214:1674511934:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:w1:1:1:1406:1406] [20230123:221214:1674511934:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:w1:1:1:1406:1406] [20230123:221214:1674511934:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:w1:1:1:1406:1406] [20230123:221214:1674511934:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:w1:1:1:1406:1406] [20230123:221214:1674511934:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:w1:1:1:1406:1406] [20230123:221214:1674511934:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:w1:1:1:1406:1406] [20230123:221214:1674511934:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:w1:1:1:1406:1406] [20230123:221214:1674511934:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:w1:1:1:1406:1406] [20230123:221214:1674511934:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:w1:1:1:1406:1406] [20230123:221214:1674511934:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:w1:1:1:1406:1406] [20230123:221214:1674511934:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:w1:1:1:1406:1406] [20230123:221214:1674511934:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:w1:1:1:1406:1406] [20230123:221214:1674511934:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:w1:1:1:1406:1406] [20230123:221214:1674511934:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:w1:1:1:1406:1406] [20230123:221214:1674511934:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:w1:1:1:1406:1406] [20230123:221214:1674511934:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:w1:1:1:1406:1406] [20230123:221214:1674511934:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:w1:1:1:1406:1406] [20230123:221214:1674511934:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:w1:1:1:1406:1406] [20230123:221214:1674511934:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:w1:1:1:1406:1406] [20230123:221214:1674511934:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:w1:1:1:1406:1406] [20230123:221214:1674511934:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:w1:1:1:1406:1406] [20230123:221214:1674511934:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:w1:1:1:1406:1406] [20230123:221214:1674511934:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:w1:1:1:1406:1406] [20230123:221214:1674511934:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:w1:1:1:1406:1406] [20230123:221214:1674511934:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  22:13:30 STEP 2832 H=2832:00 +CPU= 34.895
  22:13:48 STEP 2833 H=2833:00 +CPU= 18.626
  22:14:08 STEP 2834 H=2834:00 +CPU= 19.959
  22:14:45 STEP 2835 H=2835:00 +CPU= 36.308
  22:15:09 STEP 2836 H=2836:00 +CPU= 24.203
  22:15:30 STEP 2837 H=2837:00 +CPU= 20.886
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 1991050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0085
(argv4) batchid: 960
(argv5) wuid: 12176729
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client_40000/slots/4
Project directory is: /var/lib/boinc-client_40000/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_12176729
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client_40000/slots/4/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client_40000/slots/4/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_67e954eea24220fb7d10bf54d05cbdfc to: /var/lib/boinc-client_40000/slots/4/oifs_43r3_ps_0085_1991050100_123_960_12176729.zip
Unzipping the namelist zip file: /var/lib/boinc-client_40000/slots/4/oifs_43r3_ps_0085_1991050100_123_960_12176729.zip
ic_ancil_file: ic_ancil_12176729
ifsdata_file: ifsdata_12176729
climate_data_file: clim_data_12176729
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_db87540e98a3a0efd46881405dbb7807 to: /var/lib/boinc-client_40000/slots/4/ic_ancil_12176729.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client_40000/slots/4/ic_ancil_12176729.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_0949eccd6e810ff71bfa0ab9d379e664 to: /var/lib/boinc-client_40000/slots/4/ifsdata/ifsdata_12176729.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client_40000/slots/4/ifsdata/ifsdata_12176729.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client_40000/slots/4/159l_2/clim_data_12176729.zip
Unzipping the climate data zip file: /var/lib/boinc-client_40000/slots/4/159l_2/clim_data_12176729.zip
Checking for progress XML file: /var/lib/boinc-client_40000/slots/4/progress_file_12176729.xml
Opened progress file ok : /var/lib/boinc-client_40000/slots/4/progress_file_12176729.xml
-- Model is restarting --
Adjusting last_iter, 2838, to previous model restart step.
Creating progress file: /var/lib/boinc-client_40000/slots/4/progress_file_12176729.xml
last_cpu_time: 77771
upload_file_number: 118
last_iter: 2831
last_upload: 10195200
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0085_1991050100_123_960_12176729_0_r826685277
The child process has been launched with process id: 1418
Executing the command: /var/lib/boinc-client_40000/slots/4/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:w1:1:1:1418:1418] [20230123:222930:1674512970:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:w1:1:1:1418:1418] [20230123:222930:1674512970:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:w1:1:1:1418:1418] [20230123:222930:1674512970:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:w1:1:1:1418:1418] [20230123:222930:1674512970:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:w1:1:1:1418:1418] [20230123:222930:1674512970:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:w1:1:1:1418:1418] [20230123:222930:1674512970:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:w1:1:1:1418:1418] [20230123:222930:1674512970:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:w1:1:1:1418:1418] [20230123:222930:1674512970:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:w1:1:1:1418:1418] [20230123:222930:1674512970:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:w1:1:1:1418:1418] [20230123:222930:1674512970:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:w1:1:1:1418:1418] [20230123:222930:1674512970:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:w1:1:1:1418:1418] [20230123:222930:1674512970:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:w1:1:1:1418:1418] [20230123:222930:1674512970:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:w1:1:1:1418:1418] [20230123:222930:1674512970:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:w1:1:1:1418:1418] [20230123:222930:1674512970:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:w1:1:1:1418:1418] [20230123:222930:1674512970:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:w1:1:1:1418:1418] [20230123:222930:1674512970:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:w1:1:1:1418:1418] [20230123:222930:1674512970:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:w1:1:1:1418:1418] [20230123:222930:1674512970:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:w1:1:1:1418:1418] [20230123:222930:1674512970:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:w1:1:1:1418:1418] [20230123:222930:1674512970:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:w1:1:1:1418:1418] [20230123:222930:1674512970:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:w1:1:1:1418:1418] [20230123:222930:1674512970:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:w1:1:1:1418:1418] [20230123:222930:1674512970:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:w1:1:1:1418:1418] [20230123:222930:1674512970:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:w1:1:1:1418:1418] [20230123:222930:1674512970:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:w1:1:1:1418:1418] [20230123:222930:1674512970:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:w1:1:1:1418:1418] [20230123:222930:1674512970:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:w1:1:1:1418:1418] [20230123:222930:1674512970:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:w1:1:1:1418:1418] [20230123:222930:1674512970:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  22:30:41 STEP 2832 H=2832:00 +CPU= 32.272
  22:31:01 STEP 2833 H=2833:00 +CPU= 19.444
  22:31:21 STEP 2834 H=2834:00 +CPU= 20.754
  22:31:53 STEP 2835 H=2835:00 +CPU= 32.013
  22:32:13 STEP 2836 H=2836:00 +CPU= 19.841
  22:32:29 STEP 2837 H=2837:00 +CPU= 15.940
  22:33:00 STEP 2838 H=2838:00 +CPU= 30.551
  22:33:18 STEP 2839 H=2839:00 +CPU= 17.917
  22:33:38 STEP 2840 H=2840:00 +CPU= 20.036
  22:34:09 STEP 2841 H=2841:00 +CPU= 30.868
  22:34:30 STEP 2842 H=2842:00 +CPU= 21.273
  22:34:51 STEP 2843 H=2843:00 +CPU= 21.586
  22:35:29 STEP 2844 H=2844:00 +CPU= 37.566
Moving to projects directory: /var/lib/boinc-client_40000/slots/4/ICMGGhq0f+002844
Moving to projects directory: /var/lib/boinc-client_40000/slots/4/ICMSHhq0f+002844
Moving to projects directory: /var/lib/boinc-client_40000/slots/4/ICMUAhq0f+002844
  22:35:49 STEP 2845 H=2845:00 +CPU= 20.317
  22:36:10 STEP 2846 H=2846:00 +CPU= 20.195
  22:36:41 STEP 2847 H=2847:00 +CPU= 31.795
  22:37:03 STEP 2848 H=2848:00 +CPU= 21.239
  22:37:23 STEP 2849 H=2849:00 +CPU= 20.519
  22:37:56 STEP 2850 H=2850:00 +CPU= 32.658
  22:38:16 STEP 2851 H=2851:00 +CPU= 19.957
  22:38:37 STEP 2852 H=2852:00 +CPU= 20.895
  22:39:09 STEP 2853 H=2853:00 +CPU= 32.482
  22:39:31 STEP 2854 H=2854:00 +CPU= 22.331
  22:39:53 STEP 2855 H=2855:00 +CPU= 22.031
  22:40:32 STEP 2856 H=2856:00 +CPU= 38.177
Moving to projects directory: /var/lib/boinc-client_40000/slots/4/ICMGGhq0f+002856
Moving to projects directory: /var/lib/boinc-client_40000/slots/4/ICMSHhq0f+002856
Moving to projects directory: /var/lib/boinc-client_40000/slots/4/ICMUAhq0f+002856
Adding to the zip: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_12176729/ICMGGhq0f+002832
Adding to the zip: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_12176729/ICMSHhq0f+002832
Adding to the zip: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_12176729/ICMUAhq0f+002832
Adding to the zip: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_12176729/ICMGGhq0f+002844
Adding to the zip: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_12176729/ICMSHhq0f+002844
Adding to the zip: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_12176729/ICMUAhq0f+002844
Zipping up the intermediate file: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_0085_1991050100_123_960_12176729_0_r826685277_118.zip
Uploading the intermediate file: upload_file_118.zip
  22:40:50 STEP 2857 H=2857:00 +CPU= 18.487
  22:41:12 STEP 2858 H=2858:00 +CPU= 21.331
  22:41:43 STEP 2859 H=2859:00 +CPU= 31.747
  22:42:03 STEP 2860 H=2860:00 +CPU= 19.456
  22:42:21 STEP 2861 H=2861:00 +CPU= 18.286
  22:42:49 STEP 2862 H=2862:00 +CPU= 27.606
  22:43:05 STEP 2863 H=2863:00 +CPU= 15.933
  22:43:23 STEP 2864 H=2864:00 +CPU= 18.650
  22:43:55 STEP 2865 H=2865:00 +CPU= 32.241
  22:44:12 STEP 2866 H=2866:00 +CPU= 16.912
  22:44:31 STEP 2867 H=2867:00 +CPU= 18.868
  22:45:06 STEP 2868 H=2868:00 +CPU= 34.494
Moving to projects directory: /var/lib/boinc-client_40000/slots/4/ICMGGhq0f+002868
Moving to projects directory: /var/lib/boinc-client_40000/slots/4/ICMSHhq0f+002868
Moving to projects directory: /var/lib/boinc-client_40000/slots/4/ICMUAhq0f+002868
  22:45:23 STEP 2869 H=2869:00 +CPU= 17.684
  22:45:44 STEP 2870 H=2870:00 +CPU= 20.495
  22:46:14 STEP 2871 H=2871:00 +CPU= 29.651
  22:46:36 STEP 2872 H=2872:00 +CPU= 22.073
  22:46:55 STEP 2873 H=2873:00 +CPU= 19.802
  22:47:23 STEP 2874 H=2874:00 +CPU= 27.518
  22:47:41 STEP 2875 H=2875:00 +CPU= 18.536
  22:48:00 STEP 2876 H=2876:00 +CPU= 18.625
  22:48:30 STEP 2877 H=2877:00 +CPU= 30.355
  22:48:45 STEP 2878 H=2878:00 +CPU= 14.396
  22:49:04 STEP 2879 H=2879:00 +CPU= 18.872
  22:49:38 STEP 2880 H=2880:00 +CPU= 34.365
Moving to projects directory: /var/lib/boinc-client_40000/slots/4/ICMGGhq0f+002880
Moving to projects directory: /var/lib/boinc-client_40000/slots/4/ICMSHhq0f+002880
Moving to projects directory: /var/lib/boinc-client_40000/slots/4/ICMUAhq0f+002880
Adding to the zip: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_12176729/ICMGGhq0f+002856
Adding to the zip: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_12176729/ICMSHhq0f+002856
Adding to the zip: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_12176729/ICMUAhq0f+002856
Adding to the zip: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_12176729/ICMGGhq0f+002868
Adding to the zip: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_12176729/ICMSHhq0f+002868
Adding to the zip: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_12176729/ICMUAhq0f+002868
Zipping up the intermediate file: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_0085_1991050100_123_960_12176729_0_r826685277_119.zip
Uploading the intermediate file: upload_file_119.zip
  22:49:59 STEP 2881 H=2881:00 +CPU= 20.787
  22:50:19 STEP 2882 H=2882:00 +CPU= 19.616
  22:50:52 STEP 2883 H=2883:00 +CPU= 33.537
  22:51:14 STEP 2884 H=2884:00 +CPU= 22.072
  22:51:33 STEP 2885 H=2885:00 +CPU= 18.733
  22:52:07 STEP 2886 H=2886:00 +CPU= 34.350
  22:52:26 STEP 2887 H=2887:00 +CPU= 19.125
  22:52:48 STEP 2888 H=2888:00 +CPU= 21.583
  22:53:20 STEP 2889 H=2889:00 +CPU= 32.464
  22:53:39 STEP 2890 H=2890:00 +CPU= 18.249
  22:53:59 STEP 2891 H=2891:00 +CPU= 20.379
  22:54:35 STEP 2892 H=2892:00 +CPU= 35.563
Moving to projects directory: /var/lib/boinc-client_40000/slots/4/ICMGGhq0f+002892
Moving to projects directory: /var/lib/boinc-client_40000/slots/4/ICMSHhq0f+002892
Moving to projects directory: /var/lib/boinc-client_40000/slots/4/ICMUAhq0f+002892
  22:54:53 STEP 2893 H=2893:00 +CPU= 18.020
  22:55:13 STEP 2894 H=2894:00 +CPU= 20.783
  22:55:44 STEP 2895 H=2895:00 +CPU= 30.197
  22:56:06 STEP 2896 H=2896:00 +CPU= 21.963
  22:56:27 STEP 2897 H=2897:00 +CPU= 21.127
  22:57:00 STEP 2898 H=2898:00 +CPU= 33.206
  22:57:21 STEP 2899 H=2899:00 +CPU= 21.201
  22:57:37 STEP 2900 H=2900:00 +CPU= 15.492
  22:58:06 STEP 2901 H=2901:00 +CPU= 29.620
  22:58:27 STEP 2902 H=2902:00 +CPU= 20.481
  22:58:47 STEP 2903 H=2903:00 +CPU= 20.039
  22:59:25 STEP 2904 H=2904:00 +CPU= 38.661
Moving to projects directory: /var/lib/boinc-client_40000/slots/4/ICMGGhq0f+002904
Moving to projects directory: /var/lib/boinc-client_40000/slots/4/ICMSHhq0f+002904
Moving to projects directory: /var/lib/boinc-client_40000/slots/4/ICMUAhq0f+002904
Adding to the zip: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_12176729/ICMGGhq0f+002880
Adding to the zip: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_12176729/ICMSHhq0f+002880
Adding to the zip: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_12176729/ICMUAhq0f+002880
Adding to the zip: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_12176729/ICMGGhq0f+002892
Adding to the zip: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_12176729/ICMSHhq0f+002892
Adding to the zip: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_12176729/ICMUAhq0f+002892
Zipping up the intermediate file: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_0085_1991050100_123_960_12176729_0_r826685277_120.zip
Uploading the intermediate file: upload_file_120.zip
  22:59:46 STEP 2905 H=2905:00 +CPU= 20.609
  23:00:04 STEP 2906 H=2906:00 +CPU= 17.659
  23:00:32 STEP 2907 H=2907:00 +CPU= 28.690
  23:00:51 STEP 2908 H=2908:00 +CPU= 18.936
  23:01:11 STEP 2909 H=2909:00 +CPU= 20.010
  23:01:42 STEP 2910 H=2910:00 +CPU= 30.549
  23:02:03 STEP 2911 H=2911:00 +CPU= 20.662
  23:02:23 STEP 2912 H=2912:00 +CPU= 20.268
  23:02:56 STEP 2913 H=2913:00 +CPU= 32.943
  23:03:18 STEP 2914 H=2914:00 +CPU= 22.041
  23:03:40 STEP 2915 H=2915:00 +CPU= 21.660
  23:04:17 STEP 2916 H=2916:00 +CPU= 37.802
Moving to projects directory: /var/lib/boinc-client_40000/slots/4/ICMGGhq0f+002916
Moving to projects directory: /var/lib/boinc-client_40000/slots/4/ICMSHhq0f+002916
Moving to projects directory: /var/lib/boinc-client_40000/slots/4/ICMUAhq0f+002916
  23:04:39 STEP 2917 H=2917:00 +CPU= 21.623
  23:05:01 STEP 2918 H=2918:00 +CPU= 21.996
  23:05:30 STEP 2919 H=2919:00 +CPU= 28.925
  23:05:52 STEP 2920 H=2920:00 +CPU= 22.426
  23:06:09 STEP 2921 H=2921:00 +CPU= 16.663
  23:06:37 STEP 2922 H=2922:00 +CPU= 27.729
  23:06:58 STEP 2923 H=2923:00 +CPU= 21.520
  23:07:20 STEP 2924 H=2924:00 +CPU= 22.071
  23:07:51 STEP 2925 H=2925:00 +CPU= 31.041
  23:08:09 STEP 2926 H=2926:00 +CPU= 17.623
  23:08:26 STEP 2927 H=2927:00 +CPU= 17.405
  23:09:05 STEP 2928 H=2928:00 +CPU= 38.572
Moving to projects directory: /var/lib/boinc-client_40000/slots/4/ICMGGhq0f+002928
Moving to projects directory: /var/lib/boinc-client_40000/slots/4/ICMSHhq0f+002928
Moving to projects directory: /var/lib/boinc-client_40000/slots/4/ICMUAhq0f+002928
Adding to the zip: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_12176729/ICMGGhq0f+002904
Adding to the zip: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_12176729/ICMSHhq0f+002904
Adding to the zip: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_12176729/ICMUAhq0f+002904
Adding to the zip: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_12176729/ICMGGhq0f+002916
Adding to the zip: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_12176729/ICMSHhq0f+002916
Adding to the zip: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_12176729/ICMUAhq0f+002916
Zipping up the intermediate file: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_0085_1991050100_123_960_12176729_0_r826685277_121.zip
Uploading the intermediate file: upload_file_121.zip
  23:09:25 STEP 2929 H=2929:00 +CPU= 20.450
  23:09:47 STEP 2930 H=2930:00 +CPU= 21.103
  23:10:18 STEP 2931 H=2931:00 +CPU= 31.198
  23:10:39 STEP 2932 H=2932:00 +CPU= 20.907
  23:10:59 STEP 2933 H=2933:00 +CPU= 20.319
  23:11:30 STEP 2934 H=2934:00 +CPU= 31.288
  23:11:49 STEP 2935 H=2935:00 +CPU= 18.991
  23:12:10 STEP 2936 H=2936:00 +CPU= 21.094
  23:12:43 STEP 2937 H=2937:00 +CPU= 32.372
  23:13:04 STEP 2938 H=2938:00 +CPU= 21.099
  23:13:24 STEP 2939 H=2939:00 +CPU= 19.939
  23:13:58 STEP 2940 H=2940:00 +CPU= 34.495
Moving to projects directory: /var/lib/boinc-client_40000/slots/4/ICMGGhq0f+002940
Moving to projects directory: /var/lib/boinc-client_40000/slots/4/ICMSHhq0f+002940
Moving to projects directory: /var/lib/boinc-client_40000/slots/4/ICMUAhq0f+002940
  23:14:19 STEP 2941 H=2941:00 +CPU= 21.223
  23:14:40 STEP 2942 H=2942:00 +CPU= 20.761
  23:15:09 STEP 2943 H=2943:00 +CPU= 28.455
  23:15:25 STEP 2944 H=2944:00 +CPU= 15.920
  23:15:46 STEP 2945 H=2945:00 +CPU= 21.057
  23:16:14 STEP 2946 H=2946:00 +CPU= 28.547
  23:16:31 STEP 2947 H=2947:00 +CPU= 17.134
  23:16:49 STEP 2948 H=2948:00 +CPU= 17.355
  23:17:18 STEP 2949 H=2949:00 +CPU= 28.773
  23:17:39 STEP 2950 H=2950:00 +CPU= 21.885
  23:18:01 STEP 2951 H=2951:00 +CPU= 21.171
  23:18:36 STEP 2952 H=2952:00 +CPU= 35.683
The child process terminated with status: 0
Moving to projects directory: /var/lib/boinc-client_40000/slots/4/ICMGGhq0f+002952
Moving to projects directory: /var/lib/boinc-client_40000/slots/4/ICMSHhq0f+002952
Moving to projects directory: /var/lib/boinc-client_40000/slots/4/ICMUAhq0f+002952
Adding to the zip: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_12176729/ICMUAhq0f+002928
Adding to the zip: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_12176729/ICMGGhq0f+002940
Adding to the zip: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_12176729/ICMUAhq0f+002940
Adding to the zip: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_12176729/ICMGGhq0f+002928
Adding to the zip: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_12176729/ICMSHhq0f+002952
Adding to the zip: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_12176729/ICMSHhq0f+002940
Adding to the zip: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_12176729/ICMGGhq0f+002952
Adding to the zip: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_12176729/ICMUAhq0f+002952
Adding to the zip: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_12176729/ICMSHhq0f+002928
Zipping up the final file: /var/lib/boinc-client_40000/projects/climateprediction.net/oifs_43r3_ps_0085_1991050100_123_960_12176729_0_r826685277_122.zip
Uploading the final file: upload_file_122.zip
Uploading trickle at timestep: 10623600
23:22:00 (892): called boinc_finish(0)

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 May 2023 16:38:27 1538243 22261491 oifs_43r3_ps_0085_1991050100_123_960_12176729_0 10,623,600 80,681 0.0076
09 May 2023 16:38:27 1538243 22261491 oifs_43r3_ps_0085_1991050100_123_960_12176729_0 10,623,600 80,681 0.0076
09 May 2023 16:38:18 1538243 22261491 oifs_43r3_ps_0085_1991050100_123_960_12176729_0 9,504,000 71,756 0.0076
09 May 2023 16:38:18 1538243 22261491 oifs_43r3_ps_0085_1991050100_123_960_12176729_0 9,504,000 71,756 0.0076
09 May 2023 16:38:09 1538243 22261491 oifs_43r3_ps_0085_1991050100_123_960_12176729_0 8,640,000 65,173 0.0075
09 May 2023 16:37:59 1538243 22261491 oifs_43r3_ps_0085_1991050100_123_960_12176729_0 7,776,000 58,589 0.0075
09 May 2023 16:37:48 1538243 22261491 oifs_43r3_ps_0085_1991050100_123_960_12176729_0 6,912,000 52,005 0.0075
09 May 2023 16:37:38 1538243 22261491 oifs_43r3_ps_0085_1991050100_123_960_12176729_0 6,048,000 45,461 0.0075
09 May 2023 16:37:34 1538243 22261491 oifs_43r3_ps_0085_1991050100_123_960_12176729_0 5,184,000 38,959 0.0075
09 May 2023 16:37:34 1538243 22261491 oifs_43r3_ps_0085_1991050100_123_960_12176729_0 5,184,000 38,959 0.0075
09 May 2023 16:37:24 1538243 22261491 oifs_43r3_ps_0085_1991050100_123_960_12176729_0 4,320,000 32,556 0.0075
09 May 2023 16:37:14 1538243 22261491 oifs_43r3_ps_0085_1991050100_123_960_12176729_0 3,456,000 26,014 0.0075
09 May 2023 16:37:14 1538243 22261491 oifs_43r3_ps_0085_1991050100_123_960_12176729_0 3,456,000 26,014 0.0075
09 May 2023 16:37:03 1538243 22261491 oifs_43r3_ps_0085_1991050100_123_960_12176729_0 2,592,000 19,452 0.0075
09 May 2023 16:36:53 1538243 22261491 oifs_43r3_ps_0085_1991050100_123_960_12176729_0 1,728,000 12,931 0.0075
09 May 2023 16:36:48 1538243 22261491 oifs_43r3_ps_0085_1991050100_123_960_12176729_0 864,000 6,459 0.0075
09 May 2023 16:36:48 1538243 22261491 oifs_43r3_ps_0085_1991050100_123_960_12176729_0 864,000 6,459 0.0075


©2024 climateprediction.net