climateprediction.net home page
Task 22316307

Task 22316307

Name oifs_43r3_0152_2019110100_123_993_12214916_0
Workunit 12214916
Created 24 Feb 2023, 16:43:32 UTC
Sent 24 Feb 2023, 20:47:58 UTC
Report deadline 25 Apr 2023, 20:47:58 UTC
Received 25 Feb 2023, 12:08:35 UTC
Server state Over
Outcome Success
Client state Done
Exit status 0 (0x00000000)
Computer ID 1414915
Run time 13 hours 51 min 53 sec
CPU time 13 hours 49 min 56 sec
Validate state Valid
Credit 14,873.04
Device peak FLOPS 4.73 GFLOPS
Application version OpenIFS 43r3 v1.21
x86_64-pc-linux-gnu
Peak working set size 4,424.00 MB
Peak swap size 4,984.13 MB
Peak disk usage 1,263.80 MB
Stderr
<core_client_version>7.6.31</core_client_version>
<![CDATA[
<stderr_txt>
New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x810730 (old at 0x1ce8440)
[EC_DRHOOK:atlas:1:1:29304:29304] [20230225:101824:1677316704:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x810730 (old at 0x1ce8440)
[EC_DRHOOK:atlas:1:1:29304:29304] [20230225:101824:1677316704:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x810730 (old at (nil))
[EC_DRHOOK:atlas:1:1:29304:29304] [20230225:101824:1677316704:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x810730 (old at 0x1ce8440)
[EC_DRHOOK:atlas:1:1:29304:29304] [20230225:101824:1677316704:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x810730 (old at 0x1ce8440)
[EC_DRHOOK:atlas:1:1:29304:29304] [20230225:101824:1677316704:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x810730 (old at 0x1ce8440)
[EC_DRHOOK:atlas:1:1:29304:29304] [20230225:101824:1677316704:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x810730 (old at (nil))
[EC_DRHOOK:atlas:1:1:29304:29304] [20230225:101824:1677316704:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x810730 (old at (nil))
[EC_DRHOOK:atlas:1:1:29304:29304] [20230225:101824:1677316704:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x810730 (old at (nil))
[EC_DRHOOK:atlas:1:1:29304:29304] [20230225:101824:1677316704:0.000] [catch_signals@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  10:18:50 STEP 2568 H=2568:00 +CPU= 24.356
  10:19:04 STEP 2569 H=2569:00 +CPU= 13.628
  10:19:17 STEP 2570 H=2570:00 +CPU= 13.232
  10:19:38 STEP 2571 H=2571:00 +CPU= 21.116
  10:19:52 STEP 2572 H=2572:00 +CPU= 13.576
  10:20:05 STEP 2573 H=2573:00 +CPU= 12.852
  10:20:25 STEP 2574 H=2574:00 +CPU= 20.480
  10:20:38 STEP 2575 H=2575:00 +CPU= 12.700
  10:20:52 STEP 2576 H=2576:00 +CPU= 13.552
  10:21:12 STEP 2577 H=2577:00 +CPU= 20.104
  10:21:25 STEP 2578 H=2578:00 +CPU= 13.060
  10:21:38 STEP 2579 H=2579:00 +CPU= 12.796
  10:21:59 STEP 2580 H=2580:00 +CPU= 21.752
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhnw4+002580
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhnw4+002580
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhnw4+002580
  10:22:15 STEP 2581 H=2581:00 +CPU= 15.420
  10:22:29 STEP 2582 H=2582:00 +CPU= 14.092
  10:22:49 STEP 2583 H=2583:00 +CPU= 20.048
  10:23:03 STEP 2584 H=2584:00 +CPU= 14.232
  10:23:16 STEP 2585 H=2585:00 +CPU= 12.816
  10:23:36 STEP 2586 H=2586:00 +CPU= 20.296
  10:23:49 STEP 2587 H=2587:00 +CPU= 12.844
  10:24:04 STEP 2588 H=2588:00 +CPU= 14.912
  10:24:29 STEP 2589 H=2589:00 +CPU= 24.548
  10:24:44 STEP 2590 H=2590:00 +CPU= 15.648
  10:24:58 STEP 2591 H=2591:00 +CPU= 13.232
  10:25:24 STEP 2592 H=2592:00 +CPU= 25.884
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhnw4+002592
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhnw4+002592
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhnw4+002592
  10:25:36 STEP 2593 H=2593:00 +CPU= 12.824
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMGGhnw4+002580
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMSHhnw4+002580
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMUAhnw4+002580
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMGGhnw4+002592
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMSHhnw4+002592
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMUAhnw4+002592
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_0152_2019110100_123_993_12214916_0_r1735244742_107.zip
Uploading the intermediate file: upload_file_107.zip
  10:25:52 STEP 2594 H=2594:00 +CPU= 15.280
  10:26:12 STEP 2595 H=2595:00 +CPU= 20.084
  10:26:25 STEP 2596 H=2596:00 +CPU= 13.156
  10:26:38 STEP 2597 H=2597:00 +CPU= 12.824
  10:26:58 STEP 2598 H=2598:00 +CPU= 20.140
  10:27:13 STEP 2599 H=2599:00 +CPU= 14.968
  10:27:26 STEP 2600 H=2600:00 +CPU= 13.072
  10:27:46 STEP 2601 H=2601:00 +CPU= 19.976
  10:27:59 STEP 2602 H=2602:00 +CPU= 13.344
  10:28:13 STEP 2603 H=2603:00 +CPU= 13.576
  10:28:36 STEP 2604 H=2604:00 +CPU= 22.996
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhnw4+002604
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhnw4+002604
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhnw4+002604
  10:28:49 STEP 2605 H=2605:00 +CPU= 12.916
  10:29:02 STEP 2606 H=2606:00 +CPU= 13.232
  10:29:22 STEP 2607 H=2607:00 +CPU= 19.840
  10:29:37 STEP 2608 H=2608:00 +CPU= 14.972
  10:29:50 STEP 2609 H=2609:00 +CPU= 12.808
  10:30:11 STEP 2610 H=2610:00 +CPU= 21.496
  10:30:26 STEP 2611 H=2611:00 +CPU= 14.480
  10:30:40 STEP 2612 H=2612:00 +CPU= 14.656
  10:31:01 STEP 2613 H=2613:00 +CPU= 20.440
  10:31:14 STEP 2614 H=2614:00 +CPU= 13.060
  10:31:27 STEP 2615 H=2615:00 +CPU= 13.048
  10:31:51 STEP 2616 H=2616:00 +CPU= 24.288
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhnw4+002616
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhnw4+002616
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhnw4+002616
  10:32:05 STEP 2617 H=2617:00 +CPU= 13.904
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMGGhnw4+002604
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMSHhnw4+002604
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMUAhnw4+002604
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMGGhnw4+002616
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMSHhnw4+002616
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMUAhnw4+002616
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_0152_2019110100_123_993_12214916_0_r1735244742_108.zip
Uploading the intermediate file: upload_file_108.zip
  10:32:20 STEP 2618 H=2618:00 +CPU= 14.244
  10:32:39 STEP 2619 H=2619:00 +CPU= 19.872
  10:32:53 STEP 2620 H=2620:00 +CPU= 13.120
  10:33:07 STEP 2621 H=2621:00 +CPU= 14.320
  10:33:31 STEP 2622 H=2622:00 +CPU= 24.504
  10:33:47 STEP 2623 H=2623:00 +CPU= 15.152
  10:34:00 STEP 2624 H=2624:00 +CPU= 13.208
  10:34:20 STEP 2625 H=2625:00 +CPU= 20.188
  10:34:34 STEP 2626 H=2626:00 +CPU= 13.548
  10:34:47 STEP 2627 H=2627:00 +CPU= 13.228
  10:35:11 STEP 2628 H=2628:00 +CPU= 23.560
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhnw4+002628
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhnw4+002628
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhnw4+002628
  10:35:25 STEP 2629 H=2629:00 +CPU= 14.220
  10:35:38 STEP 2630 H=2630:00 +CPU= 13.112
  10:35:58 STEP 2631 H=2631:00 +CPU= 20.312
  10:36:14 STEP 2632 H=2632:00 +CPU= 15.184
  10:36:26 STEP 2633 H=2633:00 +CPU= 12.924
  10:36:47 STEP 2634 H=2634:00 +CPU= 20.132
  10:36:59 STEP 2635 H=2635:00 +CPU= 12.820
  10:37:13 STEP 2636 H=2636:00 +CPU= 13.536
  10:37:34 STEP 2637 H=2637:00 +CPU= 21.252
  10:37:47 STEP 2638 H=2638:00 +CPU= 13.204
  10:38:01 STEP 2639 H=2639:00 +CPU= 13.244
  10:38:24 STEP 2640 H=2640:00 +CPU= 23.156
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhnw4+002640
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhnw4+002640
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhnw4+002640
  10:38:37 STEP 2641 H=2641:00 +CPU= 13.036
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMGGhnw4+002628
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMSHhnw4+002628
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMUAhnw4+002628
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMGGhnw4+002640
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMSHhnw4+002640
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMUAhnw4+002640
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_0152_2019110100_123_993_12214916_0_r1735244742_109.zip
Uploading the intermediate file: upload_file_109.zip
  10:38:50 STEP 2642 H=2642:00 +CPU= 13.544
  10:39:10 STEP 2643 H=2643:00 +CPU= 19.720
  10:39:23 STEP 2644 H=2644:00 +CPU= 13.176
  10:39:36 STEP 2645 H=2645:00 +CPU= 12.976
  10:39:59 STEP 2646 H=2646:00 +CPU= 22.436
  10:40:13 STEP 2647 H=2647:00 +CPU= 13.940
  10:40:27 STEP 2648 H=2648:00 +CPU= 14.332
  10:40:46 STEP 2649 H=2649:00 +CPU= 19.296
  10:41:01 STEP 2650 H=2650:00 +CPU= 14.432
  10:41:16 STEP 2651 H=2651:00 +CPU= 15.464
  10:41:42 STEP 2652 H=2652:00 +CPU= 26.004
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhnw4+002652
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhnw4+002652
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhnw4+002652
  10:41:58 STEP 2653 H=2653:00 +CPU= 15.848
  10:42:13 STEP 2654 H=2654:00 +CPU= 14.832
  10:42:34 STEP 2655 H=2655:00 +CPU= 20.620
  10:42:47 STEP 2656 H=2656:00 +CPU= 13.384
  10:43:00 STEP 2657 H=2657:00 +CPU= 12.868
  10:43:20 STEP 2658 H=2658:00 +CPU= 20.344
  10:43:33 STEP 2659 H=2659:00 +CPU= 12.844
  10:43:47 STEP 2660 H=2660:00 +CPU= 13.608
  10:44:07 STEP 2661 H=2661:00 +CPU= 19.916
  10:44:21 STEP 2662 H=2662:00 +CPU= 14.280
  10:44:35 STEP 2663 H=2663:00 +CPU= 13.840
  10:44:58 STEP 2664 H=2664:00 +CPU= 23.008
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhnw4+002664
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhnw4+002664
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhnw4+002664
  10:45:11 STEP 2665 H=2665:00 +CPU= 13.464
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMGGhnw4+002652
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMSHhnw4+002652
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMUAhnw4+002652
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMGGhnw4+002664
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMSHhnw4+002664
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMUAhnw4+002664
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_0152_2019110100_123_993_12214916_0_r1735244742_110.zip
Uploading the intermediate file: upload_file_110.zip
  10:45:26 STEP 2666 H=2666:00 +CPU= 14.464
  10:45:48 STEP 2667 H=2667:00 +CPU= 21.952
  10:46:01 STEP 2668 H=2668:00 +CPU= 13.368
  10:46:14 STEP 2669 H=2669:00 +CPU= 12.840
  10:46:34 STEP 2670 H=2670:00 +CPU= 20.316
  10:46:48 STEP 2671 H=2671:00 +CPU= 13.508
  10:47:03 STEP 2672 H=2672:00 +CPU= 15.780
  10:47:28 STEP 2673 H=2673:00 +CPU= 24.648
  10:47:42 STEP 2674 H=2674:00 +CPU= 14.160
  10:47:55 STEP 2675 H=2675:00 +CPU= 12.764
  10:48:17 STEP 2676 H=2676:00 +CPU= 21.800
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhnw4+002676
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhnw4+002676
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhnw4+002676
  10:48:30 STEP 2677 H=2677:00 +CPU= 12.864
  10:48:43 STEP 2678 H=2678:00 +CPU= 13.744
  10:49:07 STEP 2679 H=2679:00 +CPU= 23.468
  10:49:21 STEP 2680 H=2680:00 +CPU= 13.724
  10:49:34 STEP 2681 H=2681:00 +CPU= 13.184
  10:49:54 STEP 2682 H=2682:00 +CPU= 20.384
  10:50:08 STEP 2683 H=2683:00 +CPU= 13.752
  10:50:25 STEP 2684 H=2684:00 +CPU= 16.432
  10:50:47 STEP 2685 H=2685:00 +CPU= 22.584
  10:51:00 STEP 2686 H=2686:00 +CPU= 13.168
  10:51:13 STEP 2687 H=2687:00 +CPU= 13.136
  10:51:37 STEP 2688 H=2688:00 +CPU= 23.212
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhnw4+002688
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhnw4+002688
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhnw4+002688
  10:51:51 STEP 2689 H=2689:00 +CPU= 14.620
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMGGhnw4+002676
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMSHhnw4+002676
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMUAhnw4+002676
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMGGhnw4+002688
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMSHhnw4+002688
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMUAhnw4+002688
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_0152_2019110100_123_993_12214916_0_r1735244742_111.zip
Uploading the intermediate file: upload_file_111.zip
  10:52:07 STEP 2690 H=2690:00 +CPU= 15.800
  10:52:30 STEP 2691 H=2691:00 +CPU= 22.336
  10:52:44 STEP 2692 H=2692:00 +CPU= 13.824
  10:52:56 STEP 2693 H=2693:00 +CPU= 12.636
  10:53:17 STEP 2694 H=2694:00 +CPU= 20.344
  10:53:29 STEP 2695 H=2695:00 +CPU= 12.844
  10:53:45 STEP 2696 H=2696:00 +CPU= 15.804
  10:54:05 STEP 2697 H=2697:00 +CPU= 20.072
  10:54:18 STEP 2698 H=2698:00 +CPU= 13.044
  10:54:31 STEP 2699 H=2699:00 +CPU= 12.748
  10:54:55 STEP 2700 H=2700:00 +CPU= 23.864
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhnw4+002700
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhnw4+002700
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhnw4+002700
  10:55:08 STEP 2701 H=2701:00 +CPU= 13.444
  10:55:23 STEP 2702 H=2702:00 +CPU= 14.652
  10:55:43 STEP 2703 H=2703:00 +CPU= 20.032
  10:55:58 STEP 2704 H=2704:00 +CPU= 13.880
  10:56:14 STEP 2705 H=2705:00 +CPU= 15.968
  10:56:36 STEP 2706 H=2706:00 +CPU= 22.272
  10:56:49 STEP 2707 H=2707:00 +CPU= 12.676
  10:57:02 STEP 2708 H=2708:00 +CPU= 13.112
  10:57:22 STEP 2709 H=2709:00 +CPU= 19.816
  10:57:36 STEP 2710 H=2710:00 +CPU= 13.628
  10:57:49 STEP 2711 H=2711:00 +CPU= 12.868
  10:58:12 STEP 2712 H=2712:00 +CPU= 23.388
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhnw4+002712
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhnw4+002712
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhnw4+002712
  10:58:25 STEP 2713 H=2713:00 +CPU= 12.936
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMGGhnw4+002700
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMSHhnw4+002700
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMUAhnw4+002700
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMGGhnw4+002712
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMSHhnw4+002712
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMUAhnw4+002712
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_0152_2019110100_123_993_12214916_0_r1735244742_112.zip
Uploading the intermediate file: upload_file_112.zip
  10:58:39 STEP 2714 H=2714:00 +CPU= 13.724
  10:58:59 STEP 2715 H=2715:00 +CPU= 20.212
  10:59:12 STEP 2716 H=2716:00 +CPU= 13.304
  10:59:25 STEP 2717 H=2717:00 +CPU= 12.872
  10:59:46 STEP 2718 H=2718:00 +CPU= 21.212
  11:00:02 STEP 2719 H=2719:00 +CPU= 15.604
  11:00:18 STEP 2720 H=2720:00 +CPU= 16.204
  11:00:42 STEP 2721 H=2721:00 +CPU= 24.100
  11:00:57 STEP 2722 H=2722:00 +CPU= 14.816
  11:01:10 STEP 2723 H=2723:00 +CPU= 12.844
  11:01:32 STEP 2724 H=2724:00 +CPU= 21.748
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhnw4+002724
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhnw4+002724
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhnw4+002724
  11:01:45 STEP 2725 H=2725:00 +CPU= 12.868
  11:01:59 STEP 2726 H=2726:00 +CPU= 13.996
  11:02:19 STEP 2727 H=2727:00 +CPU= 20.800
  11:02:33 STEP 2728 H=2728:00 +CPU= 13.340
  11:02:46 STEP 2729 H=2729:00 +CPU= 12.824
  11:03:07 STEP 2730 H=2730:00 +CPU= 21.808
  11:03:21 STEP 2731 H=2731:00 +CPU= 13.864
  11:03:35 STEP 2732 H=2732:00 +CPU= 13.328
  11:03:54 STEP 2733 H=2733:00 +CPU= 19.816
  11:04:08 STEP 2734 H=2734:00 +CPU= 13.892
  11:04:21 STEP 2735 H=2735:00 +CPU= 13.108
  11:04:46 STEP 2736 H=2736:00 +CPU= 24.448
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhnw4+002736
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhnw4+002736
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhnw4+002736
  11:04:59 STEP 2737 H=2737:00 +CPU= 12.856
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMGGhnw4+002724
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMSHhnw4+002724
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMUAhnw4+002724
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMGGhnw4+002736
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMSHhnw4+002736
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMUAhnw4+002736
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_0152_2019110100_123_993_12214916_0_r1735244742_113.zip
Uploading the intermediate file: upload_file_113.zip
  11:05:14 STEP 2738 H=2738:00 +CPU= 15.284
  11:05:35 STEP 2739 H=2739:00 +CPU= 20.644
  11:05:49 STEP 2740 H=2740:00 +CPU= 14.244
  11:06:03 STEP 2741 H=2741:00 +CPU= 14.000
  11:06:26 STEP 2742 H=2742:00 +CPU= 23.364
  11:06:40 STEP 2743 H=2743:00 +CPU= 13.676
  11:06:53 STEP 2744 H=2744:00 +CPU= 13.212
  11:07:13 STEP 2745 H=2745:00 +CPU= 19.740
  11:07:26 STEP 2746 H=2746:00 +CPU= 13.060
  11:07:40 STEP 2747 H=2747:00 +CPU= 14.148
  11:08:03 STEP 2748 H=2748:00 +CPU= 22.472
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhnw4+002748
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhnw4+002748
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhnw4+002748
  11:08:15 STEP 2749 H=2749:00 +CPU= 12.780
  11:08:29 STEP 2750 H=2750:00 +CPU= 13.108
  11:08:48 STEP 2751 H=2751:00 +CPU= 19.604
  11:09:01 STEP 2752 H=2752:00 +CPU= 13.320
  11:09:17 STEP 2753 H=2753:00 +CPU= 15.608
  11:09:41 STEP 2754 H=2754:00 +CPU= 23.988
  11:09:55 STEP 2755 H=2755:00 +CPU= 14.320
  11:10:12 STEP 2756 H=2756:00 +CPU= 16.104
  11:10:36 STEP 2757 H=2757:00 +CPU= 24.508
  11:10:51 STEP 2758 H=2758:00 +CPU= 14.428
  11:11:04 STEP 2759 H=2759:00 +CPU= 13.756
  11:11:28 STEP 2760 H=2760:00 +CPU= 23.600
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhnw4+002760
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhnw4+002760
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhnw4+002760
  11:11:41 STEP 2761 H=2761:00 +CPU= 12.732
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMGGhnw4+002748
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMSHhnw4+002748
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMUAhnw4+002748
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMGGhnw4+002760
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMSHhnw4+002760
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMUAhnw4+002760
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_0152_2019110100_123_993_12214916_0_r1735244742_114.zip
Uploading the intermediate file: upload_file_114.zip
  11:11:57 STEP 2762 H=2762:00 +CPU= 15.772
  11:12:17 STEP 2763 H=2763:00 +CPU= 20.924
  11:12:30 STEP 2764 H=2764:00 +CPU= 12.964
  11:12:43 STEP 2765 H=2765:00 +CPU= 12.760
  11:13:04 STEP 2766 H=2766:00 +CPU= 20.420
  11:13:18 STEP 2767 H=2767:00 +CPU= 14.524
  11:13:32 STEP 2768 H=2768:00 +CPU= 13.784
  11:13:52 STEP 2769 H=2769:00 +CPU= 20.168
  11:14:05 STEP 2770 H=2770:00 +CPU= 12.980
  11:14:18 STEP 2771 H=2771:00 +CPU= 12.776
  11:14:42 STEP 2772 H=2772:00 +CPU= 23.644
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhnw4+002772
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhnw4+002772
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhnw4+002772
  11:14:54 STEP 2773 H=2773:00 +CPU= 12.696
  11:15:08 STEP 2774 H=2774:00 +CPU= 14.088
  11:15:30 STEP 2775 H=2775:00 +CPU= 21.808
  11:15:43 STEP 2776 H=2776:00 +CPU= 13.220
  11:15:59 STEP 2777 H=2777:00 +CPU= 15.656
  11:16:20 STEP 2778 H=2778:00 +CPU= 20.564
  11:16:32 STEP 2779 H=2779:00 +CPU= 12.784
  11:16:46 STEP 2780 H=2780:00 +CPU= 13.328
  11:17:07 STEP 2781 H=2781:00 +CPU= 21.156
  11:17:22 STEP 2782 H=2782:00 +CPU= 14.992
  11:17:35 STEP 2783 H=2783:00 +CPU= 13.280
  11:17:58 STEP 2784 H=2784:00 +CPU= 23.296
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhnw4+002784
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhnw4+002784
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhnw4+002784
  11:18:11 STEP 2785 H=2785:00 +CPU= 12.828
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMGGhnw4+002772
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMSHhnw4+002772
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMUAhnw4+002772
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMGGhnw4+002784
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMSHhnw4+002784
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMUAhnw4+002784
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_0152_2019110100_123_993_12214916_0_r1735244742_115.zip
Uploading the intermediate file: upload_file_115.zip
  11:18:26 STEP 2786 H=2786:00 +CPU= 14.844
  11:18:46 STEP 2787 H=2787:00 +CPU= 19.860
  11:18:59 STEP 2788 H=2788:00 +CPU= 13.120
  11:19:12 STEP 2789 H=2789:00 +CPU= 12.956
  11:19:34 STEP 2790 H=2790:00 +CPU= 21.652
  11:19:47 STEP 2791 H=2791:00 +CPU= 13.196
  11:20:00 STEP 2792 H=2792:00 +CPU= 13.156
  11:20:22 STEP 2793 H=2793:00 +CPU= 22.392
  11:20:36 STEP 2794 H=2794:00 +CPU= 13.096
  11:20:49 STEP 2795 H=2795:00 +CPU= 12.924
  11:21:13 STEP 2796 H=2796:00 +CPU= 24.948
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhnw4+002796
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhnw4+002796
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhnw4+002796
  11:21:28 STEP 2797 H=2797:00 +CPU= 14.072
  11:21:43 STEP 2798 H=2798:00 +CPU= 15.744
  11:22:06 STEP 2799 H=2799:00 +CPU= 22.180
  11:22:19 STEP 2800 H=2800:00 +CPU= 13.136
  11:22:32 STEP 2801 H=2801:00 +CPU= 13.260
  11:22:52 STEP 2802 H=2802:00 +CPU= 20.384
  11:23:07 STEP 2803 H=2803:00 +CPU= 14.336
  11:23:21 STEP 2804 H=2804:00 +CPU= 14.732
  11:23:41 STEP 2805 H=2805:00 +CPU= 19.960
  11:23:54 STEP 2806 H=2806:00 +CPU= 13.024
  11:24:08 STEP 2807 H=2807:00 +CPU= 13.664
Quit request received from BOINC client, ending the child process
>>> Printing last 70 lines from file: NODE.001_01

         AVE   0.651225292364986E-06 0.000000000000000E+00 0.296573509571777E-03
 GPNORM  CLOUD FRACTION      AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.638057862277699E-01 0.000000000000000E+00 0.100000000000000E+01
 GPNORM  OZONE               AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.309570850952564E-05 0.253222825558104E-07 0.163314801768092E-04
 GPNORM TENUNOGW             AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.533783756128957E-07 -.244042026263194E-02 0.582198277998125E-03
 GPNORM TENVNOGW             AVERAGE               MINIMUM               MAXIMUM        
         AVE   -.850425119489500E-09 -.419769762456842E-03 0.100280896844845E-02
 GPNORM CLW                  AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.365394529875688E-05 0.000000000000000E+00 0.133259251037359E-02
 GPNORM CIW                  AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.136698934812306E-05 0.000000000000000E+00 0.743320953413669E-03
 GPNORM CC                   AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.636365695913811E-01 0.000000000000000E+00 0.100000000000000E+01
 GPNORM RFL                  AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.268212711281384E-05 0.000000000000000E+00 0.112405296308405E-02
 GPNORM SFL                  AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.221209744319209E-05 0.000000000000000E+00 0.116261006630749E-02
 GPNORM CRFL                 AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.461390799503408E-05 0.000000000000000E+00 0.239538619907498E-02
 GPNORM CSFL                 AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.144313395251614E-05 0.000000000000000E+00 0.162904313986215E-02
 GPNORM PFRC                 AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.136884638321915E+00 0.000000000000000E+00 0.100000000000000E+01
 NSTEP =  2808 STEPO   F00000000
 CREATE RESTART FILE ./srf01170000.0001                                                                                                                                                                                                                                              
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ./srf01170000.0001 MODE=w
 IO-STREAM CLOSED - ./srf01170000.0001
 MPI-TASK:   1 - ************ BYTES IN     35 RECORDS TRANSFERRED IN    1.0021 SECONDS******** Mbytes/s, TOTAL TIME=    1.3557(73.9%)
 CREATE RESTART CONTROL FILE ./rcf                                                                                                                   
 WRRESF: DELETED PREVIOUS RESTART FILE OK: ./srf01160000.0001
  MAXGPFV : MAX. VALUE =    1.22194413814145     
  MAXGPFV : MAX. VALUE =   0.737391498119973     
 NSTEP =  2808 SCAN2M_HPOS  P
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ICMGGhnw4+002808 MODE=w
 IO-STREAM CLOSED - ICMGGhnw4+002808
 MPI-TASK:   1 -   1340532895 BYTES IN     27 RECORDS TRANSFERRED IN    0.0007 SECONDS******** Mbytes/s, TOTAL TIME=    0.0198( 3.5%)
 NSTEP =  2808 STEPO   0AA000000
  SPECTRAL NORMS: Pattern  1
 STEPO, YSPSDT_AR1     (  1,...), SP NORMS 2D: 5.203718527471E-01 1.563028090220E-01 6.150272408863E-02
 STEPO, YSPSDT_AR1     (SUM,...), SP NORMS 2D: 5.424520217285E-01
 STEPO, YGPSDT         (  1,...), Grid Point Norms
  FIELD LEV        AVERAGE               MINIMUM               MAXIMUM       
    101  60  1.57000219511253E-03 -1.76937499553706E+00  2.00321341122186E+00
 STEPO, YGPSDT[clipped](  1,...), Grid Point Norms
  FIELD LEV        AVERAGE               MINIMUM               MAXIMUM       
    101  60  6.46883123814271E-04 -1.00000000000000E+00  1.00000000000000E+00
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ./ICMSHhnw4+002808 MODE=a
 IO-STREAM CLOSED - ./ICMSHhnw4+002808
 MPI-TASK:   1 -   7945363124 BYTES IN    115 RECORDS TRANSFERRED IN    0.0028 SECONDS******** Mbytes/s, TOTAL TIME=    0.0872( 3.2%)
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ./ICMUAhnw4+002808 MODE=a
 IO-STREAM CLOSED - ./ICMUAhnw4+002808
 MPI-TASK:   1 -   1005399669 BYTES IN     19 RECORDS TRANSFERRED IN    0.0006 SECONDS******** Mbytes/s, TOTAL TIME=    0.0142( 4.2%)
 STEPO_FPOS WAS : B00000F00E
 NSTEP =  2808 STEPO   0AAA00AAA
  SPECTRAL NORMS: Pattern  1
 STEPO, YSPSDT_AR1     (  1,...), SP NORMS 2D: 5.203718527471E-01 1.563028090220E-01 6.150272408863E-02
 STEPO, YSPSDT_AR1     (SUM,...), SP NORMS 2D: 5.424520217285E-01
 STEPO, YGPSDT         (  1,...), Grid Point Norms
  FIELD LEV        AVERAGE               MINIMUM               MAXIMUM       
    101  60  1.57000219511253E-03 -1.76937499553706E+00  2.00321341122186E+00
 STEPO, YGPSDT[clipped](  1,...), Grid Point Norms
  FIELD LEV        AVERAGE               MINIMUM               MAXIMUM       
    101  60  6.46883123814271E-04 -1.00000000000000E+00  1.00000000000000E+00
------------------------------------------------
CNT0 not found; string returned was: 'STEPO'
>>> Printing last 8 lines from file: ifs.stat

 11:23:21 0AAA00AAA STEPO     2805    14.732   14.732   14.738     64:53     64:57 0.11581761816882E-04       2GB       0MB
 11:23:41 0AAA00AAA STEPO     2806    19.960   19.960   19.959     65:13     65:17 0.11581761816882E-04       2GB       0MB
 11:23:54 0AAA00AAA STEPO     2807    13.024   13.024   13.023     65:26     65:30 0.11581761816882E-04       2GB       0MB
 11:24:08 F00000000 STEPO     2808    13.804   13.804   13.806     65:39     65:44 0.11841316136459E-04       2GB       0MB
 11:24:10 0AA000000 STEPO     2808     1.592    1.592    1.592     65:41     65:45 0.11841316136459E-04       2GB       0MB
 11:24:10 FULLPOS-B DYNFPOS   2808     0.516    0.516    0.518     65:42     65:46 0.11841316136459E-04       2GB       0MB
 11:24:11 0AAA00AAA STEPO     2808     0.732    0.732    0.729     65:42     65:47 0.11841316136459E-04       2GB       0MB
------------------------------------------------
>>> Printing last 11 lines from file: rcf

 CSTEP   = "    2808",
 CTIME   = "01170000      ",
 NSTEPLPP        =         201,         202,          49,      228226,      228227, 5*0,
 IPRGPNSRES      =           1,
 IPRGPEWRES      =           1,
 IPRTRWRES       =           1,
 IPRTRVRES       =           1,
 GMASS0  =   98321.0380868182     ,
 GMASSI  =   98320.2982703738     
 /
------------------------------------------------
>>> Printing last 17 lines from file: waminfo

PARAM=10U/10V,                                                          
REPRES=GG,                                                              
ANALYSIS PERIOD =       0                                               
FORECAST PERIOD =    9763200                                            
WIND TIME STEP IN SECONDS =    3600                                     
ANALYSIS FROM 20191101000000 TO 20191101000000                          
ASSIMILATION NO                                                         
NEW FORECAST NO                                                         
STATUS STORM1= UNFINISHED                                               
STATUS STORM2= UNFINISHED                                               
STATUS STORM3= UNFINISHED                                               
DATE FOR OUTPUT OF BOTH RESTART FILES = 00000000000000                  
LAST DATE FOR SPECTRA FILE OUTPUT = 20200303000000                      
BEGIN DATE FOR USING SURFACE CURRENT = 20191101000000                   
PARALLEL RESTART = YES                                                  
NUMBER OF MPI TASKS USED =          1                                   
------------------------------------------------
>>> Printing last 8 lines from file: /var/lib/boinc-client/slots/5/progress_file_12214916.xml

<running_values>
  <last_cpu_time>47416.000000</last_cpu_time>
  <upload_file_number>116</upload_file_number>
  <last_iter>2808</last_iter>
  <last_upload>10026000</last_upload>
  <model_completed>0</model_completed>
</running_values>
------------------------------------------------
..Failed, model did not complete successfully
(argv0) ../../projects/climateprediction.net/oifs_43r3_1.21_x86_64-pc-linux-gnu
(argv1) start_date: 2019110100
(argv2) exptid: hnw4
(argv3) unique_member_id: 0152
(argv4) batchid: 993
(argv5) wuid: 12214916
(argv6) fclen: 123
(argv7) app_name: oifs_43r3
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/5
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3
version: 1.21
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_app_1.21_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/5/oifs_43r3_app_1.21_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/5/oifs_43r3_app_1.21_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_97c12f576281151a868442c1875f5e03 to: /var/lib/boinc-client/slots/5/oifs_43r3_0152_2019110100_123_993_12214916.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/5/oifs_43r3_0152_2019110100_123_993_12214916.zip
ic_ancil_file: ic_ancil_12214916
ifsdata_file: ifsdata_12214916
climate_data_file: clim_data_12214916
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_5a730c103f895dc577315b995aaa18f6 to: /var/lib/boinc-client/slots/5/ic_ancil_12214916.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/5/ic_ancil_12214916.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_11bd6f4fcae1fc7053a9475300fe7505 to: /var/lib/boinc-client/slots/5/ifsdata/ifsdata_12214916.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/5/ifsdata/ifsdata_12214916.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/5/159l_2/clim_data_12214916.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/5/159l_2/clim_data_12214916.zip
Checking for progress XML file: /var/lib/boinc-client/slots/5/progress_file_12214916.xml
Opened progress file ok : /var/lib/boinc-client/slots/5/progress_file_12214916.xml
-- Model is restarting --
Adjusting last_iter, 2808, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/5/progress_file_12214916.xml
last_cpu_time: 47416
upload_file_number: 116
last_iter: 2809
last_upload: 10026000
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_0152_2019110100_123_993_12214916_0_r1735244742
The child process has been launched with process id: 10043
Executing the command: /var/lib/boinc-client/slots/5/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:atlas:1:1:10043:10043] [20230225:112552:1677320752:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2091] fp = 0x3124020
[EC_DRHOOK:atlas:1:1:10043:10043] [20230225:112552:1677320752:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:atlas:1:1:10043:10043] [20230225:112552:1677320752:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:atlas:1:1:10043:10043] [20230225:112552:1677320752:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:atlas:1:1:10043:10043] [20230225:112552:1677320752:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:atlas:1:1:10043:10043] [20230225:112552:1677320752:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:atlas:1:1:10043:10043] [20230225:112552:1677320752:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:atlas:1:1:10043:10043] [20230225:112552:1677320752:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:atlas:1:1:10043:10043] [20230225:112552:1677320752:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:atlas:1:1:10043:10043] [20230225:112552:1677320752:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:atlas:1:1:10043:10043] [20230225:112552:1677320752:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:atlas:1:1:10043:10043] [20230225:112552:1677320752:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:atlas:1:1:10043:10043] [20230225:112552:1677320752:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:atlas:1:1:10043:10043] [20230225:112552:1677320752:0.000] [ignore_signals@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:atlas:1:1:10043:10043] [20230225:112552:1677320752:0.000] [restore_default_signals@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:atlas:1:1:10043:10043] [20230225:112552:1677320752:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x810730 (old at 0x1ce8440)
[EC_DRHOOK:atlas:1:1:10043:10043] [20230225:112552:1677320752:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x810730 (old at (nil))
[EC_DRHOOK:atlas:1:1:10043:10043] [20230225:112552:1677320752:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x810730 (old at 0x1ce8440)
[EC_DRHOOK:atlas:1:1:10043:10043] [20230225:112552:1677320752:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x810730 (old at (nil))
[EC_DRHOOK:atlas:1:1:10043:10043] [20230225:112552:1677320752:0.000] [trapfpe_treatment@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:atlas:1:1:10043:10043] [20230225:112552:1677320752:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x810730 (old at 0x1ce8440)
[EC_DRHOOK:atlas:1:1:10043:10043] [20230225:112552:1677320752:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x810730 (old at 0x1ce8440)
[EC_DRHOOK:atlas:1:1:10043:10043] [20230225:112552:1677320752:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x810730 (old at (nil))
[EC_DRHOOK:atlas:1:1:10043:10043] [20230225:112552:1677320752:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x810730 (old at 0x1ce8440)
[EC_DRHOOK:atlas:1:1:10043:10043] [20230225:112552:1677320752:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x810730 (old at 0x1ce8440)
[EC_DRHOOK:atlas:1:1:10043:10043] [20230225:112552:1677320752:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x810730 (old at 0x1ce8440)
[EC_DRHOOK:atlas:1:1:10043:10043] [20230225:112552:1677320752:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x810730 (old at (nil))
[EC_DRHOOK:atlas:1:1:10043:10043] [20230225:112552:1677320752:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x810730 (old at (nil))
[EC_DRHOOK:atlas:1:1:10043:10043] [20230225:112552:1677320752:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x810730 (old at (nil))
[EC_DRHOOK:atlas:1:1:10043:10043] [20230225:112552:1677320752:0.000] [catch_signals@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMGGhnw4+002796
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMSHhnw4+002796
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMUAhnw4+002796
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_0152_2019110100_123_993_12214916_0_r1735244742_116.zip
Uploading the intermediate file: upload_file_116.zip
  11:26:17 STEP 2808 H=2808:00 +CPU= 23.532
  11:26:30 STEP 2809 H=2809:00 +CPU= 12.492
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhnw4+002808
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhnw4+002808
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhnw4+002808
  11:26:43 STEP 2810 H=2810:00 +CPU= 13.260
  11:27:04 STEP 2811 H=2811:00 +CPU= 21.044
  11:27:19 STEP 2812 H=2812:00 +CPU= 14.700
  11:27:31 STEP 2813 H=2813:00 +CPU= 12.872
  11:27:52 STEP 2814 H=2814:00 +CPU= 20.620
  11:28:05 STEP 2815 H=2815:00 +CPU= 12.644
  11:28:19 STEP 2816 H=2816:00 +CPU= 14.396
  11:28:41 STEP 2817 H=2817:00 +CPU= 22.164
  11:28:55 STEP 2818 H=2818:00 +CPU= 13.436
  11:29:08 STEP 2819 H=2819:00 +CPU= 12.828
  11:29:29 STEP 2820 H=2820:00 +CPU= 21.672
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhnw4+002820
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhnw4+002820
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhnw4+002820
  11:29:42 STEP 2821 H=2821:00 +CPU= 12.592
  11:29:57 STEP 2822 H=2822:00 +CPU= 15.472
  11:30:20 STEP 2823 H=2823:00 +CPU= 22.496
  11:30:34 STEP 2824 H=2824:00 +CPU= 13.704
  11:30:47 STEP 2825 H=2825:00 +CPU= 12.972
  11:31:08 STEP 2826 H=2826:00 +CPU= 21.372
  11:31:21 STEP 2827 H=2827:00 +CPU= 12.560
  11:31:34 STEP 2828 H=2828:00 +CPU= 13.144
  11:31:54 STEP 2829 H=2829:00 +CPU= 20.096
  11:32:07 STEP 2830 H=2830:00 +CPU= 13.240
  11:32:23 STEP 2831 H=2831:00 +CPU= 15.304
  11:32:46 STEP 2832 H=2832:00 +CPU= 23.320
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhnw4+002832
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhnw4+002832
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhnw4+002832
  11:32:59 STEP 2833 H=2833:00 +CPU= 12.904
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMGGhnw4+002820
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMSHhnw4+002820
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMUAhnw4+002820
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMGGhnw4+002832
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMSHhnw4+002832
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMUAhnw4+002832
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_0152_2019110100_123_993_12214916_0_r1735244742_117.zip
Uploading the intermediate file: upload_file_117.zip
  11:33:12 STEP 2834 H=2834:00 +CPU= 13.220
  11:33:33 STEP 2835 H=2835:00 +CPU= 21.184
  11:33:47 STEP 2836 H=2836:00 +CPU= 13.192
  11:34:00 STEP 2837 H=2837:00 +CPU= 13.132
  11:34:20 STEP 2838 H=2838:00 +CPU= 20.424
  11:34:33 STEP 2839 H=2839:00 +CPU= 13.220
  11:34:48 STEP 2840 H=2840:00 +CPU= 14.176
  11:35:08 STEP 2841 H=2841:00 +CPU= 20.648
  11:35:22 STEP 2842 H=2842:00 +CPU= 14.080
  11:35:35 STEP 2843 H=2843:00 +CPU= 12.744
  11:35:57 STEP 2844 H=2844:00 +CPU= 21.924
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhnw4+002844
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhnw4+002844
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhnw4+002844
  11:36:10 STEP 2845 H=2845:00 +CPU= 13.316
  11:36:25 STEP 2846 H=2846:00 +CPU= 14.536
  11:36:47 STEP 2847 H=2847:00 +CPU= 22.444
  11:37:03 STEP 2848 H=2848:00 +CPU= 15.492
  11:37:19 STEP 2849 H=2849:00 +CPU= 16.040
  11:37:40 STEP 2850 H=2850:00 +CPU= 20.608
  11:37:52 STEP 2851 H=2851:00 +CPU= 12.460
  11:38:05 STEP 2852 H=2852:00 +CPU= 13.164
  11:38:27 STEP 2853 H=2853:00 +CPU= 22.260
  11:38:40 STEP 2854 H=2854:00 +CPU= 12.908
  11:38:54 STEP 2855 H=2855:00 +CPU= 13.320
  11:39:17 STEP 2856 H=2856:00 +CPU= 23.468
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhnw4+002856
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhnw4+002856
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhnw4+002856
  11:39:31 STEP 2857 H=2857:00 +CPU= 14.240
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMGGhnw4+002844
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMSHhnw4+002844
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMUAhnw4+002844
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMGGhnw4+002856
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMSHhnw4+002856
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMUAhnw4+002856
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_0152_2019110100_123_993_12214916_0_r1735244742_118.zip
Uploading the intermediate file: upload_file_118.zip
  11:39:47 STEP 2858 H=2858:00 +CPU= 15.712
  11:40:08 STEP 2859 H=2859:00 +CPU= 20.776
  11:40:22 STEP 2860 H=2860:00 +CPU= 14.364
  11:40:35 STEP 2861 H=2861:00 +CPU= 12.776
  11:40:57 STEP 2862 H=2862:00 +CPU= 22.352
  11:41:10 STEP 2863 H=2863:00 +CPU= 12.836
  11:41:23 STEP 2864 H=2864:00 +CPU= 13.232
  11:41:43 STEP 2865 H=2865:00 +CPU= 19.976
  11:41:57 STEP 2866 H=2866:00 +CPU= 14.000
  11:42:11 STEP 2867 H=2867:00 +CPU= 13.636
  11:42:33 STEP 2868 H=2868:00 +CPU= 21.816
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhnw4+002868
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhnw4+002868
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhnw4+002868
  11:42:46 STEP 2869 H=2869:00 +CPU= 12.868
  11:42:59 STEP 2870 H=2870:00 +CPU= 13.416
  11:43:22 STEP 2871 H=2871:00 +CPU= 22.400
  11:43:35 STEP 2872 H=2872:00 +CPU= 13.056
  11:43:48 STEP 2873 H=2873:00 +CPU= 13.044
  11:44:08 STEP 2874 H=2874:00 +CPU= 20.652
  11:44:21 STEP 2875 H=2875:00 +CPU= 12.676
  11:44:35 STEP 2876 H=2876:00 +CPU= 14.040
  11:44:55 STEP 2877 H=2877:00 +CPU= 20.096
  11:45:08 STEP 2878 H=2878:00 +CPU= 13.216
  11:45:22 STEP 2879 H=2879:00 +CPU= 13.536
  11:45:46 STEP 2880 H=2880:00 +CPU= 24.412
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhnw4+002880
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhnw4+002880
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhnw4+002880
  11:46:01 STEP 2881 H=2881:00 +CPU= 14.532
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMGGhnw4+002868
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMSHhnw4+002868
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMUAhnw4+002868
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMGGhnw4+002880
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMSHhnw4+002880
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMUAhnw4+002880
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_0152_2019110100_123_993_12214916_0_r1735244742_119.zip
Uploading the intermediate file: upload_file_119.zip
  11:46:17 STEP 2882 H=2882:00 +CPU= 15.948
  11:46:40 STEP 2883 H=2883:00 +CPU= 23.508
  11:46:54 STEP 2884 H=2884:00 +CPU= 13.956
  11:47:08 STEP 2885 H=2885:00 +CPU= 13.872
  11:47:29 STEP 2886 H=2886:00 +CPU= 20.240
  11:47:41 STEP 2887 H=2887:00 +CPU= 12.636
  11:47:55 STEP 2888 H=2888:00 +CPU= 13.332
  11:48:16 STEP 2889 H=2889:00 +CPU= 21.496
  11:48:29 STEP 2890 H=2890:00 +CPU= 13.352
  11:48:42 STEP 2891 H=2891:00 +CPU= 12.924
  11:49:04 STEP 2892 H=2892:00 +CPU= 22.076
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhnw4+002892
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhnw4+002892
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhnw4+002892
  11:49:18 STEP 2893 H=2893:00 +CPU= 13.700
  11:49:32 STEP 2894 H=2894:00 +CPU= 14.096
  11:49:52 STEP 2895 H=2895:00 +CPU= 19.876
  11:50:06 STEP 2896 H=2896:00 +CPU= 13.704
  11:50:20 STEP 2897 H=2897:00 +CPU= 13.824
  11:50:41 STEP 2898 H=2898:00 +CPU= 21.456
  11:50:54 STEP 2899 H=2899:00 +CPU= 13.128
  11:51:08 STEP 2900 H=2900:00 +CPU= 14.020
  11:51:29 STEP 2901 H=2901:00 +CPU= 20.308
  11:51:41 STEP 2902 H=2902:00 +CPU= 12.888
  11:51:55 STEP 2903 H=2903:00 +CPU= 13.164
  11:52:19 STEP 2904 H=2904:00 +CPU= 24.680
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhnw4+002904
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhnw4+002904
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhnw4+002904
  11:52:33 STEP 2905 H=2905:00 +CPU= 13.460
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMGGhnw4+002892
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMSHhnw4+002892
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMUAhnw4+002892
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMGGhnw4+002904
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMSHhnw4+002904
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMUAhnw4+002904
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_0152_2019110100_123_993_12214916_0_r1735244742_120.zip
Uploading the intermediate file: upload_file_120.zip
  11:52:46 STEP 2906 H=2906:00 +CPU= 13.560
  11:53:07 STEP 2907 H=2907:00 +CPU= 20.316
  11:53:20 STEP 2908 H=2908:00 +CPU= 13.036
  11:53:33 STEP 2909 H=2909:00 +CPU= 13.236
  11:53:55 STEP 2910 H=2910:00 +CPU= 21.868
  11:54:08 STEP 2911 H=2911:00 +CPU= 12.844
  11:54:21 STEP 2912 H=2912:00 +CPU= 13.212
  11:54:41 STEP 2913 H=2913:00 +CPU= 20.024
  11:54:54 STEP 2914 H=2914:00 +CPU= 13.356
  11:55:09 STEP 2915 H=2915:00 +CPU= 14.452
  11:55:32 STEP 2916 H=2916:00 +CPU= 23.244
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhnw4+002916
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhnw4+002916
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhnw4+002916
  11:55:45 STEP 2917 H=2917:00 +CPU= 12.916
  11:55:58 STEP 2918 H=2918:00 +CPU= 13.176
  11:56:19 STEP 2919 H=2919:00 +CPU= 20.924
  11:56:32 STEP 2920 H=2920:00 +CPU= 12.988
  11:56:45 STEP 2921 H=2921:00 +CPU= 12.892
  11:57:05 STEP 2922 H=2922:00 +CPU= 20.560
  11:57:20 STEP 2923 H=2923:00 +CPU= 14.412
  11:57:34 STEP 2924 H=2924:00 +CPU= 14.240
  11:57:57 STEP 2925 H=2925:00 +CPU= 22.448
  11:58:10 STEP 2926 H=2926:00 +CPU= 13.512
  11:58:23 STEP 2927 H=2927:00 +CPU= 13.352
  11:58:47 STEP 2928 H=2928:00 +CPU= 23.288
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhnw4+002928
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhnw4+002928
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhnw4+002928
  11:59:00 STEP 2929 H=2929:00 +CPU= 13.200
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMGGhnw4+002916
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMSHhnw4+002916
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMUAhnw4+002916
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMGGhnw4+002928
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMSHhnw4+002928
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMUAhnw4+002928
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_0152_2019110100_123_993_12214916_0_r1735244742_121.zip
Uploading the intermediate file: upload_file_121.zip
  11:59:15 STEP 2930 H=2930:00 +CPU= 15.324
  11:59:36 STEP 2931 H=2931:00 +CPU= 20.308
  11:59:49 STEP 2932 H=2932:00 +CPU= 12.980
  12:00:01 STEP 2933 H=2933:00 +CPU= 12.604
  12:00:24 STEP 2934 H=2934:00 +CPU= 23.040
  12:00:39 STEP 2935 H=2935:00 +CPU= 14.556
  12:00:53 STEP 2936 H=2936:00 +CPU= 13.552
  12:01:15 STEP 2937 H=2937:00 +CPU= 22.340
  12:01:30 STEP 2938 H=2938:00 +CPU= 14.944
  12:01:46 STEP 2939 H=2939:00 +CPU= 15.960
  12:02:11 STEP 2940 H=2940:00 +CPU= 25.416
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhnw4+002940
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhnw4+002940
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhnw4+002940
  12:02:27 STEP 2941 H=2941:00 +CPU= 15.856
  12:02:43 STEP 2942 H=2942:00 +CPU= 15.484
  12:03:06 STEP 2943 H=2943:00 +CPU= 23.512
  12:03:20 STEP 2944 H=2944:00 +CPU= 13.264
  12:03:33 STEP 2945 H=2945:00 +CPU= 13.108
  12:03:53 STEP 2946 H=2946:00 +CPU= 20.320
  12:04:06 STEP 2947 H=2947:00 +CPU= 13.212
  12:04:21 STEP 2948 H=2948:00 +CPU= 14.656
  12:04:41 STEP 2949 H=2949:00 +CPU= 20.216
  12:04:54 STEP 2950 H=2950:00 +CPU= 13.128
  12:05:08 STEP 2951 H=2951:00 +CPU= 14.104
  12:05:34 STEP 2952 H=2952:00 +CPU= 25.592
..The child process terminated with status: 0
>>> Printing last 70 lines from file: NODE.001_01

 STEPO, YSPSDT_AR1     (  1,...), SP NORMS 2D: 4.865283731608E-01 1.865286143845E-01 6.362598939121E-02
 STEPO, YSPSDT_AR1     (SUM,...), SP NORMS 2D: 5.260318722649E-01
 STEPO, YGPSDT         (  1,...), Grid Point Norms
  FIELD LEV        AVERAGE               MINIMUM               MAXIMUM       
    101  60  3.48989191816039E-02 -1.72994898449055E+00  1.89761310057317E+00
 STEPO, YGPSDT[clipped](  1,...), Grid Point Norms
  FIELD LEV        AVERAGE               MINIMUM               MAXIMUM       
    101  60  3.19925486346338E-02 -1.00000000000000E+00  1.00000000000000E+00
  MAXGPFV : MAX. VALUE =    1.16657433498722     
  MAXGPFV : MAX. VALUE =   0.738901897780111     
 NSTEP =  2952 SCAN2M_HPOS  P
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ICMGGhnw4+002952 MODE=a
 IO-STREAM CLOSED - ICMGGhnw4+002952
 MPI-TASK:   1 -    345225290 BYTES IN      8 RECORDS TRANSFERRED IN    0.0002 SECONDS******** Mbytes/s, TOTAL TIME=    0.0070( 2.9%)
   !!!!!!!!!!!!!! WAVE FIELDS INTEGRATED.  DATE IS: 20200302231500  !!!!!!!!!!!! 
   !!!!!!!!!!!!!! WAVE FIELDS INTEGRATED.  DATE IS: 20200302233000  !!!!!!!!!!!! 
   !!!!!!!!!!!!!! WAVE FIELDS INTEGRATED.  DATE IS: 20200302234500  !!!!!!!!!!!! 
   !!!!!!!!!!!!!! WAVE FIELDS INTEGRATED.  DATE IS: 20200303000000  !!!!!!!!!!!! 
 12:05:34 STEP 2952 H=2952:00 +CPU= 25.592
 END CNT3
 NSTEP =  2952 CNT0   000000000  22.548  22.548  22.589   0.998               0               0               0               0               0
IO-STREAM STATISTICS, TOTAL NO OF REC/BYTES READ           35/   804992476 WRITTEN         2238/************
 -TASK-OPENED-OPEN-RECS IN -KBYTE IN    -RECS OUT-KBYTE OUT-WALL   -WALL IN-WALL
  OU-TOT IN -TOT OUT
   1     57    0       35       786125     2238 ************     7.0     0.7     6.2     1.1     9.8
===-=== START OF TIMING STATISTICS ===-===
 
STATS FOR ALL TASKS
 NUM ROUTINE                                     CALLS  MEAN(ms)   MAX(ms)   FRAC(%)  UNBAL(%)
   0 CNT0     - COMPLETE EXECUTION                   1 2382142.9 2382142.9    100.00      0.00
   1 CNT4     - FORWARD INTEGRATION                  1 2380377.4 2380377.4     99.93      0.00
   8 SCAN2M - GRID-POINT DYNAMICS                  157    1389.0    1389.0      9.15      0.00
   9 SPCM     - SPECTRAL COMP.                     144     222.8     222.8      1.35      0.00
  10 SCAN2M - PHYSICS                              145    6933.4    6933.4     42.20      0.00
  11 IOPACK   - OUTPUT P.P. RESULTS                 12     780.0     780.0      0.39      0.00
  12 SPNORM   - SPECTRAL NORM COMP.                  7      10.3      10.3      0.00      0.00
  13 SCAN2M - RADIATION CALC.                       49    7350.0    7350.0     15.12      0.00
  17 GRIDFPOS IN CNT4                               12      26.3      26.3      0.01      0.00
  27 RESTART FILES - WRITING                         6    1560.0    1560.0      0.39      0.00
  28 RESTART FILES - READING                         1    1492.1    1492.1      0.06      0.00
  29 SU4FPOS IN CNT4                                12       0.0       0.0      0.00      0.00
  30 DYNFPOS IN CNT4                                12    1264.7    1264.7      0.64      0.00
  31 POSDDH IN STEPO                                 1       0.5       0.5      0.00      0.00
  37 CPGLAG   - SL COMPUTATIONS                    145    2306.2    2306.2     14.04      0.00
  38 WAM      - TOTAL COST OF WAVE MODEL           145    1723.9    1723.9     10.49      0.00
  39 SU0YOMB                                         1     174.8     174.8      0.01      0.00
  51 SCAN2M   - SL COMM. PART 1                    145      13.3      13.3      0.08      0.00
  54 SPCM     - M TO S/S TO M TRANSP.              144     209.9     209.9      1.27      0.00
  55 SPCIMPF  - S TO M/M TO S TRANSP.              144      32.4      32.4      0.20      0.00
  56 SPNORM   - SPECTRAL NORM COMM.                  7       0.1       0.1      0.00      0.00
 102 LTINV_CTL   - INVERSE LEGENDRE TRANSFORM      498      91.3      91.3      1.91      0.00
 103 LTDIR_CTL   - DIRECT LEGENDRE TRANSFORM       300      87.3      87.3      1.10      0.00
 106 FTDIR_CTL   - DIRECT FOURIER TRANSFORM        300      35.5      35.5      0.45      0.00
 107 FTINV_CTL   - INVERSE FOURIER TRANSFORM       498      32.0      32.0      0.67      0.00
 140 SULEG       - COMP. OF LEGENDRE POL.            2      12.5      12.5      0.00      0.00
 152 LTINV_CTL   - M TO L TRANSPOSITION            498      13.1      13.1      0.27      0.00
 153 LTDIR_CTL   - L TO M TRANSPOSITION            300      11.2      11.2      0.14      0.00
 157 FTINV_CTL   - L TO G TRANSPOSITION            498      13.5      13.5      0.28      0.00
 158 FTDIR_CTL   - G TO L TRANSPOSITION            300       7.7       7.7      0.10      0.00
 400 GSTATS                                      29885       0.0       0.0      0.00      0.00
 401 GSTATS HOOK                                 28556       0.0       0.0      0.00      0.00
TOTAL MEASURED IMBALANCE =       0.0 SECONDS,  0.0 PERCENT
TOTAL WALLCLOCK TIME     2382.1 CPU TIME    2380.4 VECTOR TIME     2380.4


===-=== END   OF TIMING STATISTICS ===-===
FORECAST DAYS PER DAY     4464.5
  *** END CNT0 *** 
------------------------------------------------
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhnw4+002952
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhnw4+002952
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhnw4+002952
Adding to the zip: /var/lib/boinc-client/slots/5/NODE.001_01
Adding to the zip: /var/lib/boinc-client/slots/5/ifs.stat
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMGGhnw4+001152
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMSHhnw4+002952
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMUAhnw4+002940
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMGGhnw4+002952
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMUAhnw4+002328
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMUAhnw4+002952
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMSHhnw4+002328
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMSHhnw4+002940
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMSHhnw4+001152
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMSHhnw4+001608
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMGGhnw4+001608
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMGGhnw4+002808
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMGGhnw4+002940
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMUAhnw4+002808
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMSHhnw4+002808
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMUAhnw4+001608
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMUAhnw4+001152
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214916/ICMGGhnw4+002328
Zipping up the final file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_0152_2019110100_123_993_12214916_0_r1735244742_122.zip
Uploading the final file: upload_file_122.zip
Uploading trickle at timestep: 10623600
12:08:58 (10040): called boinc_finish(0)

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 May 2023 16:53:32 1414915 22316307 oifs_43r3_0152_2019110100_123_993_12214916_0 10,623,600 49,796 0.0047
09 May 2023 16:53:28 1414915 22316307 oifs_43r3_0152_2019110100_123_993_12214916_0 9,162,000 43,071 0.0047
09 May 2023 16:53:22 1414915 22316307 oifs_43r3_0152_2019110100_123_993_12214916_0 7,434,000 35,168 0.0047
09 May 2023 16:53:19 1414915 22316307 oifs_43r3_0152_2019110100_123_993_12214916_0 6,570,000 31,247 0.0048
09 May 2023 16:53:12 1414915 22316307 oifs_43r3_0152_2019110100_123_993_12214916_0 5,706,000 27,299 0.0048
09 May 2023 16:52:59 1414915 22316307 oifs_43r3_0152_2019110100_123_993_12214916_0 2,678,400 12,538 0.0047
09 May 2023 16:52:56 1414915 22316307 oifs_43r3_0152_2019110100_123_993_12214916_0 1,814,400 8,661 0.0048
09 May 2023 16:52:52 1414915 22316307 oifs_43r3_0152_2019110100_123_993_12214916_0 864,000 4,248 0.0049


©2024 cpdn.org