Task 22316905

Name	oifs_43r3_00ns_2019110100_123_993_12214294_1
Workunit	12214294
Created	24 Feb 2023, 23:02:36 UTC
Sent	25 Feb 2023, 2:22:56 UTC
Report deadline	26 Apr 2023, 2:22:56 UTC
Received	25 Feb 2023, 21:24:20 UTC
Server state	Over
Outcome	Success
Client state	Done
Exit status	0 (0x00000000)
Computer ID	1523950
Run time	11 hours 43 min 43 sec
CPU time	11 hours 43 min 43 sec
Validate state	Valid
Credit	14,873.04
Device peak FLOPS	6.37 GFLOPS
Application version	OpenIFS 43r3 v1.21 x86_64-pc-linux-gnu
Peak working set size	4,473.04 MB
Peak swap size	4,983.13 MB
Peak disk usage	3,218.25 MB
Stderr	<core_client_version>7.16.16</core_client_version> <![CDATA[ <stderr_txt> 549:00 +CPU= 11.669 13:34:16 STEP 2550 H=2550:00 +CPU= 18.996 13:34:28 STEP 2551 H=2551:00 +CPU= 11.784 13:34:40 STEP 2552 H=2552:00 +CPU= 12.029 13:34:58 STEP 2553 H=2553:00 +CPU= 18.573 13:35:11 STEP 2554 H=2554:00 +CPU= 12.126 13:35:22 STEP 2555 H=2555:00 +CPU= 11.702 13:35:43 STEP 2556 H=2556:00 +CPU= 20.212 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhnw4+002556 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhnw4+002556 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhnw4+002556 13:35:54 STEP 2557 H=2557:00 +CPU= 11.814 13:36:06 STEP 2558 H=2558:00 +CPU= 12.034 13:36:25 STEP 2559 H=2559:00 +CPU= 18.602 13:36:37 STEP 2560 H=2560:00 +CPU= 12.070 13:36:49 STEP 2561 H=2561:00 +CPU= 11.681 13:37:08 STEP 2562 H=2562:00 +CPU= 19.037 13:37:20 STEP 2563 H=2563:00 +CPU= 11.658 13:37:32 STEP 2564 H=2564:00 +CPU= 12.000 13:37:50 STEP 2565 H=2565:00 +CPU= 18.628 13:38:02 STEP 2566 H=2566:00 +CPU= 11.994 13:38:14 STEP 2567 H=2567:00 +CPU= 12.004 13:38:36 STEP 2568 H=2568:00 +CPU= 21.508 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhnw4+002568 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhnw4+002568 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhnw4+002568 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMGGhnw4+002544 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMSHhnw4+002544 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMUAhnw4+002544 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMGGhnw4+002556 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMSHhnw4+002556 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMUAhnw4+002556 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_00ns_2019110100_123_993_12214294_1_r627899484_106.zip Uploading the intermediate file: upload_file_106.zip 13:38:47 STEP 2569 H=2569:00 +CPU= 11.692 13:38:59 STEP 2570 H=2570:00 +CPU= 12.002 13:39:18 STEP 2571 H=2571:00 +CPU= 18.625 13:39:30 STEP 2572 H=2572:00 +CPU= 12.018 13:39:42 STEP 2573 H=2573:00 +CPU= 11.669 13:40:01 STEP 2574 H=2574:00 +CPU= 19.055 13:40:13 STEP 2575 H=2575:00 +CPU= 11.682 13:40:25 STEP 2576 H=2576:00 +CPU= 12.027 13:40:43 STEP 2577 H=2577:00 +CPU= 18.651 13:40:55 STEP 2578 H=2578:00 +CPU= 12.047 13:41:07 STEP 2579 H=2579:00 +CPU= 11.684 13:41:27 STEP 2580 H=2580:00 +CPU= 20.313 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhnw4+002580 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhnw4+002580 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhnw4+002580 13:41:39 STEP 2581 H=2581:00 +CPU= 11.688 13:41:51 STEP 2582 H=2582:00 +CPU= 12.036 13:42:10 STEP 2583 H=2583:00 +CPU= 18.708 13:42:22 STEP 2584 H=2584:00 +CPU= 12.050 13:42:34 STEP 2585 H=2585:00 +CPU= 11.677 13:42:53 STEP 2586 H=2586:00 +CPU= 19.078 13:43:04 STEP 2587 H=2587:00 +CPU= 11.671 13:43:16 STEP 2588 H=2588:00 +CPU= 12.007 13:43:35 STEP 2589 H=2589:00 +CPU= 18.544 13:43:47 STEP 2590 H=2590:00 +CPU= 12.014 13:43:59 STEP 2591 H=2591:00 +CPU= 12.009 13:44:21 STEP 2592 H=2592:00 +CPU= 21.484 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhnw4+002592 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhnw4+002592 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhnw4+002592 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMGGhnw4+002568 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMSHhnw4+002568 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMUAhnw4+002568 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMGGhnw4+002580 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMSHhnw4+002580 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMUAhnw4+002580 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_00ns_2019110100_123_993_12214294_1_r627899484_107.zip Uploading the intermediate file: upload_file_107.zip 13:44:32 STEP 2593 H=2593:00 +CPU= 11.698 13:44:44 STEP 2594 H=2594:00 +CPU= 12.001 13:45:03 STEP 2595 H=2595:00 +CPU= 18.592 13:45:15 STEP 2596 H=2596:00 +CPU= 12.004 13:45:27 STEP 2597 H=2597:00 +CPU= 11.669 13:45:46 STEP 2598 H=2598:00 +CPU= 19.005 13:45:57 STEP 2599 H=2599:00 +CPU= 11.692 13:46:09 STEP 2600 H=2600:00 +CPU= 12.033 13:46:28 STEP 2601 H=2601:00 +CPU= 18.679 13:46:40 STEP 2602 H=2602:00 +CPU= 12.050 13:46:52 STEP 2603 H=2603:00 +CPU= 11.691 13:47:12 STEP 2604 H=2604:00 +CPU= 20.261 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhnw4+002604 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhnw4+002604 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhnw4+002604 13:47:24 STEP 2605 H=2605:00 +CPU= 11.708 13:47:36 STEP 2606 H=2606:00 +CPU= 12.039 13:47:55 STEP 2607 H=2607:00 +CPU= 18.602 13:48:07 STEP 2608 H=2608:00 +CPU= 12.032 13:48:18 STEP 2609 H=2609:00 +CPU= 11.681 13:48:37 STEP 2610 H=2610:00 +CPU= 18.969 13:48:49 STEP 2611 H=2611:00 +CPU= 12.063 13:49:01 STEP 2612 H=2612:00 +CPU= 12.022 13:49:20 STEP 2613 H=2613:00 +CPU= 18.588 13:49:32 STEP 2614 H=2614:00 +CPU= 11.988 13:49:44 STEP 2615 H=2615:00 +CPU= 11.969 13:50:06 STEP 2616 H=2616:00 +CPU= 21.466 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhnw4+002616 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhnw4+002616 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhnw4+002616 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMGGhnw4+002592 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMSHhnw4+002592 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMUAhnw4+002592 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMGGhnw4+002604 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMSHhnw4+002604 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMUAhnw4+002604 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_00ns_2019110100_123_993_12214294_1_r627899484_108.zip Uploading the intermediate file: upload_file_108.zip 13:50:17 STEP 2617 H=2617:00 +CPU= 11.659 13:50:29 STEP 2618 H=2618:00 +CPU= 11.972 13:50:48 STEP 2619 H=2619:00 +CPU= 18.489 13:51:00 STEP 2620 H=2620:00 +CPU= 11.981 13:51:11 STEP 2621 H=2621:00 +CPU= 11.699 13:51:30 STEP 2622 H=2622:00 +CPU= 18.884 13:51:42 STEP 2623 H=2623:00 +CPU= 11.634 13:51:54 STEP 2624 H=2624:00 +CPU= 12.077 13:52:13 STEP 2625 H=2625:00 +CPU= 18.504 13:52:25 STEP 2626 H=2626:00 +CPU= 11.987 13:52:36 STEP 2627 H=2627:00 +CPU= 11.697 13:52:57 STEP 2628 H=2628:00 +CPU= 20.239 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhnw4+002628 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhnw4+002628 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhnw4+002628 13:53:08 STEP 2629 H=2629:00 +CPU= 11.620 13:53:20 STEP 2630 H=2630:00 +CPU= 12.060 13:53:39 STEP 2631 H=2631:00 +CPU= 18.485 13:53:51 STEP 2632 H=2632:00 +CPU= 12.006 13:54:02 STEP 2633 H=2633:00 +CPU= 11.710 13:54:21 STEP 2634 H=2634:00 +CPU= 18.891 13:54:33 STEP 2635 H=2635:00 +CPU= 11.622 13:54:45 STEP 2636 H=2636:00 +CPU= 12.044 13:55:04 STEP 2637 H=2637:00 +CPU= 18.469 13:55:15 STEP 2638 H=2638:00 +CPU= 11.938 13:55:28 STEP 2639 H=2639:00 +CPU= 12.044 13:55:49 STEP 2640 H=2640:00 +CPU= 21.389 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhnw4+002640 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhnw4+002640 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhnw4+002640 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMGGhnw4+002616 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMSHhnw4+002616 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMUAhnw4+002616 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMGGhnw4+002628 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMSHhnw4+002628 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMUAhnw4+002628 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_00ns_2019110100_123_993_12214294_1_r627899484_109.zip Uploading the intermediate file: upload_file_109.zip 13:56:01 STEP 2641 H=2641:00 +CPU= 11.595 13:56:13 STEP 2642 H=2642:00 +CPU= 11.993 Uploading trickle at timestep: 9504000 13:56:31 STEP 2643 H=2643:00 +CPU= 18.471 13:56:43 STEP 2644 H=2644:00 +CPU= 11.953 13:56:55 STEP 2645 H=2645:00 +CPU= 11.628 13:57:14 STEP 2646 H=2646:00 +CPU= 18.851 13:57:25 STEP 2647 H=2647:00 +CPU= 11.612 13:57:37 STEP 2648 H=2648:00 +CPU= 11.953 13:57:56 STEP 2649 H=2649:00 +CPU= 18.404 13:58:08 STEP 2650 H=2650:00 +CPU= 11.952 13:58:19 STEP 2651 H=2651:00 +CPU= 11.593 13:58:39 STEP 2652 H=2652:00 +CPU= 20.056 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhnw4+002652 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhnw4+002652 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhnw4+002652 13:58:51 STEP 2653 H=2653:00 +CPU= 11.644 13:59:03 STEP 2654 H=2654:00 +CPU= 11.958 13:59:21 STEP 2655 H=2655:00 +CPU= 18.396 13:59:33 STEP 2656 H=2656:00 +CPU= 11.952 13:59:45 STEP 2657 H=2657:00 +CPU= 11.609 14:00:04 STEP 2658 H=2658:00 +CPU= 18.828 14:00:15 STEP 2659 H=2659:00 +CPU= 11.652 14:00:27 STEP 2660 H=2660:00 +CPU= 11.950 14:00:46 STEP 2661 H=2661:00 +CPU= 18.402 14:00:58 STEP 2662 H=2662:00 +CPU= 11.988 14:01:10 STEP 2663 H=2663:00 +CPU= 11.906 14:01:31 STEP 2664 H=2664:00 +CPU= 21.320 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhnw4+002664 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhnw4+002664 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhnw4+002664 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMGGhnw4+002640 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMSHhnw4+002640 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMUAhnw4+002640 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMGGhnw4+002652 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMSHhnw4+002652 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMUAhnw4+002652 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_00ns_2019110100_123_993_12214294_1_r627899484_110.zip Uploading the intermediate file: upload_file_110.zip 14:01:43 STEP 2665 H=2665:00 +CPU= 11.582 14:01:55 STEP 2666 H=2666:00 +CPU= 11.917 14:02:13 STEP 2667 H=2667:00 +CPU= 18.345 14:02:25 STEP 2668 H=2668:00 +CPU= 11.975 14:02:36 STEP 2669 H=2669:00 +CPU= 11.559 14:02:55 STEP 2670 H=2670:00 +CPU= 18.771 14:03:07 STEP 2671 H=2671:00 +CPU= 11.673 14:03:19 STEP 2672 H=2672:00 +CPU= 11.916 14:03:37 STEP 2673 H=2673:00 +CPU= 18.382 14:03:49 STEP 2674 H=2674:00 +CPU= 11.934 14:04:01 STEP 2675 H=2675:00 +CPU= 11.570 14:04:21 STEP 2676 H=2676:00 +CPU= 20.064 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhnw4+002676 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhnw4+002676 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhnw4+002676 14:04:32 STEP 2677 H=2677:00 +CPU= 11.577 14:04:44 STEP 2678 H=2678:00 +CPU= 11.913 14:05:03 STEP 2679 H=2679:00 +CPU= 18.411 14:05:15 STEP 2680 H=2680:00 +CPU= 11.863 14:05:26 STEP 2681 H=2681:00 +CPU= 11.562 14:05:45 STEP 2682 H=2682:00 +CPU= 18.780 14:05:57 STEP 2683 H=2683:00 +CPU= 11.534 14:06:09 STEP 2684 H=2684:00 +CPU= 11.891 14:06:27 STEP 2685 H=2685:00 +CPU= 18.353 14:06:39 STEP 2686 H=2686:00 +CPU= 11.882 14:06:51 STEP 2687 H=2687:00 +CPU= 11.906 14:07:12 STEP 2688 H=2688:00 +CPU= 21.252 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhnw4+002688 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhnw4+002688 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhnw4+002688 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMGGhnw4+002664 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMSHhnw4+002664 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMUAhnw4+002664 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMGGhnw4+002676 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMSHhnw4+002676 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMUAhnw4+002676 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_00ns_2019110100_123_993_12214294_1_r627899484_111.zip Uploading the intermediate file: upload_file_111.zip 14:07:24 STEP 2689 H=2689:00 +CPU= 11.531 14:07:35 STEP 2690 H=2690:00 +CPU= 11.910 14:07:54 STEP 2691 H=2691:00 +CPU= 18.358 14:08:06 STEP 2692 H=2692:00 +CPU= 11.886 14:08:17 STEP 2693 H=2693:00 +CPU= 11.548 14:08:36 STEP 2694 H=2694:00 +CPU= 18.773 14:08:48 STEP 2695 H=2695:00 +CPU= 11.543 14:09:00 STEP 2696 H=2696:00 +CPU= 11.906 14:09:18 STEP 2697 H=2697:00 +CPU= 18.382 14:09:30 STEP 2698 H=2698:00 +CPU= 11.892 14:09:41 STEP 2699 H=2699:00 +CPU= 11.542 14:10:01 STEP 2700 H=2700:00 +CPU= 20.027 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhnw4+002700 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhnw4+002700 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhnw4+002700 14:10:13 STEP 2701 H=2701:00 +CPU= 11.532 14:10:25 STEP 2702 H=2702:00 +CPU= 11.897 14:10:43 STEP 2703 H=2703:00 +CPU= 18.428 14:10:55 STEP 2704 H=2704:00 +CPU= 11.902 14:11:07 STEP 2705 H=2705:00 +CPU= 11.536 14:11:26 STEP 2706 H=2706:00 +CPU= 18.735 14:11:37 STEP 2707 H=2707:00 +CPU= 11.528 14:11:49 STEP 2708 H=2708:00 +CPU= 11.881 14:12:07 STEP 2709 H=2709:00 +CPU= 18.274 14:12:19 STEP 2710 H=2710:00 +CPU= 11.888 14:12:31 STEP 2711 H=2711:00 +CPU= 11.891 14:12:52 STEP 2712 H=2712:00 +CPU= 21.231 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhnw4+002712 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhnw4+002712 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhnw4+002712 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMGGhnw4+002688 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMSHhnw4+002688 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMUAhnw4+002688 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMGGhnw4+002700 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMSHhnw4+002700 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMUAhnw4+002700 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_00ns_2019110100_123_993_12214294_1_r627899484_112.zip Uploading the intermediate file: upload_file_112.zip 14:13:04 STEP 2713 H=2713:00 +CPU= 11.521 14:13:16 STEP 2714 H=2714:00 +CPU= 11.908 14:13:34 STEP 2715 H=2715:00 +CPU= 18.301 14:13:46 STEP 2716 H=2716:00 +CPU= 11.883 14:13:58 STEP 2717 H=2717:00 +CPU= 11.551 14:14:16 STEP 2718 H=2718:00 +CPU= 18.737 14:14:28 STEP 2719 H=2719:00 +CPU= 11.568 14:14:40 STEP 2720 H=2720:00 +CPU= 11.908 14:14:58 STEP 2721 H=2721:00 +CPU= 18.417 14:15:10 STEP 2722 H=2722:00 +CPU= 11.898 14:15:22 STEP 2723 H=2723:00 +CPU= 11.556 14:15:42 STEP 2724 H=2724:00 +CPU= 20.010 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhnw4+002724 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhnw4+002724 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhnw4+002724 14:15:53 STEP 2725 H=2725:00 +CPU= 11.540 14:16:05 STEP 2726 H=2726:00 +CPU= 11.892 14:16:24 STEP 2727 H=2727:00 +CPU= 18.366 14:16:35 STEP 2728 H=2728:00 +CPU= 11.883 14:16:47 STEP 2729 H=2729:00 +CPU= 11.547 14:17:06 STEP 2730 H=2730:00 +CPU= 18.671 14:17:17 STEP 2731 H=2731:00 +CPU= 11.528 14:17:29 STEP 2732 H=2732:00 +CPU= 11.872 14:17:47 STEP 2733 H=2733:00 +CPU= 18.223 14:17:59 STEP 2734 H=2734:00 +CPU= 11.943 14:18:11 STEP 2735 H=2735:00 +CPU= 11.877 14:18:32 STEP 2736 H=2736:00 +CPU= 21.179 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhnw4+002736 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhnw4+002736 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhnw4+002736 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMGGhnw4+002712 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMSHhnw4+002712 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMUAhnw4+002712 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMGGhnw4+002724 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMSHhnw4+002724 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMUAhnw4+002724 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_00ns_2019110100_123_993_12214294_1_r627899484_113.zip Uploading the intermediate file: upload_file_113.zip 14:18:44 STEP 2737 H=2737:00 +CPU= 11.564 14:18:56 STEP 2738 H=2738:00 +CPU= 11.887 14:19:14 STEP 2739 H=2739:00 +CPU= 18.268 14:19:26 STEP 2740 H=2740:00 +CPU= 11.902 14:19:38 STEP 2741 H=2741:00 +CPU= 11.525 14:19:56 STEP 2742 H=2742:00 +CPU= 18.655 14:20:08 STEP 2743 H=2743:00 +CPU= 11.540 14:20:20 STEP 2744 H=2744:00 +CPU= 11.974 14:20:38 STEP 2745 H=2745:00 +CPU= 18.262 14:20:50 STEP 2746 H=2746:00 +CPU= 11.899 14:21:02 STEP 2747 H=2747:00 +CPU= 11.619 14:21:22 STEP 2748 H=2748:00 +CPU= 19.920 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhnw4+002748 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhnw4+002748 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhnw4+002748 14:21:33 STEP 2749 H=2749:00 +CPU= 11.528 14:21:45 STEP 2750 H=2750:00 +CPU= 11.934 14:22:03 STEP 2751 H=2751:00 +CPU= 18.273 14:22:15 STEP 2752 H=2752:00 +CPU= 11.887 14:22:27 STEP 2753 H=2753:00 +CPU= 11.541 14:22:46 STEP 2754 H=2754:00 +CPU= 18.706 14:22:57 STEP 2755 H=2755:00 +CPU= 11.525 14:23:09 STEP 2756 H=2756:00 +CPU= 11.854 14:23:27 STEP 2757 H=2757:00 +CPU= 18.286 14:23:39 STEP 2758 H=2758:00 +CPU= 11.863 14:23:51 STEP 2759 H=2759:00 +CPU= 11.870 14:24:12 STEP 2760 H=2760:00 +CPU= 21.199 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhnw4+002760 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhnw4+002760 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhnw4+002760 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMGGhnw4+002736 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMSHhnw4+002736 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMUAhnw4+002736 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMGGhnw4+002748 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMSHhnw4+002748 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMUAhnw4+002748 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_00ns_2019110100_123_993_12214294_1_r627899484_114.zip Uploading the intermediate file: upload_file_114.zip 14:24:24 STEP 2761 H=2761:00 +CPU= 11.525 14:24:36 STEP 2762 H=2762:00 +CPU= 11.869 14:24:54 STEP 2763 H=2763:00 +CPU= 18.407 14:25:06 STEP 2764 H=2764:00 +CPU= 11.871 14:25:18 STEP 2765 H=2765:00 +CPU= 11.524 14:25:36 STEP 2766 H=2766:00 +CPU= 18.722 14:25:48 STEP 2767 H=2767:00 +CPU= 11.543 14:26:00 STEP 2768 H=2768:00 +CPU= 11.879 14:26:18 STEP 2769 H=2769:00 +CPU= 18.300 14:26:30 STEP 2770 H=2770:00 +CPU= 11.892 14:26:41 STEP 2771 H=2771:00 +CPU= 11.507 14:27:01 STEP 2772 H=2772:00 +CPU= 19.887 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhnw4+002772 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhnw4+002772 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhnw4+002772 14:27:13 STEP 2773 H=2773:00 +CPU= 11.623 14:27:25 STEP 2774 H=2774:00 +CPU= 11.898 14:27:43 STEP 2775 H=2775:00 +CPU= 18.209 14:27:55 STEP 2776 H=2776:00 +CPU= 11.933 14:28:07 STEP 2777 H=2777:00 +CPU= 11.523 14:28:25 STEP 2778 H=2778:00 +CPU= 18.633 14:28:37 STEP 2779 H=2779:00 +CPU= 11.551 14:28:49 STEP 2780 H=2780:00 +CPU= 11.874 14:29:07 STEP 2781 H=2781:00 +CPU= 18.210 14:29:19 STEP 2782 H=2782:00 +CPU= 11.908 14:29:31 STEP 2783 H=2783:00 +CPU= 11.874 14:29:52 STEP 2784 H=2784:00 +CPU= 21.118 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhnw4+002784 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhnw4+002784 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhnw4+002784 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMGGhnw4+002760 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMSHhnw4+002760 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMUAhnw4+002760 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMGGhnw4+002772 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMSHhnw4+002772 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMUAhnw4+002772 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_00ns_2019110100_123_993_12214294_1_r627899484_115.zip Uploading the intermediate file: upload_file_115.zip 14:30:03 STEP 2785 H=2785:00 +CPU= 11.524 14:30:15 STEP 2786 H=2786:00 +CPU= 11.836 14:30:33 STEP 2787 H=2787:00 +CPU= 18.205 14:30:45 STEP 2788 H=2788:00 +CPU= 11.846 14:30:57 STEP 2789 H=2789:00 +CPU= 11.557 14:31:16 STEP 2790 H=2790:00 +CPU= 18.602 14:31:27 STEP 2791 H=2791:00 +CPU= 11.517 14:31:39 STEP 2792 H=2792:00 +CPU= 11.922 14:31:57 STEP 2793 H=2793:00 +CPU= 18.214 14:32:09 STEP 2794 H=2794:00 +CPU= 11.875 14:32:21 STEP 2795 H=2795:00 +CPU= 11.530 14:32:41 STEP 2796 H=2796:00 +CPU= 19.926 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhnw4+002796 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhnw4+002796 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhnw4+002796 14:32:52 STEP 2797 H=2797:00 +CPU= 11.514 14:33:04 STEP 2798 H=2798:00 +CPU= 11.887 14:33:22 STEP 2799 H=2799:00 +CPU= 18.270 14:33:34 STEP 2800 H=2800:00 +CPU= 11.878 14:33:46 STEP 2801 H=2801:00 +CPU= 11.543 14:34:04 STEP 2802 H=2802:00 +CPU= 18.627 14:34:16 STEP 2803 H=2803:00 +CPU= 11.517 14:34:28 STEP 2804 H=2804:00 +CPU= 11.870 14:34:46 STEP 2805 H=2805:00 +CPU= 18.267 14:34:58 STEP 2806 H=2806:00 +CPU= 11.859 14:35:10 STEP 2807 H=2807:00 +CPU= 11.893 14:35:31 STEP 2808 H=2808:00 +CPU= 21.167 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhnw4+002808 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhnw4+002808 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhnw4+002808 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMGGhnw4+002784 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMSHhnw4+002784 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMUAhnw4+002784 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMGGhnw4+002796 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMSHhnw4+002796 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMUAhnw4+002796 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_00ns_2019110100_123_993_12214294_1_r627899484_116.zip Uploading the intermediate file: upload_file_116.zip 14:35:43 STEP 2809 H=2809:00 +CPU= 11.533 14:35:54 STEP 2810 H=2810:00 +CPU= 11.864 14:36:13 STEP 2811 H=2811:00 +CPU= 18.263 14:36:25 STEP 2812 H=2812:00 +CPU= 11.868 14:36:36 STEP 2813 H=2813:00 +CPU= 11.524 14:36:55 STEP 2814 H=2814:00 +CPU= 18.653 14:37:06 STEP 2815 H=2815:00 +CPU= 11.590 14:37:18 STEP 2816 H=2816:00 +CPU= 11.868 14:37:37 STEP 2817 H=2817:00 +CPU= 18.247 14:37:49 STEP 2818 H=2818:00 +CPU= 11.923 14:38:00 STEP 2819 H=2819:00 +CPU= 11.526 14:38:20 STEP 2820 H=2820:00 +CPU= 19.867 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhnw4+002820 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhnw4+002820 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhnw4+002820 14:38:32 STEP 2821 H=2821:00 +CPU= 11.546 14:38:43 STEP 2822 H=2822:00 +CPU= 11.863 14:39:02 STEP 2823 H=2823:00 +CPU= 18.234 14:39:14 STEP 2824 H=2824:00 +CPU= 11.911 14:39:25 STEP 2825 H=2825:00 +CPU= 11.512 14:39:44 STEP 2826 H=2826:00 +CPU= 18.610 14:39:55 STEP 2827 H=2827:00 +CPU= 11.508 14:40:07 STEP 2828 H=2828:00 +CPU= 11.902 14:40:25 STEP 2829 H=2829:00 +CPU= 18.191 14:40:37 STEP 2830 H=2830:00 +CPU= 11.866 14:40:49 STEP 2831 H=2831:00 +CPU= 11.890 14:41:10 STEP 2832 H=2832:00 +CPU= 21.114 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhnw4+002832 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhnw4+002832 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhnw4+002832 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMGGhnw4+002808 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMSHhnw4+002808 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMUAhnw4+002808 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMGGhnw4+002820 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMSHhnw4+002820 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMUAhnw4+002820 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_00ns_2019110100_123_993_12214294_1_r627899484_117.zip Uploading the intermediate file: upload_file_117.zip 14:41:22 STEP 2833 H=2833:00 +CPU= 11.513 14:41:34 STEP 2834 H=2834:00 +CPU= 11.881 14:41:52 STEP 2835 H=2835:00 +CPU= 18.199 14:42:04 STEP 2836 H=2836:00 +CPU= 11.848 14:42:15 STEP 2837 H=2837:00 +CPU= 11.516 14:42:34 STEP 2838 H=2838:00 +CPU= 18.623 14:42:46 STEP 2839 H=2839:00 +CPU= 11.521 14:42:57 STEP 2840 H=2840:00 +CPU= 11.874 14:43:16 STEP 2841 H=2841:00 +CPU= 18.270 14:43:28 STEP 2842 H=2842:00 +CPU= 11.884 14:43:39 STEP 2843 H=2843:00 +CPU= 11.514 14:43:59 STEP 2844 H=2844:00 +CPU= 19.922 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhnw4+002844 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhnw4+002844 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhnw4+002844 14:44:11 STEP 2845 H=2845:00 +CPU= 11.535 14:44:23 STEP 2846 H=2846:00 +CPU= 11.882 14:44:41 STEP 2847 H=2847:00 +CPU= 18.358 14:44:53 STEP 2848 H=2848:00 +CPU= 11.874 14:45:04 STEP 2849 H=2849:00 +CPU= 11.519 14:45:23 STEP 2850 H=2850:00 +CPU= 18.686 14:45:35 STEP 2851 H=2851:00 +CPU= 11.517 14:45:46 STEP 2852 H=2852:00 +CPU= 11.885 14:46:05 STEP 2853 H=2853:00 +CPU= 18.274 14:46:17 STEP 2854 H=2854:00 +CPU= 11.844 14:46:28 STEP 2855 H=2855:00 +CPU= 11.873 14:46:50 STEP 2856 H=2856:00 +CPU= 21.157 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhnw4+002856 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhnw4+002856 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhnw4+002856 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMGGhnw4+002832 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMSHhnw4+002832 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMUAhnw4+002832 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMGGhnw4+002844 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMSHhnw4+002844 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMUAhnw4+002844 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_00ns_2019110100_123_993_12214294_1_r627899484_118.zip Uploading the intermediate file: upload_file_118.zip 14:47:01 STEP 2857 H=2857:00 +CPU= 11.520 14:47:13 STEP 2858 H=2858:00 +CPU= 11.861 14:47:31 STEP 2859 H=2859:00 +CPU= 18.258 14:47:43 STEP 2860 H=2860:00 +CPU= 11.909 14:47:55 STEP 2861 H=2861:00 +CPU= 11.507 14:48:13 STEP 2862 H=2862:00 +CPU= 18.616 14:48:25 STEP 2863 H=2863:00 +CPU= 11.572 14:48:37 STEP 2864 H=2864:00 +CPU= 11.869 14:48:55 STEP 2865 H=2865:00 +CPU= 18.249 14:49:07 STEP 2866 H=2866:00 +CPU= 11.886 14:49:19 STEP 2867 H=2867:00 +CPU= 11.539 14:49:39 STEP 2868 H=2868:00 +CPU= 19.923 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhnw4+002868 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhnw4+002868 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhnw4+002868 14:49:50 STEP 2869 H=2869:00 +CPU= 11.538 14:50:02 STEP 2870 H=2870:00 +CPU= 11.937 14:50:20 STEP 2871 H=2871:00 +CPU= 18.261 14:50:32 STEP 2872 H=2872:00 +CPU= 11.873 14:50:44 STEP 2873 H=2873:00 +CPU= 11.569 14:51:02 STEP 2874 H=2874:00 +CPU= 18.610 14:51:14 STEP 2875 H=2875:00 +CPU= 11.508 14:51:26 STEP 2876 H=2876:00 +CPU= 11.884 14:51:44 STEP 2877 H=2877:00 +CPU= 18.222 14:51:56 STEP 2878 H=2878:00 +CPU= 11.846 14:52:08 STEP 2879 H=2879:00 +CPU= 11.862 14:52:29 STEP 2880 H=2880:00 +CPU= 21.189 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhnw4+002880 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhnw4+002880 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhnw4+002880 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMGGhnw4+002856 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMSHhnw4+002856 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMUAhnw4+002856 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMGGhnw4+002868 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMSHhnw4+002868 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMUAhnw4+002868 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_00ns_2019110100_123_993_12214294_1_r627899484_119.zip Uploading the intermediate file: upload_file_119.zip 14:52:41 STEP 2881 H=2881:00 +CPU= 11.538 Uploading trickle at timestep: 10368000 14:52:52 STEP 2882 H=2882:00 +CPU= 11.856 14:53:11 STEP 2883 H=2883:00 +CPU= 18.293 14:53:23 STEP 2884 H=2884:00 +CPU= 11.866 14:53:34 STEP 2885 H=2885:00 +CPU= 11.527 14:53:53 STEP 2886 H=2886:00 +CPU= 18.705 14:54:05 STEP 2887 H=2887:00 +CPU= 11.550 14:54:16 STEP 2888 H=2888:00 +CPU= 11.869 14:54:35 STEP 2889 H=2889:00 +CPU= 18.340 14:54:47 STEP 2890 H=2890:00 +CPU= 11.887 14:54:58 STEP 2891 H=2891:00 +CPU= 11.536 14:55:18 STEP 2892 H=2892:00 +CPU= 19.981 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhnw4+002892 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhnw4+002892 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhnw4+002892 14:55:30 STEP 2893 H=2893:00 +CPU= 11.524 14:55:42 STEP 2894 H=2894:00 +CPU= 11.870 14:56:00 STEP 2895 H=2895:00 +CPU= 18.281 14:56:12 STEP 2896 H=2896:00 +CPU= 11.927 14:56:23 STEP 2897 H=2897:00 +CPU= 11.518 14:56:42 STEP 2898 H=2898:00 +CPU= 18.656 14:56:54 STEP 2899 H=2899:00 +CPU= 11.571 14:57:06 STEP 2900 H=2900:00 +CPU= 11.862 14:57:24 STEP 2901 H=2901:00 +CPU= 18.263 14:57:36 STEP 2902 H=2902:00 +CPU= 11.916 14:57:48 STEP 2903 H=2903:00 +CPU= 11.874 14:58:09 STEP 2904 H=2904:00 +CPU= 21.190 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhnw4+002904 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhnw4+002904 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhnw4+002904 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMGGhnw4+002880 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMSHhnw4+002880 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMUAhnw4+002880 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMGGhnw4+002892 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMSHhnw4+002892 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMUAhnw4+002892 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_00ns_2019110100_123_993_12214294_1_r627899484_120.zip Uploading the intermediate file: upload_file_120.zip 14:58:20 STEP 2905 H=2905:00 +CPU= 11.531 14:58:32 STEP 2906 H=2906:00 +CPU= 11.867 14:58:51 STEP 2907 H=2907:00 +CPU= 18.264 14:59:02 STEP 2908 H=2908:00 +CPU= 11.862 14:59:14 STEP 2909 H=2909:00 +CPU= 11.564 14:59:33 STEP 2910 H=2910:00 +CPU= 18.653 14:59:44 STEP 2911 H=2911:00 +CPU= 11.486 14:59:56 STEP 2912 H=2912:00 +CPU= 11.955 15:00:14 STEP 2913 H=2913:00 +CPU= 18.316 15:00:26 STEP 2914 H=2914:00 +CPU= 11.876 15:00:38 STEP 2915 H=2915:00 +CPU= 11.618 15:00:58 STEP 2916 H=2916:00 +CPU= 19.929 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhnw4+002916 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhnw4+002916 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhnw4+002916 15:01:09 STEP 2917 H=2917:00 +CPU= 11.533 15:01:21 STEP 2918 H=2918:00 +CPU= 11.885 15:01:40 STEP 2919 H=2919:00 +CPU= 18.342 15:01:52 STEP 2920 H=2920:00 +CPU= 11.889 15:02:03 STEP 2921 H=2921:00 +CPU= 11.545 15:02:22 STEP 2922 H=2922:00 +CPU= 18.696 15:02:33 STEP 2923 H=2923:00 +CPU= 11.509 15:02:45 STEP 2924 H=2924:00 +CPU= 11.863 Quit request received from BOINC client, ending the child process >>> Printing last 70 lines from file: NODE.001_01 NSTEP = 2922 STEPO 0AAA00AAA SPECTRAL NORMS: Pattern 1 STEPO, YSPSDT_AR1 ( 1,...), SP NORMS 2D: 5.093703079699E-01 1.815907612454E-01 6.406908249740E-02 STEPO, YSPSDT_AR1 (SUM,...), SP NORMS 2D: 5.356401319879E-01 STEPO, YGPSDT ( 1,...), Grid Point Norms FIELD LEV AVERAGE MINIMUM MAXIMUM 101 60 5.09842352097039E-02 -1.78905341103120E+00 2.62085111350809E+00 STEPO, YGPSDT[clipped]( 1,...), Grid Point Norms FIELD LEV AVERAGE MINIMUM MAXIMUM 101 60 4.34322865626389E-02 -1.00000000000000E+00 1.00000000000000E+00 STEPO: evolving YSPSDT_AR1( 1) !!!!!!!!!!!!!! WAVE FIELDS INTEGRATED. DATE IS: 20200301171500 !!!!!!!!!!!! !!!!!!!!!!!!!! WAVE FIELDS INTEGRATED. DATE IS: 20200301173000 !!!!!!!!!!!! !!!!!!!!!!!!!! WAVE FIELDS INTEGRATED. DATE IS: 20200301174500 !!!!!!!!!!!! !!!!!!!!!!!!!! WAVE FIELDS INTEGRATED. DATE IS: 20200301180000 !!!!!!!!!!!! 15:02:22 STEP 2922 H=2922:00 +CPU= 18.696 NSTEP = 2923 STEPO 0AAA00AAA SPECTRAL NORMS: Pattern 1 STEPO, YSPSDT_AR1 ( 1,...), SP NORMS 2D: 5.217020780678E-01 1.787855279416E-01 6.456405929077E-02 STEPO, YSPSDT_AR1 (SUM,...), SP NORMS 2D: 5.480135508661E-01 STEPO, YGPSDT ( 1,...), Grid Point Norms FIELD LEV AVERAGE MINIMUM MAXIMUM 101 60 3.38971423530101E-02 -1.94194378930600E+00 2.38885235156763E+00 STEPO, YGPSDT[clipped]( 1,...), Grid Point Norms FIELD LEV AVERAGE MINIMUM MAXIMUM 101 60 2.95240797349227E-02 -1.00000000000000E+00 1.00000000000000E+00 STEPO: evolving YSPSDT_AR1( 1) !!!!!!!!!!!!!! WAVE FIELDS INTEGRATED. DATE IS: 20200301181500 !!!!!!!!!!!! !!!!!!!!!!!!!! WAVE FIELDS INTEGRATED. DATE IS: 20200301183000 !!!!!!!!!!!! !!!!!!!!!!!!!! WAVE FIELDS INTEGRATED. DATE IS: 20200301184500 !!!!!!!!!!!! !!!!!!!!!!!!!! WAVE FIELDS INTEGRATED. DATE IS: 20200301190000 !!!!!!!!!!!! 15:02:33 STEP 2923 H=2923:00 +CPU= 11.509 NSTEP = 2924 STEPO 0AAA00AAA SPECTRAL NORMS: Pattern 1 STEPO, YSPSDT_AR1 ( 1,...), SP NORMS 2D: 5.227130066730E-01 1.769061522677E-01 6.553896800849E-02 STEPO, YSPSDT_AR1 (SUM,...), SP NORMS 2D: 5.504970767988E-01 STEPO, YGPSDT ( 1,...), Grid Point Norms FIELD LEV AVERAGE MINIMUM MAXIMUM 101 60 2.05048555128458E-02 -1.95590448487378E+00 2.07422579437768E+00 STEPO, YGPSDT[clipped]( 1,...), Grid Point Norms FIELD LEV AVERAGE MINIMUM MAXIMUM 101 60 1.62094629336749E-02 -1.00000000000000E+00 1.00000000000000E+00 STEPO: evolving YSPSDT_AR1( 1) !!!!!!!!!!!!!! WAVE FIELDS INTEGRATED. DATE IS: 20200301191500 !!!!!!!!!!!! !!!!!!!!!!!!!! WAVE FIELDS INTEGRATED. DATE IS: 20200301193000 !!!!!!!!!!!! !!!!!!!!!!!!!! WAVE FIELDS INTEGRATED. DATE IS: 20200301194500 !!!!!!!!!!!! !!!!!!!!!!!!!! WAVE FIELDS INTEGRATED. DATE IS: 20200301200000 !!!!!!!!!!!! 15:02:45 STEP 2924 H=2924:00 +CPU= 11.863 ISTEP= 2925 RSOLINC= 1360.9440 RI0= 1366.0000 ZI0= 1360.9440 RIP0= 1385.7303 ISTEP= 2925 ZI0= 1360.9440 ZSEASON= 1.018212563 REA=149597870000.00 RDEASO=148253919687.74 DATE= 2020 3 1 ISTEP= 2925: Reset RSOLINC= 1360.9440 RIP0= 1385.7303 SU_GHGCLIM, RCARDI,RCH4,RN2O,RCFC11,RCFC12,RNO2: 6.251491506422999E-004 9.602631721217504E-007 4.999421765450250E-007 9.551005861072499E-010 2.020762199028000E-009 7.940283000000001E-011 SUECAEC: GLOBAL AVERAGE VOLCANIC SULPHATE FOR (KINDAT,KMINUT) 20191101 175590 1.159977703455965E-004 SUECSO4: USE AVAILABLE FILES FOR DECADES: 2010 2020 SUBROUTINE SUECSO4:, UNWEIGHTED AVERAGE: 2.376642126204221E-003 NSTEP = 2925 STEPO 0AAA00AAA SPECTRAL NORMS: Pattern 1 STEPO, YSPSDT_AR1 ( 1,...), SP NORMS 2D: 5.304787348047E-01 1.801154243641E-01 6.472228492486E-02 STEPO, YSPSDT_AR1 (SUM,...), SP NORMS 2D: 5.557807181913E-01 STEPO, YGPSDT ( 1,...), Grid Point Norms FIELD LEV AVERAGE MINIMUM MAXIMUM 101 60 2.85220684365826E-02 -1.86686447888499E+00 2.02204090773511E+00 STEPO, YGPSDT[clipped]( 1,...), Grid Point Norms FIELD LEV AVERAGE MINIMUM MAXIMUM 101 60 2.58943581585986E-02 -1.00000000000000E+00 1.00000000000000E+00 ------------------------------------------------ CNT0 not found; string returned was: 'STEPO' >>> Printing last 8 lines from file: ifs.stat 15:01:21 0AAA00AAA STEPO 2919 11.885 11.885 11.905 690:27 691:13 0.12143555920906E-04 2GB 0MB 15:01:40 0AAA00AAA STEPO 2920 18.342 18.342 18.354 690:45 691:31 0.12143555920906E-04 2GB 0MB 15:01:52 0AAA00AAA STEPO 2921 11.889 11.889 11.895 690:57 691:43 0.12143555920906E-04 2GB 0MB 15:02:03 0AAA00AAA STEPO 2922 11.593 11.593 11.606 691:09 691:55 0.12143555920906E-04 2GB 0MB 15:02:22 0AAA00AAA STEPO 2923 18.648 18.648 18.666 691:27 692:14 0.12143555920906E-04 2GB 0MB 15:02:33 0AAA00AAA STEPO 2924 11.509 11.509 11.525 691:39 692:25 0.12143555920906E-04 2GB 0MB 15:02:45 0AAA00AAA STEPO 2925 11.863 11.863 11.878 691:51 692:37 0.12143555920906E-04 2GB 0MB ------------------------------------------------ >>> Printing last 11 lines from file: rcf CSTEP = " 2904", CTIME = "01210000 ", NSTEPLPP = 201, 202, 49, 228226, 228227, 50, IPRGPNSRES = 1, IPRGPEWRES = 1, IPRTRWRES = 1, IPRTRVRES = 1, GMASS0 = 98321.1813126105 , GMASSI = 98321.0757161544 / ------------------------------------------------ >>> Printing last 17 lines from file: waminfo PARAM=10U/10V, REPRES=GG, ANALYSIS PERIOD = 0 FORECAST PERIOD = 176400 WIND TIME STEP IN SECONDS = 3600 ANALYSIS FROM 20191101000000 TO 20191101000000 ASSIMILATION NO NEW FORECAST NO STATUS STORM1= UNFINISHED STATUS STORM2= UNFINISHED STATUS STORM3= UNFINISHED DATE FOR OUTPUT OF BOTH RESTART FILES = 00000000000000 LAST DATE FOR SPECTRA FILE OUTPUT = 20200303000000 BEGIN DATE FOR USING SURFACE CURRENT = 20191101000000 PARALLEL RESTART = YES NUMBER OF MPI TASKS USED = 1 ------------------------------------------------ >>> Printing last 8 lines from file: /var/lib/boinc-client/slots/0/progress_file_12214294.xml <running_values> <last_cpu_time>41518.380000</last_cpu_time> <upload_file_number>121</upload_file_number> <last_iter>2925</last_iter> <last_upload>10454400</last_upload> <model_completed>0</model_completed> </running_values> ------------------------------------------------ ..Failed, model did not complete successfully (argv0) ../../projects/climateprediction.net/oifs_43r3_1.21_x86_64-pc-linux-gnu (argv1) start_date: 2019110100 (argv2) exptid: hnw4 (argv3) unique_member_id: 00ns (argv4) batchid: 993 (argv5) wuid: 12214294 (argv6) fclen: 123 (argv7) app_name: oifs_43r3 (argv8) nthreads: 1 Working directory is: /var/lib/boinc-client/slots/0 Project directory is: /var/lib/boinc-client/projects/climateprediction.net/ app name: oifs_43r3 version: 1.21 Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294 ..mkdir for temp folder for results failed Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_app_1.21_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/0/oifs_43r3_app_1.21_x86_64-pc-linux-gnu.zip Unzipping the app zip file: /var/lib/boinc-client/slots/0/oifs_43r3_app_1.21_x86_64-pc-linux-gnu.zip Copying the namelist files from: ../../projects/climateprediction.net/jf_d69ef3e780ea4f9d33f2a28aca66e431 to: /var/lib/boinc-client/slots/0/oifs_43r3_00ns_2019110100_123_993_12214294.zip Unzipping the namelist zip file: /var/lib/boinc-client/slots/0/oifs_43r3_00ns_2019110100_123_993_12214294.zip ic_ancil_file: ic_ancil_12214294 ifsdata_file: ifsdata_12214294 climate_data_file: clim_data_12214294 horiz_resolution: 159 vert_resolution: 60 grid_type: l_2 upload_interval: 24 utstep: 3600 nfrres: restart dump frequency (steps) 24 Copying IC ancils from: ../../projects/climateprediction.net/jf_57e4439c83e07464206981112fda6995 to: /var/lib/boinc-client/slots/0/ic_ancil_12214294.zip Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/0/ic_ancil_12214294.zip ..mkdir for ifsdata folder failed Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_1771bca796c894ca74754c2474bff371 to: /var/lib/boinc-client/slots/0/ifsdata/ifsdata_12214294.zip Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/0/ifsdata/ifsdata_12214294.zip ..mkdir for the climate data folder failed Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/0/159l_2/clim_data_12214294.zip Unzipping the climate data zip file: /var/lib/boinc-client/slots/0/159l_2/clim_data_12214294.zip Checking for progress XML file: /var/lib/boinc-client/slots/0/progress_file_12214294.xml Opened progress file ok : /var/lib/boinc-client/slots/0/progress_file_12214294.xml -- Model is restarting -- Adjusting last_iter, 2925, to previous model restart step. Creating progress file: /var/lib/boinc-client/slots/0/progress_file_12214294.xml last_cpu_time: 41518 upload_file_number: 121 last_iter: 2905 last_upload: 10454400 model_completed: 0 total_length_of_simulation: 10627200 result_base_name: oifs_43r3_00ns_2019110100_123_993_12214294_1_r627899484 The child process has been launched with process id: 3147026 Executing the command: /var/lib/boinc-client/slots/0/oifs_43r3_model.exe [EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1 [EC_DRHOOK:mhpfe3l002:1:1:3147026:3147026] [20230225:151604:1677334564:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2091] fp = 0x3124020 [EC_DRHOOK:mhpfe3l002:1:1:3147026:3147026] [20230225:151604:1677334564:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1 [EC_DRHOOK:mhpfe3l002:1:1:3147026:3147026] [20230225:151604:1677334564:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0) [EC_DRHOOK:mhpfe3l002:1:1:3147026:3147026] [20230225:151604:1677334564:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1 [EC_DRHOOK:mhpfe3l002:1:1:3147026:3147026] [20230225:151604:1677334564:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000 [EC_DRHOOK:mhpfe3l002:1:1:3147026:3147026] [20230225:151604:1677334564:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647) [EC_DRHOOK:mhpfe3l002:1:1:3147026:3147026] [20230225:151604:1677334564:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16 [EC_DRHOOK:mhpfe3l002:1:1:3147026:3147026] [20230225:151604:1677334564:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0 [EC_DRHOOK:mhpfe3l002:1:1:3147026:3147026] [20230225:151604:1677334564:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500 [EC_DRHOOK:mhpfe3l002:1:1:3147026:3147026] [20230225:151604:1677334564:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1 [EC_DRHOOK:mhpfe3l002:1:1:3147026:3147026] [20230225:151604:1677334564:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1 [EC_DRHOOK:mhpfe3l002:1:1:3147026:3147026] [20230225:151604:1677334564:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1 [EC_DRHOOK:mhpfe3l002:1:1:3147026:3147026] [20230225:151604:1677334564:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1 [EC_DRHOOK:mhpfe3l002:1:1:3147026:3147026] [20230225:151604:1677334564:0.000] [ignore_signals@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef> [EC_DRHOOK:mhpfe3l002:1:1:3147026:3147026] [20230225:151604:1677334564:0.000] [restore_default_signals@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef> [EC_DRHOOK:mhpfe3l002:1:1:3147026:3147026] [20230225:151604:1677334564:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x810730 (old at 0x1ce8440) [EC_DRHOOK:mhpfe3l002:1:1:3147026:3147026] [20230225:151604:1677334564:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x810730 (old at (nil)) [EC_DRHOOK:mhpfe3l002:1:1:3147026:3147026] [20230225:151604:1677334564:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x810730 (old at 0x1ce8440) [EC_DRHOOK:mhpfe3l002:1:1:3147026:3147026] [20230225:151604:1677334564:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x810730 (old at (nil)) [EC_DRHOOK:mhpfe3l002:1:1:3147026:3147026] [20230225:151604:1677334564:0.000] [trapfpe_treatment@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1 [EC_DRHOOK:mhpfe3l002:1:1:3147026:3147026] [20230225:151604:1677334564:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x810730 (old at 0x1ce8440) [EC_DRHOOK:mhpfe3l002:1:1:3147026:3147026] [20230225:151604:1677334564:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x810730 (old at 0x1ce8440) [EC_DRHOOK:mhpfe3l002:1:1:3147026:3147026] [20230225:151604:1677334564:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x810730 (old at (nil)) [EC_DRHOOK:mhpfe3l002:1:1:3147026:3147026] [20230225:151604:1677334564:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x810730 (old at 0x1ce8440) [EC_DRHOOK:mhpfe3l002:1:1:3147026:3147026] [20230225:151604:1677334564:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x810730 (old at 0x1ce8440) [EC_DRHOOK:mhpfe3l002:1:1:3147026:3147026] [20230225:151604:1677334564:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x810730 (old at 0x1ce8440) [EC_DRHOOK:mhpfe3l002:1:1:3147026:3147026] [20230225:151604:1677334564:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x810730 (old at (nil)) [EC_DRHOOK:mhpfe3l002:1:1:3147026:3147026] [20230225:151604:1677334564:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x810730 (old at (nil)) [EC_DRHOOK:mhpfe3l002:1:1:3147026:3147026] [20230225:151604:1677334564:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x810730 (old at (nil)) [EC_DRHOOK:mhpfe3l002:1:1:3147026:3147026] [20230225:151604:1677334564:0.000] [catch_signals@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef> 15:16:25 STEP 2904 H=2904:00 +CPU= 20.273 15:16:37 STEP 2905 H=2905:00 +CPU= 11.586 15:16:49 STEP 2906 H=2906:00 +CPU= 11.932 15:17:07 STEP 2907 H=2907:00 +CPU= 18.487 15:17:19 STEP 2908 H=2908:00 +CPU= 11.945 15:17:31 STEP 2909 H=2909:00 +CPU= 11.598 15:17:50 STEP 2910 H=2910:00 +CPU= 18.889 15:18:01 STEP 2911 H=2911:00 +CPU= 11.618 15:18:13 STEP 2912 H=2912:00 +CPU= 11.980 15:18:32 STEP 2913 H=2913:00 +CPU= 18.510 15:18:44 STEP 2914 H=2914:00 +CPU= 11.959 15:18:55 STEP 2915 H=2915:00 +CPU= 11.620 15:19:16 STEP 2916 H=2916:00 +CPU= 20.281 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhnw4+002916 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhnw4+002916 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhnw4+002916 15:19:27 STEP 2917 H=2917:00 +CPU= 11.615 15:19:39 STEP 2918 H=2918:00 +CPU= 11.974 15:19:58 STEP 2919 H=2919:00 +CPU= 18.496 15:20:10 STEP 2920 H=2920:00 +CPU= 11.982 15:20:21 STEP 2921 H=2921:00 +CPU= 11.626 15:20:40 STEP 2922 H=2922:00 +CPU= 18.941 15:20:52 STEP 2923 H=2923:00 +CPU= 11.622 15:21:04 STEP 2924 H=2924:00 +CPU= 11.957 15:21:22 STEP 2925 H=2925:00 +CPU= 18.430 15:21:34 STEP 2926 H=2926:00 +CPU= 11.962 15:21:46 STEP 2927 H=2927:00 +CPU= 11.961 15:22:08 STEP 2928 H=2928:00 +CPU= 21.332 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhnw4+002928 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhnw4+002928 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhnw4+002928 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMGGhnw4+002904 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMSHhnw4+002904 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMUAhnw4+002904 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMGGhnw4+002916 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMSHhnw4+002916 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMUAhnw4+002916 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_00ns_2019110100_123_993_12214294_1_r627899484_121.zip Uploading the intermediate file: upload_file_121.zip 15:22:19 STEP 2929 H=2929:00 +CPU= 11.622 15:22:31 STEP 2930 H=2930:00 +CPU= 11.972 15:22:50 STEP 2931 H=2931:00 +CPU= 18.415 15:23:02 STEP 2932 H=2932:00 +CPU= 11.966 15:23:13 STEP 2933 H=2933:00 +CPU= 11.608 15:23:32 STEP 2934 H=2934:00 +CPU= 18.840 15:23:44 STEP 2935 H=2935:00 +CPU= 11.584 15:23:56 STEP 2936 H=2936:00 +CPU= 11.986 15:24:14 STEP 2937 H=2937:00 +CPU= 18.464 15:24:26 STEP 2938 H=2938:00 +CPU= 11.979 15:24:38 STEP 2939 H=2939:00 +CPU= 11.629 15:24:58 STEP 2940 H=2940:00 +CPU= 20.112 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhnw4+002940 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhnw4+002940 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhnw4+002940 15:25:10 STEP 2941 H=2941:00 +CPU= 11.634 15:25:22 STEP 2942 H=2942:00 +CPU= 11.979 15:25:40 STEP 2943 H=2943:00 +CPU= 18.442 15:25:52 STEP 2944 H=2944:00 +CPU= 11.995 15:26:04 STEP 2945 H=2945:00 +CPU= 11.615 15:26:23 STEP 2946 H=2946:00 +CPU= 18.833 15:26:34 STEP 2947 H=2947:00 +CPU= 11.621 15:26:46 STEP 2948 H=2948:00 +CPU= 11.971 15:27:05 STEP 2949 H=2949:00 +CPU= 18.435 15:27:17 STEP 2950 H=2950:00 +CPU= 11.943 15:27:29 STEP 2951 H=2951:00 +CPU= 11.986 15:27:49 STEP 2952 H=2952:00 +CPU= 20.644 ..The child process terminated with status: 0 >>> Printing last 70 lines from file: NODE.001_01 SPECTRAL NORMS: Pattern 1 STEPO, YSPSDT_AR1 ( 1,...), SP NORMS 2D: 4.865283731608E-01 1.865286143845E-01 6.362598939121E-02 STEPO, YSPSDT_AR1 (SUM,...), SP NORMS 2D: 5.260318722649E-01 STEPO, YGPSDT ( 1,...), Grid Point Norms FIELD LEV AVERAGE MINIMUM MAXIMUM 101 60 3.48989191816039E-02 -1.72994898449055E+00 1.89761310057317E+00 STEPO, YGPSDT[clipped]( 1,...), Grid Point Norms FIELD LEV AVERAGE MINIMUM MAXIMUM 101 60 3.19925486346338E-02 -1.00000000000000E+00 1.00000000000000E+00 MAXGPFV : MAX. VALUE = 1.07666095238205 MAXGPFV : MAX. VALUE = 0.739988271053403 NSTEP = 2952 SCAN2M_HPOS P IO-STREAM SETUP - IOTYPE = 2 NUMIOPROCS = 1 CPATH = ICMGGhnw4+002952 MODE=a IO-STREAM CLOSED - ICMGGhnw4+002952 MPI-TASK: 1 - 344749914 BYTES IN 8 RECORDS TRANSFERRED IN 0.0004 SECONDS861874.7 Mbytes/s, TOTAL TIME= 0.0059( 6.8%) !!!!!!!!!!!!!! WAVE FIELDS INTEGRATED. DATE IS: 20200302231500 !!!!!!!!!!!! !!!!!!!!!!!!!! WAVE FIELDS INTEGRATED. DATE IS: 20200302233000 !!!!!!!!!!!! !!!!!!!!!!!!!! WAVE FIELDS INTEGRATED. DATE IS: 20200302234500 !!!!!!!!!!!! !!!!!!!!!!!!!! WAVE FIELDS INTEGRATED. DATE IS: 20200303000000 !!!!!!!!!!!! 15:27:49 STEP 2952 H=2952:00 +CPU= 20.644 END CNT3 NSTEP = 2952 CNT0 000000000 18.161 18.161 18.203 0.998 0 0 0 0 0 IO-STREAM STATISTICS, TOTAL NO OF REC/BYTES READ 35/ 804992476 WRITTEN 746/********* -TASK-OPENED-OPEN-RECS IN -KBYTE IN -RECS OUT-KBYTE OUT-WALL -WALL IN-WALL OU-TOT IN -TOT OUT 1 21 0 35 786125 746 ******** 2.5 0.6 1.9 0.8 2.7 ===-=== START OF TIMING STATISTICS ===-=== STATS FOR ALL TASKS NUM ROUTINE CALLS MEAN(ms) MAX(ms) FRAC(%) UNBAL(%) 0 CNT0 - COMPLETE EXECUTION 1 705863.7 705863.7 100.00 0.00 1 CNT4 - FORWARD INTEGRATION 1 704599.6 704599.6 99.82 0.00 8 SCAN2M - GRID-POINT DYNAMICS 53 983.5 983.5 7.38 0.00 9 SPCM - SPECTRAL COMP. 48 187.0 187.0 1.27 0.00 10 SCAN2M - PHYSICS 49 6183.3 6183.3 42.92 0.00 11 IOPACK - OUTPUT P.P. RESULTS 4 620.3 620.3 0.35 0.00 12 SPNORM - SPECTRAL NORM COMP. 3 9.0 9.0 0.00 0.00 13 SCAN2M - RADIATION CALC. 17 6731.9 6731.9 16.21 0.00 17 GRIDFPOS IN CNT4 4 21.0 21.0 0.01 0.00 27 RESTART FILES - WRITING 2 1240.5 1240.5 0.35 0.00 28 RESTART FILES - READING 1 1064.2 1064.2 0.15 0.00 29 SU4FPOS IN CNT4 4 0.0 0.0 0.00 0.00 30 DYNFPOS IN CNT4 4 1085.6 1085.6 0.62 0.00 37 CPGLAG - SL COMPUTATIONS 49 2073.5 2073.5 14.39 0.00 38 WAM - TOTAL COST OF WAVE MODEL 49 1538.3 1538.3 10.68 0.00 39 SU0YOMB 1 133.1 133.1 0.02 0.00 51 SCAN2M - SL COMM. PART 1 49 6.9 6.9 0.05 0.00 54 SPCM - M TO S/S TO M TRANSP. 48 162.3 162.3 1.10 0.00 55 SPCIMPF - S TO M/M TO S TRANSP. 48 21.8 21.8 0.15 0.00 56 SPNORM - SPECTRAL NORM COMM. 3 0.1 0.1 0.00 0.00 102 LTINV_CTL - INVERSE LEGENDRE TRANSFORM 170 75.8 75.8 1.83 0.00 103 LTDIR_CTL - DIRECT LEGENDRE TRANSFORM 100 78.4 78.4 1.11 0.00 106 FTDIR_CTL - DIRECT FOURIER TRANSFORM 100 25.9 25.9 0.37 0.00 107 FTINV_CTL - INVERSE FOURIER TRANSFORM 170 28.4 28.4 0.68 0.00 140 SULEG - COMP. OF LEGENDRE POL. 2 10.8 10.8 0.00 0.00 152 LTINV_CTL - M TO L TRANSPOSITION 170 6.4 6.4 0.15 0.00 153 LTDIR_CTL - L TO M TRANSPOSITION 100 6.4 6.4 0.09 0.00 157 FTINV_CTL - L TO G TRANSPOSITION 170 9.9 9.9 0.24 0.00 158 FTDIR_CTL - G TO L TRANSPOSITION 100 6.7 6.7 0.10 0.00 400 GSTATS 10800 0.0 0.0 0.00 0.00 401 GSTATS HOOK 10275 0.0 0.0 0.00 0.00 TOTAL MEASURED IMBALANCE = 0.0 SECONDS, 0.0 PERCENT TOTAL WALLCLOCK TIME 705.9 CPU TIME 705.1 VECTOR TIME 705.1 ===-=== END OF TIMING STATISTICS ===-=== FORECAST DAYS PER DAY 15082.6 * END CNT0 *** ------------------------------------------------ Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhnw4+002952 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhnw4+002952 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhnw4+002952 Adding to the zip: /var/lib/boinc-client/slots/0/NODE.001_01 Adding to the zip: /var/lib/boinc-client/slots/0/ifs.stat Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMGGhnw4+002928 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMGGhnw4+002940 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMUAhnw4+002940 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMUAhnw4+002952 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMGGhnw4+002952 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMSHhnw4+002928 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMSHhnw4+002952 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMUAhnw4+002928 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_12214294/ICMSHhnw4+002940 Zipping up the final file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_00ns_2019110100_123_993_12214294_1_r627899484_122.zip Uploading the final file: upload_file_122.zip Uploading trickle at timestep: 10623600 15:31:11 (3147013): called boinc_finish(0) </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
09 May 2023 16:53:42	1523950	22316905	oifs_43r3_00ns_2019110100_123_993_12214294_1	10,623,600	42,223	0.0040
09 May 2023 16:53:42	1523950	22316905	oifs_43r3_00ns_2019110100_123_993_12214294_1	10,368,000	40,896	0.0039
09 May 2023 16:53:40	1523950	22316905	oifs_43r3_00ns_2019110100_123_993_12214294_1	9,504,000	37,505	0.0039
09 May 2023 16:53:37	1523950	22316905	oifs_43r3_00ns_2019110100_123_993_12214294_1	8,640,000	34,046	0.0039
09 May 2023 16:53:34	1523950	22316905	oifs_43r3_00ns_2019110100_123_993_12214294_1	7,776,000	30,695	0.0039
09 May 2023 16:53:31	1523950	22316905	oifs_43r3_00ns_2019110100_123_993_12214294_1	6,912,000	27,334	0.0040
09 May 2023 16:53:28	1523950	22316905	oifs_43r3_00ns_2019110100_123_993_12214294_1	6,048,000	23,963	0.0040
09 May 2023 16:53:25	1523950	22316905	oifs_43r3_00ns_2019110100_123_993_12214294_1	5,184,000	20,573	0.0040
09 May 2023 16:53:21	1523950	22316905	oifs_43r3_00ns_2019110100_123_993_12214294_1	4,320,000	17,172	0.0040
09 May 2023 16:53:18	1523950	22316905	oifs_43r3_00ns_2019110100_123_993_12214294_1	3,456,000	13,751	0.0040
09 May 2023 16:53:15	1523950	22316905	oifs_43r3_00ns_2019110100_123_993_12214294_1	2,592,000	10,330	0.0040
09 May 2023 16:53:11	1523950	22316905	oifs_43r3_00ns_2019110100_123_993_12214294_1	1,728,000	6,889	0.0040
09 May 2023 16:53:08	1523950	22316905	oifs_43r3_00ns_2019110100_123_993_12214294_1	864,000	3,449	0.0040