climateprediction.net home page
Task 22302736

Task 22302736

Name oifs_43r3_ps_0634_2017050100_123_986_12203278_2
Workunit 12203278
Created 24 Jan 2023, 6:05:11 UTC
Sent 24 Jan 2023, 6:12:26 UTC
Report deadline 25 Mar 2023, 6:12:26 UTC
Received 26 Jan 2023, 1:50:29 UTC
Server state Over
Outcome Success
Client state Done
Exit status 0 (0x00000000)
Computer ID 1532331
Run time 23 hours 54 min 47 sec
CPU time 23 hours 45 min 36 sec
Validate state Valid
Credit 14,873.04
Device peak FLOPS 7.10 GFLOPS
Application version OpenIFS 43r3 Perturbed Surface v1.05
x86_64-pc-linux-gnu
Peak working set size 4,419.27 MB
Peak swap size 4,978.07 MB
Peak disk usage 6,110.03 MB
Stderr
<core_client_version>7.16.6</core_client_version>
<![CDATA[
<stderr_txt>
.069
  12:31:55 STEP 2532 H=2532:00 +CPU= 58.845
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMGGhq0f+002532
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMSHhq0f+002532
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMUAhq0f+002532
  12:32:33 STEP 2533 H=2533:00 +CPU= 37.631
  12:33:12 STEP 2534 H=2534:00 +CPU= 38.565
  12:34:07 STEP 2535 H=2535:00 +CPU= 54.797
  12:34:46 STEP 2536 H=2536:00 +CPU= 38.864
  12:35:22 STEP 2537 H=2537:00 +CPU= 36.080
  12:36:21 STEP 2538 H=2538:00 +CPU= 58.271
  12:36:58 STEP 2539 H=2539:00 +CPU= 37.506
  12:37:36 STEP 2540 H=2540:00 +CPU= 37.644
  12:38:33 STEP 2541 H=2541:00 +CPU= 56.339
  12:39:12 STEP 2542 H=2542:00 +CPU= 39.287
  12:39:51 STEP 2543 H=2543:00 +CPU= 38.349
  12:40:55 STEP 2544 H=2544:00 +CPU= 64.167
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMGGhq0f+002544
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMSHhq0f+002544
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMUAhq0f+002544
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMGGhq0f+002520
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMSHhq0f+002520
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMUAhq0f+002520
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMGGhq0f+002532
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMSHhq0f+002532
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMUAhq0f+002532
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0634_2017050100_123_986_12203278_2_r1541080119_105.zip
Uploading the intermediate file: upload_file_105.zip
  12:41:33 STEP 2545 H=2545:00 +CPU= 37.697
  12:42:12 STEP 2546 H=2546:00 +CPU= 38.349
  12:43:06 STEP 2547 H=2547:00 +CPU= 54.310
  12:43:44 STEP 2548 H=2548:00 +CPU= 37.211
  12:44:19 STEP 2549 H=2549:00 +CPU= 35.236
  12:45:15 STEP 2550 H=2550:00 +CPU= 55.558
  12:45:51 STEP 2551 H=2551:00 +CPU= 35.908
  12:46:29 STEP 2552 H=2552:00 +CPU= 38.126
  12:47:24 STEP 2553 H=2553:00 +CPU= 54.762
  12:48:04 STEP 2554 H=2554:00 +CPU= 39.876
  12:48:42 STEP 2555 H=2555:00 +CPU= 37.129
  12:49:43 STEP 2556 H=2556:00 +CPU= 61.443
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMGGhq0f+002556
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMSHhq0f+002556
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMUAhq0f+002556
  12:50:22 STEP 2557 H=2557:00 +CPU= 38.594
  12:51:02 STEP 2558 H=2558:00 +CPU= 39.740
  12:51:56 STEP 2559 H=2559:00 +CPU= 53.698
  12:52:36 STEP 2560 H=2560:00 +CPU= 39.478
  12:53:12 STEP 2561 H=2561:00 +CPU= 36.244
  12:54:12 STEP 2562 H=2562:00 +CPU= 58.874
  12:54:51 STEP 2563 H=2563:00 +CPU= 38.642
  12:55:28 STEP 2564 H=2564:00 +CPU= 37.514
  12:56:23 STEP 2565 H=2565:00 +CPU= 54.952
  12:57:02 STEP 2566 H=2566:00 +CPU= 38.392
  12:57:39 STEP 2567 H=2567:00 +CPU= 36.679
  12:58:48 STEP 2568 H=2568:00 +CPU= 68.219
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMGGhq0f+002568
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMSHhq0f+002568
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMUAhq0f+002568
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMGGhq0f+002544
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMSHhq0f+002544
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMUAhq0f+002544
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMGGhq0f+002556
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMSHhq0f+002556
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMUAhq0f+002556
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0634_2017050100_123_986_12203278_2_r1541080119_106.zip
Uploading the intermediate file: upload_file_106.zip
  12:59:27 STEP 2569 H=2569:00 +CPU= 38.557
  13:00:06 STEP 2570 H=2570:00 +CPU= 38.849
  13:01:02 STEP 2571 H=2571:00 +CPU= 56.417
  13:01:40 STEP 2572 H=2572:00 +CPU= 37.493
  13:02:17 STEP 2573 H=2573:00 +CPU= 36.510
  13:03:15 STEP 2574 H=2574:00 +CPU= 57.999
  13:03:52 STEP 2575 H=2575:00 +CPU= 36.899
  13:04:32 STEP 2576 H=2576:00 +CPU= 39.630
  13:05:30 STEP 2577 H=2577:00 +CPU= 58.505
  13:06:10 STEP 2578 H=2578:00 +CPU= 39.276
  13:06:49 STEP 2579 H=2579:00 +CPU= 38.584
  13:07:53 STEP 2580 H=2580:00 +CPU= 63.609
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMGGhq0f+002580
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMSHhq0f+002580
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMUAhq0f+002580
  13:08:27 STEP 2581 H=2581:00 +CPU= 34.399
  13:09:07 STEP 2582 H=2582:00 +CPU= 39.571
  13:10:04 STEP 2583 H=2583:00 +CPU= 56.808
  13:10:42 STEP 2584 H=2584:00 +CPU= 37.356
  13:11:19 STEP 2585 H=2585:00 +CPU= 36.970
  13:12:16 STEP 2586 H=2586:00 +CPU= 57.417
  13:12:55 STEP 2587 H=2587:00 +CPU= 38.297
  13:13:37 STEP 2588 H=2588:00 +CPU= 41.996
  13:14:33 STEP 2589 H=2589:00 +CPU= 56.087
  13:15:13 STEP 2590 H=2590:00 +CPU= 39.263
  13:15:52 STEP 2591 H=2591:00 +CPU= 39.514
  13:16:58 STEP 2592 H=2592:00 +CPU= 65.236
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMGGhq0f+002592
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMSHhq0f+002592
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMUAhq0f+002592
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMGGhq0f+002568
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMSHhq0f+002568
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMUAhq0f+002568
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMGGhq0f+002580
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMSHhq0f+002580
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMUAhq0f+002580
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0634_2017050100_123_986_12203278_2_r1541080119_107.zip
Uploading the intermediate file: upload_file_107.zip
  13:17:36 STEP 2593 H=2593:00 +CPU= 37.760
  13:18:13 STEP 2594 H=2594:00 +CPU= 36.982
  13:19:10 STEP 2595 H=2595:00 +CPU= 56.469
  13:19:50 STEP 2596 H=2596:00 +CPU= 40.249
  13:20:31 STEP 2597 H=2597:00 +CPU= 40.976
  13:21:32 STEP 2598 H=2598:00 +CPU= 60.766
  13:22:12 STEP 2599 H=2599:00 +CPU= 39.803
  13:22:54 STEP 2600 H=2600:00 +CPU= 41.180
  13:23:50 STEP 2601 H=2601:00 +CPU= 55.877
  13:24:27 STEP 2602 H=2602:00 +CPU= 36.821
  13:25:08 STEP 2603 H=2603:00 +CPU= 40.099
  13:26:12 STEP 2604 H=2604:00 +CPU= 63.689
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMGGhq0f+002604
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMSHhq0f+002604
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMUAhq0f+002604
  13:26:50 STEP 2605 H=2605:00 +CPU= 37.724
  13:27:29 STEP 2606 H=2606:00 +CPU= 39.193
  13:28:25 STEP 2607 H=2607:00 +CPU= 54.823
  13:29:01 STEP 2608 H=2608:00 +CPU= 35.736
  13:29:37 STEP 2609 H=2609:00 +CPU= 36.413
  13:30:31 STEP 2610 H=2610:00 +CPU= 53.235
  13:31:09 STEP 2611 H=2611:00 +CPU= 37.950
  13:31:47 STEP 2612 H=2612:00 +CPU= 37.983
  13:32:44 STEP 2613 H=2613:00 +CPU= 54.202
  13:33:23 STEP 2614 H=2614:00 +CPU= 38.689
  13:34:03 STEP 2615 H=2615:00 +CPU= 39.885
  13:35:11 STEP 2616 H=2616:00 +CPU= 67.420
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMGGhq0f+002616
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMSHhq0f+002616
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMUAhq0f+002616
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMGGhq0f+002592
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMSHhq0f+002592
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMUAhq0f+002592
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMGGhq0f+002604
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMSHhq0f+002604
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMUAhq0f+002604
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0634_2017050100_123_986_12203278_2_r1541080119_108.zip
Uploading the intermediate file: upload_file_108.zip
  13:35:49 STEP 2617 H=2617:00 +CPU= 38.009
  13:36:29 STEP 2618 H=2618:00 +CPU= 39.118
  13:37:25 STEP 2619 H=2619:00 +CPU= 56.216
  13:38:03 STEP 2620 H=2620:00 +CPU= 37.666
  13:38:41 STEP 2621 H=2621:00 +CPU= 37.485
  13:39:37 STEP 2622 H=2622:00 +CPU= 54.770
  13:40:16 STEP 2623 H=2623:00 +CPU= 38.388
  13:40:56 STEP 2624 H=2624:00 +CPU= 40.012
  13:41:50 STEP 2625 H=2625:00 +CPU= 53.760
  13:42:29 STEP 2626 H=2626:00 +CPU= 39.264
  13:43:06 STEP 2627 H=2627:00 +CPU= 36.688
  13:44:08 STEP 2628 H=2628:00 +CPU= 61.669
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMGGhq0f+002628
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMSHhq0f+002628
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMUAhq0f+002628
  13:44:46 STEP 2629 H=2629:00 +CPU= 38.038
  13:45:27 STEP 2630 H=2630:00 +CPU= 40.942
  13:46:25 STEP 2631 H=2631:00 +CPU= 56.730
  13:47:03 STEP 2632 H=2632:00 +CPU= 37.879
  13:47:39 STEP 2633 H=2633:00 +CPU= 35.929
  13:48:36 STEP 2634 H=2634:00 +CPU= 56.693
  13:49:13 STEP 2635 H=2635:00 +CPU= 37.537
  13:49:50 STEP 2636 H=2636:00 +CPU= 36.249
  13:50:45 STEP 2637 H=2637:00 +CPU= 55.104
  13:51:25 STEP 2638 H=2638:00 +CPU= 39.297
  13:52:02 STEP 2639 H=2639:00 +CPU= 37.531
  13:53:10 STEP 2640 H=2640:00 +CPU= 67.777
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMGGhq0f+002640
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMSHhq0f+002640
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMUAhq0f+002640
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMGGhq0f+002616
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMSHhq0f+002616
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMUAhq0f+002616
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMGGhq0f+002628
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMSHhq0f+002628
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMUAhq0f+002628
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0634_2017050100_123_986_12203278_2_r1541080119_109.zip
Uploading the intermediate file: upload_file_109.zip
Uploading trickle at timestep: 9504000
  13:53:48 STEP 2641 H=2641:00 +CPU= 37.004
  13:54:26 STEP 2642 H=2642:00 +CPU= 38.028
  13:55:25 STEP 2643 H=2643:00 +CPU= 58.636
  13:56:06 STEP 2644 H=2644:00 +CPU= 40.640
  13:56:43 STEP 2645 H=2645:00 +CPU= 36.488
  13:57:41 STEP 2646 H=2646:00 +CPU= 57.593
  13:58:19 STEP 2647 H=2647:00 +CPU= 38.239
  13:58:59 STEP 2648 H=2648:00 +CPU= 39.680
  13:59:59 STEP 2649 H=2649:00 +CPU= 59.626
  14:00:40 STEP 2650 H=2650:00 +CPU= 40.174
  14:01:17 STEP 2651 H=2651:00 +CPU= 36.772
  14:02:16 STEP 2652 H=2652:00 +CPU= 59.437
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMGGhq0f+002652
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMSHhq0f+002652
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMUAhq0f+002652
  14:02:55 STEP 2653 H=2653:00 +CPU= 37.972
  14:03:33 STEP 2654 H=2654:00 +CPU= 38.127
  14:04:32 STEP 2655 H=2655:00 +CPU= 58.442
  14:05:10 STEP 2656 H=2656:00 +CPU= 38.042
  14:05:47 STEP 2657 H=2657:00 +CPU= 36.518
  14:06:45 STEP 2658 H=2658:00 +CPU= 57.758
  14:07:25 STEP 2659 H=2659:00 +CPU= 39.839
  14:08:05 STEP 2660 H=2660:00 +CPU= 39.351
  14:09:05 STEP 2661 H=2661:00 +CPU= 59.777
  14:09:46 STEP 2662 H=2662:00 +CPU= 41.073
  14:10:24 STEP 2663 H=2663:00 +CPU= 37.776
  14:11:28 STEP 2664 H=2664:00 +CPU= 63.525
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMGGhq0f+002664
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMSHhq0f+002664
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMUAhq0f+002664
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMGGhq0f+002640
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMSHhq0f+002640
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMUAhq0f+002640
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMGGhq0f+002652
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMSHhq0f+002652
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMUAhq0f+002652
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0634_2017050100_123_986_12203278_2_r1541080119_110.zip
Uploading the intermediate file: upload_file_110.zip
  14:12:05 STEP 2665 H=2665:00 +CPU= 36.973
  14:12:41 STEP 2666 H=2666:00 +CPU= 36.208
  14:13:38 STEP 2667 H=2667:00 +CPU= 56.345
  14:14:18 STEP 2668 H=2668:00 +CPU= 39.742
  14:15:01 STEP 2669 H=2669:00 +CPU= 42.606
  14:16:01 STEP 2670 H=2670:00 +CPU= 60.192
  14:16:42 STEP 2671 H=2671:00 +CPU= 40.473
  14:17:22 STEP 2672 H=2672:00 +CPU= 40.241
  14:18:21 STEP 2673 H=2673:00 +CPU= 58.472
  14:19:00 STEP 2674 H=2674:00 +CPU= 39.404
  14:19:39 STEP 2675 H=2675:00 +CPU= 38.130
  14:20:44 STEP 2676 H=2676:00 +CPU= 65.207
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMGGhq0f+002676
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMSHhq0f+002676
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMUAhq0f+002676
  14:21:23 STEP 2677 H=2677:00 +CPU= 38.001
  14:22:02 STEP 2678 H=2678:00 +CPU= 38.834
  14:23:00 STEP 2679 H=2679:00 +CPU= 58.024
  14:23:39 STEP 2680 H=2680:00 +CPU= 38.292
  14:24:16 STEP 2681 H=2681:00 +CPU= 37.294
  14:25:13 STEP 2682 H=2682:00 +CPU= 55.878
  14:25:51 STEP 2683 H=2683:00 +CPU= 38.238
  14:26:31 STEP 2684 H=2684:00 +CPU= 39.537
  14:27:27 STEP 2685 H=2685:00 +CPU= 56.085
  14:28:06 STEP 2686 H=2686:00 +CPU= 38.739
  14:28:47 STEP 2687 H=2687:00 +CPU= 40.237
  14:29:53 STEP 2688 H=2688:00 +CPU= 65.217
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMGGhq0f+002688
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMSHhq0f+002688
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMUAhq0f+002688
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMGGhq0f+002664
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMSHhq0f+002664
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMUAhq0f+002664
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMGGhq0f+002676
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMSHhq0f+002676
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMUAhq0f+002676
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0634_2017050100_123_986_12203278_2_r1541080119_111.zip
Uploading the intermediate file: upload_file_111.zip
  14:30:31 STEP 2689 H=2689:00 +CPU= 37.943
  14:31:12 STEP 2690 H=2690:00 +CPU= 41.340
  14:32:08 STEP 2691 H=2691:00 +CPU= 55.462
  14:32:48 STEP 2692 H=2692:00 +CPU= 39.271
  14:33:26 STEP 2693 H=2693:00 +CPU= 38.145
  14:34:22 STEP 2694 H=2694:00 +CPU= 56.132
  14:35:01 STEP 2695 H=2695:00 +CPU= 38.826
  14:35:41 STEP 2696 H=2696:00 +CPU= 39.024
  14:36:35 STEP 2697 H=2697:00 +CPU= 54.398
  14:37:15 STEP 2698 H=2698:00 +CPU= 39.250
  14:37:50 STEP 2699 H=2699:00 +CPU= 35.562
  14:38:51 STEP 2700 H=2700:00 +CPU= 60.641
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMGGhq0f+002700
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMSHhq0f+002700
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMUAhq0f+002700
  14:39:30 STEP 2701 H=2701:00 +CPU= 37.847
  14:40:09 STEP 2702 H=2702:00 +CPU= 39.216
  14:41:07 STEP 2703 H=2703:00 +CPU= 57.261
  14:41:46 STEP 2704 H=2704:00 +CPU= 38.542
  14:42:26 STEP 2705 H=2705:00 +CPU= 40.122
  14:43:26 STEP 2706 H=2706:00 +CPU= 59.403
  14:44:04 STEP 2707 H=2707:00 +CPU= 37.928
  14:44:45 STEP 2708 H=2708:00 +CPU= 40.725
  14:45:42 STEP 2709 H=2709:00 +CPU= 56.572
  14:46:22 STEP 2710 H=2710:00 +CPU= 39.372
  14:47:03 STEP 2711 H=2711:00 +CPU= 40.562
  14:48:10 STEP 2712 H=2712:00 +CPU= 66.586
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMGGhq0f+002712
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMSHhq0f+002712
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMUAhq0f+002712
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMGGhq0f+002688
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMSHhq0f+002688
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMUAhq0f+002688
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMGGhq0f+002700
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMSHhq0f+002700
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMUAhq0f+002700
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0634_2017050100_123_986_12203278_2_r1541080119_112.zip
Uploading the intermediate file: upload_file_112.zip
  14:48:49 STEP 2713 H=2713:00 +CPU= 39.199
  14:49:31 STEP 2714 H=2714:00 +CPU= 40.923
  14:50:30 STEP 2715 H=2715:00 +CPU= 58.327
  14:51:07 STEP 2716 H=2716:00 +CPU= 37.495
  14:51:45 STEP 2717 H=2717:00 +CPU= 37.416
  14:52:44 STEP 2718 H=2718:00 +CPU= 58.860
  14:53:25 STEP 2719 H=2719:00 +CPU= 40.733
  14:54:03 STEP 2720 H=2720:00 +CPU= 38.164
  14:54:58 STEP 2721 H=2721:00 +CPU= 54.695
  14:55:39 STEP 2722 H=2722:00 +CPU= 40.155
  14:56:15 STEP 2723 H=2723:00 +CPU= 36.039
  14:57:14 STEP 2724 H=2724:00 +CPU= 58.905
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMGGhq0f+002724
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMSHhq0f+002724
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMUAhq0f+002724
  14:57:51 STEP 2725 H=2725:00 +CPU= 36.304
  14:58:29 STEP 2726 H=2726:00 +CPU= 37.712
  14:59:24 STEP 2727 H=2727:00 +CPU= 54.912
  15:00:01 STEP 2728 H=2728:00 +CPU= 37.156
  15:00:38 STEP 2729 H=2729:00 +CPU= 36.919
  15:01:36 STEP 2730 H=2730:00 +CPU= 57.575
  15:02:17 STEP 2731 H=2731:00 +CPU= 40.350
  15:02:56 STEP 2732 H=2732:00 +CPU= 38.922
  15:03:54 STEP 2733 H=2733:00 +CPU= 58.453
  15:04:36 STEP 2734 H=2734:00 +CPU= 41.464
  15:05:14 STEP 2735 H=2735:00 +CPU= 37.711
  15:06:23 STEP 2736 H=2736:00 +CPU= 68.382
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMGGhq0f+002736
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMSHhq0f+002736
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMUAhq0f+002736
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMGGhq0f+002712
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMSHhq0f+002712
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMUAhq0f+002712
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMGGhq0f+002724
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMSHhq0f+002724
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMUAhq0f+002724
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0634_2017050100_123_986_12203278_2_r1541080119_113.zip
Uploading the intermediate file: upload_file_113.zip
  15:06:59 STEP 2737 H=2737:00 +CPU= 35.785
  15:07:41 STEP 2738 H=2738:00 +CPU= 41.729
  15:08:39 STEP 2739 H=2739:00 +CPU= 56.722
  15:09:17 STEP 2740 H=2740:00 +CPU= 37.785
  15:09:56 STEP 2741 H=2741:00 +CPU= 39.200
  15:10:56 STEP 2742 H=2742:00 +CPU= 58.080
  15:11:34 STEP 2743 H=2743:00 +CPU= 37.545
  15:12:13 STEP 2744 H=2744:00 +CPU= 38.569
  15:13:11 STEP 2745 H=2745:00 +CPU= 57.704
  15:13:53 STEP 2746 H=2746:00 +CPU= 41.180
  15:14:32 STEP 2747 H=2747:00 +CPU= 39.014
  15:15:32 STEP 2748 H=2748:00 +CPU= 60.221
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMGGhq0f+002748
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMSHhq0f+002748
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMUAhq0f+002748
  15:16:12 STEP 2749 H=2749:00 +CPU= 39.784
  15:16:55 STEP 2750 H=2750:00 +CPU= 42.091
  15:17:52 STEP 2751 H=2751:00 +CPU= 56.703
  15:18:33 STEP 2752 H=2752:00 +CPU= 41.150
  15:19:13 STEP 2753 H=2753:00 +CPU= 39.731
  15:20:12 STEP 2754 H=2754:00 +CPU= 59.271
  15:20:52 STEP 2755 H=2755:00 +CPU= 40.120
  15:21:33 STEP 2756 H=2756:00 +CPU= 40.650
  15:22:33 STEP 2757 H=2757:00 +CPU= 59.150
  15:23:11 STEP 2758 H=2758:00 +CPU= 37.477
  15:23:49 STEP 2759 H=2759:00 +CPU= 38.164
  15:24:59 STEP 2760 H=2760:00 +CPU= 69.072
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMGGhq0f+002760
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMSHhq0f+002760
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMUAhq0f+002760
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMGGhq0f+002736
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMSHhq0f+002736
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMUAhq0f+002736
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMGGhq0f+002748
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMSHhq0f+002748
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMUAhq0f+002748
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0634_2017050100_123_986_12203278_2_r1541080119_114.zip
Uploading the intermediate file: upload_file_114.zip
  15:25:36 STEP 2761 H=2761:00 +CPU= 37.115
  15:26:14 STEP 2762 H=2762:00 +CPU= 37.317
  15:27:11 STEP 2763 H=2763:00 +CPU= 56.415
  15:27:49 STEP 2764 H=2764:00 +CPU= 37.895
  15:28:29 STEP 2765 H=2765:00 +CPU= 40.136
  15:29:30 STEP 2766 H=2766:00 +CPU= 60.307
  15:30:07 STEP 2767 H=2767:00 +CPU= 36.190
  15:30:48 STEP 2768 H=2768:00 +CPU= 41.524
  15:31:45 STEP 2769 H=2769:00 +CPU= 56.174
  15:32:26 STEP 2770 H=2770:00 +CPU= 40.380
  15:33:03 STEP 2771 H=2771:00 +CPU= 37.144
  15:34:03 STEP 2772 H=2772:00 +CPU= 59.655
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMGGhq0f+002772
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMSHhq0f+002772
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMUAhq0f+002772
  15:34:42 STEP 2773 H=2773:00 +CPU= 38.608
  15:35:23 STEP 2774 H=2774:00 +CPU= 40.505
  15:36:20 STEP 2775 H=2775:00 +CPU= 57.089
  15:36:58 STEP 2776 H=2776:00 +CPU= 37.511
  15:37:38 STEP 2777 H=2777:00 +CPU= 40.197
  15:38:37 STEP 2778 H=2778:00 +CPU= 58.597
  15:39:17 STEP 2779 H=2779:00 +CPU= 40.003
  15:40:00 STEP 2780 H=2780:00 +CPU= 42.643
  15:40:55 STEP 2781 H=2781:00 +CPU= 54.606
  15:41:31 STEP 2782 H=2782:00 +CPU= 36.042
  15:42:11 STEP 2783 H=2783:00 +CPU= 40.078
  15:43:21 STEP 2784 H=2784:00 +CPU= 69.254
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMGGhq0f+002784
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMSHhq0f+002784
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMUAhq0f+002784
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMGGhq0f+002760
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMSHhq0f+002760
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMUAhq0f+002760
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMGGhq0f+002772
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMSHhq0f+002772
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMUAhq0f+002772
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0634_2017050100_123_986_12203278_2_r1541080119_115.zip
Uploading the intermediate file: upload_file_115.zip
  15:43:57 STEP 2785 H=2785:00 +CPU= 36.017
  15:44:36 STEP 2786 H=2786:00 +CPU= 38.557
  15:45:33 STEP 2787 H=2787:00 +CPU= 56.728
  15:46:12 STEP 2788 H=2788:00 +CPU= 39.009
  15:46:52 STEP 2789 H=2789:00 +CPU= 39.393
  15:47:52 STEP 2790 H=2790:00 +CPU= 59.948
  15:48:31 STEP 2791 H=2791:00 +CPU= 38.875
  15:49:13 STEP 2792 H=2792:00 +CPU= 41.523
  15:50:14 STEP 2793 H=2793:00 +CPU= 60.597
  15:50:53 STEP 2794 H=2794:00 +CPU= 38.965
  15:51:31 STEP 2795 H=2795:00 +CPU= 37.560
  15:52:37 STEP 2796 H=2796:00 +CPU= 65.341
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMGGhq0f+002796
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMSHhq0f+002796
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMUAhq0f+002796
  15:53:20 STEP 2797 H=2797:00 +CPU= 41.731
  15:53:58 STEP 2798 H=2798:00 +CPU= 38.445
  15:54:55 STEP 2799 H=2799:00 +CPU= 55.682
  15:55:37 STEP 2800 H=2800:00 +CPU= 41.251
  15:56:19 STEP 2801 H=2801:00 +CPU= 41.696
  15:57:24 STEP 2802 H=2802:00 +CPU= 64.279
  15:58:07 STEP 2803 H=2803:00 +CPU= 43.094
  15:58:49 STEP 2804 H=2804:00 +CPU= 41.983
  15:59:50 STEP 2805 H=2805:00 +CPU= 59.964
  16:00:30 STEP 2806 H=2806:00 +CPU= 40.026
  16:01:08 STEP 2807 H=2807:00 +CPU= 38.483
  16:02:19 STEP 2808 H=2808:00 +CPU= 69.817
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMGGhq0f+002808
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMSHhq0f+002808
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMUAhq0f+002808
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMGGhq0f+002784
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMSHhq0f+002784
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMUAhq0f+002784
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMGGhq0f+002796
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMSHhq0f+002796
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMUAhq0f+002796
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0634_2017050100_123_986_12203278_2_r1541080119_116.zip
Uploading the intermediate file: upload_file_116.zip
  16:02:57 STEP 2809 H=2809:00 +CPU= 38.330
  16:03:36 STEP 2810 H=2810:00 +CPU= 38.821
  16:04:34 STEP 2811 H=2811:00 +CPU= 58.218
  16:05:15 STEP 2812 H=2812:00 +CPU= 40.247
  16:05:53 STEP 2813 H=2813:00 +CPU= 37.756
  16:06:53 STEP 2814 H=2814:00 +CPU= 59.738
  16:07:32 STEP 2815 H=2815:00 +CPU= 38.394
  16:08:12 STEP 2816 H=2816:00 +CPU= 39.212
  16:09:08 STEP 2817 H=2817:00 +CPU= 56.163
  16:09:48 STEP 2818 H=2818:00 +CPU= 38.922
  16:10:29 STEP 2819 H=2819:00 +CPU= 41.224
  16:11:33 STEP 2820 H=2820:00 +CPU= 62.711
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMGGhq0f+002820
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMSHhq0f+002820
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMUAhq0f+002820
  16:12:11 STEP 2821 H=2821:00 +CPU= 37.711
  16:12:50 STEP 2822 H=2822:00 +CPU= 39.190
  16:13:46 STEP 2823 H=2823:00 +CPU= 55.782
  16:14:26 STEP 2824 H=2824:00 +CPU= 39.557
  16:15:05 STEP 2825 H=2825:00 +CPU= 38.624
  16:16:04 STEP 2826 H=2826:00 +CPU= 58.939
  16:16:45 STEP 2827 H=2827:00 +CPU= 40.702
  16:17:25 STEP 2828 H=2828:00 +CPU= 39.885
  16:18:24 STEP 2829 H=2829:00 +CPU= 58.443
  16:19:04 STEP 2830 H=2830:00 +CPU= 40.696
  16:19:44 STEP 2831 H=2831:00 +CPU= 39.034
  16:20:49 STEP 2832 H=2832:00 +CPU= 64.834
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMGGhq0f+002832
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMSHhq0f+002832
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMUAhq0f+002832
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMGGhq0f+002808
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMSHhq0f+002808
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMUAhq0f+002808
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMGGhq0f+002820
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMSHhq0f+002820
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMUAhq0f+002820
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0634_2017050100_123_986_12203278_2_r1541080119_117.zip
Uploading the intermediate file: upload_file_117.zip
  16:21:28 STEP 2833 H=2833:00 +CPU= 38.554
  16:22:06 STEP 2834 H=2834:00 +CPU= 37.805
  16:23:03 STEP 2835 H=2835:00 +CPU= 57.279
Quit request received from BOINC client, ending the child process
..Failed, model did not complete successfully
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 2017050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0634
(argv4) batchid: 986
(argv5) wuid: 12203278
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/15
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/15/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/15/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_8cdaa693c357aa4f821843e02d308cd4 to: /var/lib/boinc-client/slots/15/oifs_43r3_ps_0634_2017050100_123_986_12203278.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/15/oifs_43r3_ps_0634_2017050100_123_986_12203278.zip
ic_ancil_file: ic_ancil_12203278
ifsdata_file: ifsdata_12203278
climate_data_file: clim_data_12203278
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_84a18eb2951c08c62807224e96149cda to: /var/lib/boinc-client/slots/15/ic_ancil_12203278.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/15/ic_ancil_12203278.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_17ec24cb117427d793d954555d39030a to: /var/lib/boinc-client/slots/15/ifsdata/ifsdata_12203278.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/15/ifsdata/ifsdata_12203278.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/15/159l_2/clim_data_12203278.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/15/159l_2/clim_data_12203278.zip
Checking for progress XML file: /var/lib/boinc-client/slots/15/progress_file_12203278.xml
Opened progress file ok : /var/lib/boinc-client/slots/15/progress_file_12203278.xml
-- Model is restarting --
Adjusting last_iter, 2836, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/15/progress_file_12203278.xml
last_cpu_time: 79557
upload_file_number: 118
last_iter: 2831
last_upload: 10195200
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0634_2017050100_123_986_12203278_2_r1541080119
The child process has been launched with process id: 1540216
Executing the command: /var/lib/boinc-client/slots/15/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:Serveur-Master:1:1:1540216:1540216] [20230125:170223:1674666143:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:Serveur-Master:1:1:1540216:1540216] [20230125:170223:1674666143:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:Serveur-Master:1:1:1540216:1540216] [20230125:170223:1674666143:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:Serveur-Master:1:1:1540216:1540216] [20230125:170223:1674666143:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:Serveur-Master:1:1:1540216:1540216] [20230125:170223:1674666143:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:Serveur-Master:1:1:1540216:1540216] [20230125:170223:1674666143:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:Serveur-Master:1:1:1540216:1540216] [20230125:170223:1674666143:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:Serveur-Master:1:1:1540216:1540216] [20230125:170223:1674666143:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:Serveur-Master:1:1:1540216:1540216] [20230125:170223:1674666143:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:Serveur-Master:1:1:1540216:1540216] [20230125:170223:1674666143:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:Serveur-Master:1:1:1540216:1540216] [20230125:170223:1674666143:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:Serveur-Master:1:1:1540216:1540216] [20230125:170223:1674666143:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:Serveur-Master:1:1:1540216:1540216] [20230125:170223:1674666143:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:Serveur-Master:1:1:1540216:1540216] [20230125:170223:1674666143:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:Serveur-Master:1:1:1540216:1540216] [20230125:170223:1674666143:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:Serveur-Master:1:1:1540216:1540216] [20230125:170223:1674666143:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Serveur-Master:1:1:1540216:1540216] [20230125:170223:1674666143:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:Serveur-Master:1:1:1540216:1540216] [20230125:170223:1674666143:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Serveur-Master:1:1:1540216:1540216] [20230125:170223:1674666143:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:Serveur-Master:1:1:1540216:1540216] [20230125:170223:1674666143:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:Serveur-Master:1:1:1540216:1540216] [20230125:170223:1674666143:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Serveur-Master:1:1:1540216:1540216] [20230125:170223:1674666143:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Serveur-Master:1:1:1540216:1540216] [20230125:170223:1674666143:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:Serveur-Master:1:1:1540216:1540216] [20230125:170223:1674666143:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Serveur-Master:1:1:1540216:1540216] [20230125:170223:1674666143:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Serveur-Master:1:1:1540216:1540216] [20230125:170223:1674666143:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Serveur-Master:1:1:1540216:1540216] [20230125:170223:1674666143:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:Serveur-Master:1:1:1540216:1540216] [20230125:170223:1674666143:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:Serveur-Master:1:1:1540216:1540216] [20230125:170223:1674666143:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:Serveur-Master:1:1:1540216:1540216] [20230125:170223:1674666143:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  17:03:29 STEP 2832 H=2832:00 +CPU= 61.260
  17:04:07 STEP 2833 H=2833:00 +CPU= 37.424
  17:04:45 STEP 2834 H=2834:00 +CPU= 37.857
  17:05:38 STEP 2835 H=2835:00 +CPU= 53.123
  17:06:14 STEP 2836 H=2836:00 +CPU= 35.489
  17:06:52 STEP 2837 H=2837:00 +CPU= 37.995
  17:07:48 STEP 2838 H=2838:00 +CPU= 55.499
  17:08:25 STEP 2839 H=2839:00 +CPU= 37.342
  17:09:05 STEP 2840 H=2840:00 +CPU= 39.228
  17:10:01 STEP 2841 H=2841:00 +CPU= 55.296
  17:10:40 STEP 2842 H=2842:00 +CPU= 38.862
  17:11:15 STEP 2843 H=2843:00 +CPU= 35.218
  17:12:16 STEP 2844 H=2844:00 +CPU= 60.742
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMGGhq0f+002844
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMSHhq0f+002844
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMUAhq0f+002844
  17:12:53 STEP 2845 H=2845:00 +CPU= 37.035
  17:13:29 STEP 2846 H=2846:00 +CPU= 35.897
  17:14:27 STEP 2847 H=2847:00 +CPU= 57.296
  17:15:05 STEP 2848 H=2848:00 +CPU= 37.814
  17:15:44 STEP 2849 H=2849:00 +CPU= 38.318
  17:16:42 STEP 2850 H=2850:00 +CPU= 58.402
  17:17:17 STEP 2851 H=2851:00 +CPU= 34.703
  17:17:56 STEP 2852 H=2852:00 +CPU= 38.132
  17:18:51 STEP 2853 H=2853:00 +CPU= 54.762
  17:19:28 STEP 2854 H=2854:00 +CPU= 36.714
  17:20:02 STEP 2855 H=2855:00 +CPU= 34.611
  17:21:06 STEP 2856 H=2856:00 +CPU= 63.144
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMGGhq0f+002856
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMSHhq0f+002856
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMUAhq0f+002856
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMGGhq0f+002832
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMSHhq0f+002832
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMUAhq0f+002832
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMGGhq0f+002844
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMSHhq0f+002844
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMUAhq0f+002844
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0634_2017050100_123_986_12203278_2_r1541080119_118.zip
Uploading the intermediate file: upload_file_118.zip
  17:21:40 STEP 2857 H=2857:00 +CPU= 34.147
  17:22:14 STEP 2858 H=2858:00 +CPU= 33.930
  17:23:07 STEP 2859 H=2859:00 +CPU= 52.585
  17:23:46 STEP 2860 H=2860:00 +CPU= 38.472
  17:24:23 STEP 2861 H=2861:00 +CPU= 36.893
  17:25:16 STEP 2862 H=2862:00 +CPU= 52.624
  17:25:53 STEP 2863 H=2863:00 +CPU= 37.305
  17:26:31 STEP 2864 H=2864:00 +CPU= 37.917
  17:27:26 STEP 2865 H=2865:00 +CPU= 54.476
  17:28:02 STEP 2866 H=2866:00 +CPU= 36.058
  17:28:43 STEP 2867 H=2867:00 +CPU= 40.481
  17:29:43 STEP 2868 H=2868:00 +CPU= 60.317
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMGGhq0f+002868
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMSHhq0f+002868
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMUAhq0f+002868
  17:30:21 STEP 2869 H=2869:00 +CPU= 37.743
  17:30:59 STEP 2870 H=2870:00 +CPU= 37.391
  17:31:53 STEP 2871 H=2871:00 +CPU= 53.821
  17:32:30 STEP 2872 H=2872:00 +CPU= 37.110
  17:33:07 STEP 2873 H=2873:00 +CPU= 36.149
  17:34:02 STEP 2874 H=2874:00 +CPU= 55.175
  17:34:38 STEP 2875 H=2875:00 +CPU= 35.619
  17:35:15 STEP 2876 H=2876:00 +CPU= 37.236
  17:36:11 STEP 2877 H=2877:00 +CPU= 55.193
  17:36:46 STEP 2878 H=2878:00 +CPU= 35.665
  17:37:24 STEP 2879 H=2879:00 +CPU= 37.865
  17:38:31 STEP 2880 H=2880:00 +CPU= 66.374
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMGGhq0f+002880
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMSHhq0f+002880
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMUAhq0f+002880
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMGGhq0f+002856
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMSHhq0f+002856
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMUAhq0f+002856
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMGGhq0f+002868
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMSHhq0f+002868
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMUAhq0f+002868
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0634_2017050100_123_986_12203278_2_r1541080119_119.zip
Uploading the intermediate file: upload_file_119.zip
  17:39:08 STEP 2881 H=2881:00 +CPU= 36.411
  17:39:45 STEP 2882 H=2882:00 +CPU= 36.610
  17:40:39 STEP 2883 H=2883:00 +CPU= 54.350
  17:41:15 STEP 2884 H=2884:00 +CPU= 35.675
  17:41:50 STEP 2885 H=2885:00 +CPU= 34.679
  17:42:44 STEP 2886 H=2886:00 +CPU= 54.030
  17:43:19 STEP 2887 H=2887:00 +CPU= 34.879
  17:43:58 STEP 2888 H=2888:00 +CPU= 38.338
  17:44:53 STEP 2889 H=2889:00 +CPU= 55.634
  17:45:30 STEP 2890 H=2890:00 +CPU= 36.318
  17:46:05 STEP 2891 H=2891:00 +CPU= 35.189
  17:47:07 STEP 2892 H=2892:00 +CPU= 60.919
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMGGhq0f+002892
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMSHhq0f+002892
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMUAhq0f+002892
  17:47:42 STEP 2893 H=2893:00 +CPU= 35.436
  17:48:21 STEP 2894 H=2894:00 +CPU= 37.982
  17:49:14 STEP 2895 H=2895:00 +CPU= 53.154
  17:49:54 STEP 2896 H=2896:00 +CPU= 39.733
  17:50:31 STEP 2897 H=2897:00 +CPU= 37.169
  17:51:29 STEP 2898 H=2898:00 +CPU= 57.366
  17:52:06 STEP 2899 H=2899:00 +CPU= 36.090
  17:52:43 STEP 2900 H=2900:00 +CPU= 37.566
  17:53:37 STEP 2901 H=2901:00 +CPU= 53.493
  17:54:15 STEP 2902 H=2902:00 +CPU= 38.265
  17:54:52 STEP 2903 H=2903:00 +CPU= 36.331
  17:55:56 STEP 2904 H=2904:00 +CPU= 63.343
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMGGhq0f+002904
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMSHhq0f+002904
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMUAhq0f+002904
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMGGhq0f+002880
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMSHhq0f+002880
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMUAhq0f+002880
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMGGhq0f+002892
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMSHhq0f+002892
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMUAhq0f+002892
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0634_2017050100_123_986_12203278_2_r1541080119_120.zip
Uploading the intermediate file: upload_file_120.zip
  17:56:32 STEP 2905 H=2905:00 +CPU= 36.545
  17:57:10 STEP 2906 H=2906:00 +CPU= 37.101
  17:58:10 STEP 2907 H=2907:00 +CPU= 59.655
  17:58:51 STEP 2908 H=2908:00 +CPU= 41.479
  17:59:30 STEP 2909 H=2909:00 +CPU= 38.076
  18:00:26 STEP 2910 H=2910:00 +CPU= 55.960
  18:01:04 STEP 2911 H=2911:00 +CPU= 38.320
  18:01:44 STEP 2912 H=2912:00 +CPU= 39.257
  18:02:38 STEP 2913 H=2913:00 +CPU= 54.341
  18:03:17 STEP 2914 H=2914:00 +CPU= 38.406
  18:03:55 STEP 2915 H=2915:00 +CPU= 38.017
  18:04:58 STEP 2916 H=2916:00 +CPU= 62.837
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMGGhq0f+002916
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMSHhq0f+002916
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMUAhq0f+002916
  18:05:35 STEP 2917 H=2917:00 +CPU= 36.384
  18:06:13 STEP 2918 H=2918:00 +CPU= 38.155
  18:07:07 STEP 2919 H=2919:00 +CPU= 53.930
  18:07:44 STEP 2920 H=2920:00 +CPU= 36.588
  18:08:19 STEP 2921 H=2921:00 +CPU= 34.825
  18:09:16 STEP 2922 H=2922:00 +CPU= 56.079
Quit request received from BOINC client, ending the child process
..Failed, model did not complete successfully
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 2017050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0634
(argv4) batchid: 986
(argv5) wuid: 12203278
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/15
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/15/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/15/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_8cdaa693c357aa4f821843e02d308cd4 to: /var/lib/boinc-client/slots/15/oifs_43r3_ps_0634_2017050100_123_986_12203278.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/15/oifs_43r3_ps_0634_2017050100_123_986_12203278.zip
ic_ancil_file: ic_ancil_12203278
ifsdata_file: ifsdata_12203278
climate_data_file: clim_data_12203278
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_84a18eb2951c08c62807224e96149cda to: /var/lib/boinc-client/slots/15/ic_ancil_12203278.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/15/ic_ancil_12203278.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_17ec24cb117427d793d954555d39030a to: /var/lib/boinc-client/slots/15/ifsdata/ifsdata_12203278.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/15/ifsdata/ifsdata_12203278.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/15/159l_2/clim_data_12203278.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/15/159l_2/clim_data_12203278.zip
Checking for progress XML file: /var/lib/boinc-client/slots/15/progress_file_12203278.xml
Opened progress file ok : /var/lib/boinc-client/slots/15/progress_file_12203278.xml
-- Model is restarting --
Adjusting last_iter, 2923, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/15/progress_file_12203278.xml
last_cpu_time: 83571
upload_file_number: 121
last_iter: 2903
last_upload: 10454400
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0634_2017050100_123_986_12203278_2_r1541080119
The child process has been launched with process id: 1545954
Executing the command: /var/lib/boinc-client/slots/15/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:Serveur-Master:1:1:1545954:1545954] [20230125:223809:1674686289:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:Serveur-Master:1:1:1545954:1545954] [20230125:223809:1674686289:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:Serveur-Master:1:1:1545954:1545954] [20230125:223809:1674686289:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:Serveur-Master:1:1:1545954:1545954] [20230125:223809:1674686289:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:Serveur-Master:1:1:1545954:1545954] [20230125:223809:1674686289:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:Serveur-Master:1:1:1545954:1545954] [20230125:223809:1674686289:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:Serveur-Master:1:1:1545954:1545954] [20230125:223809:1674686289:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:Serveur-Master:1:1:1545954:1545954] [20230125:223809:1674686289:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:Serveur-Master:1:1:1545954:1545954] [20230125:223809:1674686289:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:Serveur-Master:1:1:1545954:1545954] [20230125:223809:1674686289:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:Serveur-Master:1:1:1545954:1545954] [20230125:223809:1674686289:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:Serveur-Master:1:1:1545954:1545954] [20230125:223809:1674686289:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:Serveur-Master:1:1:1545954:1545954] [20230125:223809:1674686289:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:Serveur-Master:1:1:1545954:1545954] [20230125:223809:1674686289:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:Serveur-Master:1:1:1545954:1545954] [20230125:223809:1674686289:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:Serveur-Master:1:1:1545954:1545954] [20230125:223809:1674686289:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Serveur-Master:1:1:1545954:1545954] [20230125:223809:1674686289:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:Serveur-Master:1:1:1545954:1545954] [20230125:223809:1674686289:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Serveur-Master:1:1:1545954:1545954] [20230125:223809:1674686289:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:Serveur-Master:1:1:1545954:1545954] [20230125:223809:1674686289:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:Serveur-Master:1:1:1545954:1545954] [20230125:223809:1674686289:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Serveur-Master:1:1:1545954:1545954] [20230125:223809:1674686289:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Serveur-Master:1:1:1545954:1545954] [20230125:223809:1674686289:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:Serveur-Master:1:1:1545954:1545954] [20230125:223809:1674686289:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Serveur-Master:1:1:1545954:1545954] [20230125:223809:1674686289:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Serveur-Master:1:1:1545954:1545954] [20230125:223809:1674686289:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Serveur-Master:1:1:1545954:1545954] [20230125:223809:1674686289:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:Serveur-Master:1:1:1545954:1545954] [20230125:223809:1674686289:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:Serveur-Master:1:1:1545954:1545954] [20230125:223809:1674686289:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:Serveur-Master:1:1:1545954:1545954] [20230125:223809:1674686289:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  22:39:07 STEP 2904 H=2904:00 +CPU= 53.915
  22:39:40 STEP 2905 H=2905:00 +CPU= 32.825
  22:40:14 STEP 2906 H=2906:00 +CPU= 33.823
  22:41:04 STEP 2907 H=2907:00 +CPU= 49.422
  22:41:37 STEP 2908 H=2908:00 +CPU= 32.665
  22:42:09 STEP 2909 H=2909:00 +CPU= 32.299
  22:43:00 STEP 2910 H=2910:00 +CPU= 50.879
  22:43:34 STEP 2911 H=2911:00 +CPU= 33.417
  22:44:07 STEP 2912 H=2912:00 +CPU= 33.695
  22:44:57 STEP 2913 H=2913:00 +CPU= 48.857
  22:45:30 STEP 2914 H=2914:00 +CPU= 33.384
  22:46:03 STEP 2915 H=2915:00 +CPU= 32.146
  22:46:57 STEP 2916 H=2916:00 +CPU= 54.203
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMGGhq0f+002916
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMSHhq0f+002916
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMUAhq0f+002916
  22:47:30 STEP 2917 H=2917:00 +CPU= 32.973
  22:48:03 STEP 2918 H=2918:00 +CPU= 32.778
  22:48:54 STEP 2919 H=2919:00 +CPU= 50.321
  22:49:29 STEP 2920 H=2920:00 +CPU= 35.225
  22:50:04 STEP 2921 H=2921:00 +CPU= 35.217
  22:50:51 STEP 2922 H=2922:00 +CPU= 46.081
  22:51:22 STEP 2923 H=2923:00 +CPU= 31.700
  22:51:57 STEP 2924 H=2924:00 +CPU= 34.450
  22:52:44 STEP 2925 H=2925:00 +CPU= 46.908
  22:53:18 STEP 2926 H=2926:00 +CPU= 33.562
  22:53:52 STEP 2927 H=2927:00 +CPU= 34.122
  22:54:54 STEP 2928 H=2928:00 +CPU= 61.591
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMGGhq0f+002928
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMSHhq0f+002928
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMUAhq0f+002928
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMGGhq0f+002904
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMSHhq0f+002904
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMUAhq0f+002904
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMGGhq0f+002916
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMSHhq0f+002916
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMUAhq0f+002916
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0634_2017050100_123_986_12203278_2_r1541080119_121.zip
Uploading the intermediate file: upload_file_121.zip
  22:55:28 STEP 2929 H=2929:00 +CPU= 33.645
  22:56:05 STEP 2930 H=2930:00 +CPU= 37.162
  22:56:53 STEP 2931 H=2931:00 +CPU= 48.169
  22:57:30 STEP 2932 H=2932:00 +CPU= 36.890
  22:58:06 STEP 2933 H=2933:00 +CPU= 35.710
  22:59:00 STEP 2934 H=2934:00 +CPU= 53.677
  22:59:35 STEP 2935 H=2935:00 +CPU= 34.499
  23:00:13 STEP 2936 H=2936:00 +CPU= 37.768
  23:01:04 STEP 2937 H=2937:00 +CPU= 51.169
  23:01:40 STEP 2938 H=2938:00 +CPU= 36.072
  23:02:15 STEP 2939 H=2939:00 +CPU= 34.347
  23:03:10 STEP 2940 H=2940:00 +CPU= 55.033
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMGGhq0f+002940
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMSHhq0f+002940
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMUAhq0f+002940
  23:03:45 STEP 2941 H=2941:00 +CPU= 34.340
  23:04:19 STEP 2942 H=2942:00 +CPU= 34.694
  23:05:07 STEP 2943 H=2943:00 +CPU= 47.447
  23:05:40 STEP 2944 H=2944:00 +CPU= 32.893
  23:06:10 STEP 2945 H=2945:00 +CPU= 29.771
  23:07:00 STEP 2946 H=2946:00 +CPU= 49.946
  23:07:33 STEP 2947 H=2947:00 +CPU= 32.880
  23:08:07 STEP 2948 H=2948:00 +CPU= 33.584
  23:08:57 STEP 2949 H=2949:00 +CPU= 50.115
  23:09:31 STEP 2950 H=2950:00 +CPU= 33.829
  23:10:07 STEP 2951 H=2951:00 +CPU= 35.102
  23:11:03 STEP 2952 H=2952:00 +CPU= 55.764
The child process terminated with status: 0
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMGGhq0f+002952
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMSHhq0f+002952
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMUAhq0f+002952
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMGGhq0f+002952
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMSHhq0f+002940
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMUAhq0f+002928
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMSHhq0f+002928
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMUAhq0f+002952
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMUAhq0f+002940
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMSHhq0f+002952
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMGGhq0f+002928
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12203278/ICMGGhq0f+002940
Zipping up the final file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0634_2017050100_123_986_12203278_2_r1541080119_122.zip
Uploading the final file: upload_file_122.zip
Uploading trickle at timestep: 10623600
23:14:26 (1545949): called boinc_finish(0)

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 May 2023 16:41:21 1532331 22302736 oifs_43r3_ps_0634_2017050100_123_986_12203278_2 10,623,600 85,536 0.0081
09 May 2023 16:41:21 1532331 22302736 oifs_43r3_ps_0634_2017050100_123_986_12203278_2 10,623,600 85,536 0.0081
09 May 2023 16:40:50 1532331 22302736 oifs_43r3_ps_0634_2017050100_123_986_12203278_2 9,504,000 70,597 0.0074
09 May 2023 16:40:32 1532331 22302736 oifs_43r3_ps_0634_2017050100_123_986_12203278_2 8,640,000 60,024 0.0069
09 May 2023 16:40:32 1532331 22302736 oifs_43r3_ps_0634_2017050100_123_986_12203278_2 8,640,000 60,024 0.0069
09 May 2023 16:40:15 1532331 22302736 oifs_43r3_ps_0634_2017050100_123_986_12203278_2 6,998,400 40,870 0.0058
09 May 2023 16:40:09 1532331 22302736 oifs_43r3_ps_0634_2017050100_123_986_12203278_2 6,134,400 32,654 0.0053
09 May 2023 16:40:05 1532331 22302736 oifs_43r3_ps_0634_2017050100_123_986_12203278_2 5,270,400 28,131 0.0053
09 May 2023 16:40:02 1532331 22302736 oifs_43r3_ps_0634_2017050100_123_986_12203278_2 4,406,400 23,589 0.0054
09 May 2023 16:39:55 1532331 22302736 oifs_43r3_ps_0634_2017050100_123_986_12203278_2 3,542,400 18,998 0.0054
09 May 2023 16:39:52 1532331 22302736 oifs_43r3_ps_0634_2017050100_123_986_12203278_2 2,678,400 14,475 0.0054
09 May 2023 16:39:48 1532331 22302736 oifs_43r3_ps_0634_2017050100_123_986_12203278_2 1,814,400 9,993 0.0055
09 May 2023 16:39:48 1532331 22302736 oifs_43r3_ps_0634_2017050100_123_986_12203278_2 1,814,400 9,993 0.0055
09 May 2023 16:39:45 1532331 22302736 oifs_43r3_ps_0634_2017050100_123_986_12203278_2 950,400 5,490 0.0058


©2024 climateprediction.net