climateprediction.net home page
Task 22301097

Task 22301097

Name oifs_43r3_ps_0269_1997050100_123_966_12182913_1
Workunit 12182913
Created 21 Jan 2023, 13:17:57 UTC
Sent 21 Jan 2023, 13:30:53 UTC
Report deadline 22 Mar 2023, 13:30:53 UTC
Received 22 Jan 2023, 10:13:06 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 196 (0x000000C4) EXIT_DISK_LIMIT_EXCEEDED
Computer ID 1491072
Run time 18 hours 17 min 34 sec
CPU time 17 hours 52 min 25 sec
Validate state Invalid
Credit 3,870.72
Device peak FLOPS 4.70 GFLOPS
Application version OpenIFS 43r3 Perturbed Surface v1.05
x86_64-pc-linux-gnu
Peak working set size 4,380.98 MB
Peak swap size 4,978.08 MB
Peak disk usage 7,320.98 MB
Stderr
<core_client_version>7.16.16</core_client_version>
<![CDATA[
<message>
Disk usage limit exceeded</message>
<stderr_txt>
540
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000540
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000540
  01:10:23 STEP  541 H= 541:00 +CPU= 56.683
  01:11:19 STEP  542 H= 542:00 +CPU= 55.567
  01:12:41 STEP  543 H= 543:00 +CPU= 80.183
  01:13:40 STEP  544 H= 544:00 +CPU= 57.651
  01:14:35 STEP  545 H= 545:00 +CPU= 53.981
  01:15:58 STEP  546 H= 546:00 +CPU= 81.797
  01:16:55 STEP  547 H= 547:00 +CPU= 56.459
  01:17:52 STEP  548 H= 548:00 +CPU= 55.765
  01:19:14 STEP  549 H= 549:00 +CPU= 80.156
  01:20:12 STEP  550 H= 550:00 +CPU= 57.165
  01:21:09 STEP  551 H= 551:00 +CPU= 56.241
  01:22:47 STEP  552 H= 552:00 +CPU= 94.897
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000552
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000552
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000552
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMGGhq0f+000528
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMSHhq0f+000528
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMUAhq0f+000528
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMGGhq0f+000540
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMSHhq0f+000540
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMUAhq0f+000540
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0269_1997050100_123_966_12182913_1_r433506247_22.zip
Uploading the intermediate file: upload_file_22.zip
  01:23:44 STEP  553 H= 553:00 +CPU= 56.443
  01:24:40 STEP  554 H= 554:00 +CPU= 54.928
  01:26:02 STEP  555 H= 555:00 +CPU= 80.745
  01:27:01 STEP  556 H= 556:00 +CPU= 57.550
  01:27:55 STEP  557 H= 557:00 +CPU= 53.147
  01:29:18 STEP  558 H= 558:00 +CPU= 81.999
  01:30:15 STEP  559 H= 559:00 +CPU= 55.915
  01:31:12 STEP  560 H= 560:00 +CPU= 55.401
  01:32:33 STEP  561 H= 561:00 +CPU= 80.187
  01:33:32 STEP  562 H= 562:00 +CPU= 57.321
  01:34:27 STEP  563 H= 563:00 +CPU= 54.430
  01:35:58 STEP  564 H= 564:00 +CPU= 89.296
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000564
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000564
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000564
  01:36:55 STEP  565 H= 565:00 +CPU= 55.548
  01:37:51 STEP  566 H= 566:00 +CPU= 55.196
  01:39:13 STEP  567 H= 567:00 +CPU= 80.613
  01:40:11 STEP  568 H= 568:00 +CPU= 57.045
  01:41:06 STEP  569 H= 569:00 +CPU= 54.265
  01:42:30 STEP  570 H= 570:00 +CPU= 82.286
  01:43:26 STEP  571 H= 571:00 +CPU= 55.058
  01:44:23 STEP  572 H= 572:00 +CPU= 56.688
  01:45:45 STEP  573 H= 573:00 +CPU= 80.316
  01:46:43 STEP  574 H= 574:00 +CPU= 56.433
  01:47:41 STEP  575 H= 575:00 +CPU= 57.074
  01:49:17 STEP  576 H= 576:00 +CPU= 94.122
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000576
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000576
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000576
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMGGhq0f+000552
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMSHhq0f+000552
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMUAhq0f+000552
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMGGhq0f+000564
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMSHhq0f+000564
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMUAhq0f+000564
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0269_1997050100_123_966_12182913_1_r433506247_23.zip
Uploading the intermediate file: upload_file_23.zip
  01:50:14 STEP  577 H= 577:00 +CPU= 55.271
  01:51:10 STEP  578 H= 578:00 +CPU= 55.448
  01:52:32 STEP  579 H= 579:00 +CPU= 80.345
  01:53:30 STEP  580 H= 580:00 +CPU= 57.062
  01:54:25 STEP  581 H= 581:00 +CPU= 53.866
  01:55:48 STEP  582 H= 582:00 +CPU= 82.134
  01:56:44 STEP  583 H= 583:00 +CPU= 54.911
  01:57:42 STEP  584 H= 584:00 +CPU= 56.776
  01:59:04 STEP  585 H= 585:00 +CPU= 80.117
  02:00:01 STEP  586 H= 586:00 +CPU= 56.549
  02:00:58 STEP  587 H= 587:00 +CPU= 55.762
  02:02:29 STEP  588 H= 588:00 +CPU= 89.425
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000588
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000588
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000588
  02:03:25 STEP  589 H= 589:00 +CPU= 54.834
  02:04:22 STEP  590 H= 590:00 +CPU= 56.300
  02:05:44 STEP  591 H= 591:00 +CPU= 80.438
  02:06:42 STEP  592 H= 592:00 +CPU= 56.581
  02:07:37 STEP  593 H= 593:00 +CPU= 54.611
  02:08:59 STEP  594 H= 594:00 +CPU= 80.723
  02:09:56 STEP  595 H= 595:00 +CPU= 55.605
  02:10:53 STEP  596 H= 596:00 +CPU= 56.453
  02:12:14 STEP  597 H= 597:00 +CPU= 79.423
  02:13:12 STEP  598 H= 598:00 +CPU= 56.866
  02:14:11 STEP  599 H= 599:00 +CPU= 58.220
  02:15:48 STEP  600 H= 600:00 +CPU= 93.954
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000600
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000600
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000600
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMGGhq0f+000576
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMSHhq0f+000576
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMUAhq0f+000576
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMGGhq0f+000588
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMSHhq0f+000588
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMUAhq0f+000588
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0269_1997050100_123_966_12182913_1_r433506247_24.zip
Uploading the intermediate file: upload_file_24.zip
  02:16:44 STEP  601 H= 601:00 +CPU= 54.894
  02:17:40 STEP  602 H= 602:00 +CPU= 55.479
  02:19:02 STEP  603 H= 603:00 +CPU= 80.412
  02:20:00 STEP  604 H= 604:00 +CPU= 56.811
  02:20:55 STEP  605 H= 605:00 +CPU= 54.499
  02:22:18 STEP  606 H= 606:00 +CPU= 81.420
  02:23:14 STEP  607 H= 607:00 +CPU= 54.917
  02:24:12 STEP  608 H= 608:00 +CPU= 57.271
  02:25:33 STEP  609 H= 609:00 +CPU= 79.556
  02:26:31 STEP  610 H= 610:00 +CPU= 56.982
  02:27:28 STEP  611 H= 611:00 +CPU= 55.724
  02:28:58 STEP  612 H= 612:00 +CPU= 88.485
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000612
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000612
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000612
  02:29:54 STEP  613 H= 613:00 +CPU= 54.988
  02:30:52 STEP  614 H= 614:00 +CPU= 56.910
  02:32:13 STEP  615 H= 615:00 +CPU= 79.053
  02:33:11 STEP  616 H= 616:00 +CPU= 57.023
  02:34:08 STEP  617 H= 617:00 +CPU= 55.911
  02:35:30 STEP  618 H= 618:00 +CPU= 80.697
  02:36:26 STEP  619 H= 619:00 +CPU= 55.352
  02:37:25 STEP  620 H= 620:00 +CPU= 57.940
  02:38:45 STEP  621 H= 621:00 +CPU= 78.479
  02:39:42 STEP  622 H= 622:00 +CPU= 56.322
  02:40:42 STEP  623 H= 623:00 +CPU= 58.592
  02:42:17 STEP  624 H= 624:00 +CPU= 93.383
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000624
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000624
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000624
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMGGhq0f+000600
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMSHhq0f+000600
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMUAhq0f+000600
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMGGhq0f+000612
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMSHhq0f+000612
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMUAhq0f+000612
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0269_1997050100_123_966_12182913_1_r433506247_25.zip
Uploading the intermediate file: upload_file_25.zip
  02:43:13 STEP  625 H= 625:00 +CPU= 54.910
  02:44:11 STEP  626 H= 626:00 +CPU= 56.417
  02:45:31 STEP  627 H= 627:00 +CPU= 79.011
  02:46:29 STEP  628 H= 628:00 +CPU= 56.542
  02:47:25 STEP  629 H= 629:00 +CPU= 55.448
  02:48:47 STEP  630 H= 630:00 +CPU= 80.161
  02:49:43 STEP  631 H= 631:00 +CPU= 54.979
  02:50:41 STEP  632 H= 632:00 +CPU= 57.940
  02:52:01 STEP  633 H= 633:00 +CPU= 78.040
  02:52:58 STEP  634 H= 634:00 +CPU= 56.703
  02:53:56 STEP  635 H= 635:00 +CPU= 56.480
  02:55:25 STEP  636 H= 636:00 +CPU= 87.717
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000636
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000636
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000636
  02:56:22 STEP  637 H= 637:00 +CPU= 55.045
  02:57:20 STEP  638 H= 638:00 +CPU= 57.639
  02:58:40 STEP  639 H= 639:00 +CPU= 78.619
  02:59:37 STEP  640 H= 640:00 +CPU= 56.127
  03:00:35 STEP  641 H= 641:00 +CPU= 56.463
  03:01:57 STEP  642 H= 642:00 +CPU= 80.496
  03:02:53 STEP  643 H= 643:00 +CPU= 55.445
  03:03:52 STEP  644 H= 644:00 +CPU= 58.229
  03:05:13 STEP  645 H= 645:00 +CPU= 78.946
  03:06:11 STEP  646 H= 646:00 +CPU= 56.981
  03:07:10 STEP  647 H= 647:00 +CPU= 58.541
  03:08:46 STEP  648 H= 648:00 +CPU= 92.904
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000648
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000648
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000648
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMGGhq0f+000624
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMSHhq0f+000624
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMUAhq0f+000624
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMGGhq0f+000636
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMSHhq0f+000636
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMUAhq0f+000636
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0269_1997050100_123_966_12182913_1_r433506247_26.zip
Uploading the intermediate file: upload_file_26.zip
  03:09:42 STEP  649 H= 649:00 +CPU= 54.771
  03:10:40 STEP  650 H= 650:00 +CPU= 57.424
  03:12:00 STEP  651 H= 651:00 +CPU= 78.556
  03:12:58 STEP  652 H= 652:00 +CPU= 57.031
  03:13:54 STEP  653 H= 653:00 +CPU= 55.346
  03:15:16 STEP  654 H= 654:00 +CPU= 80.460
  03:16:13 STEP  655 H= 655:00 +CPU= 56.288
  03:17:11 STEP  656 H= 656:00 +CPU= 56.871
  03:18:31 STEP  657 H= 657:00 +CPU= 78.913
  03:19:30 STEP  658 H= 658:00 +CPU= 57.717
  03:20:27 STEP  659 H= 659:00 +CPU= 55.761
  03:21:56 STEP  660 H= 660:00 +CPU= 87.814
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000660
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000660
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000660
  03:22:53 STEP  661 H= 661:00 +CPU= 55.287
  03:23:50 STEP  662 H= 662:00 +CPU= 56.854
  03:25:11 STEP  663 H= 663:00 +CPU= 78.712
  03:26:09 STEP  664 H= 664:00 +CPU= 57.617
  03:27:05 STEP  665 H= 665:00 +CPU= 54.944
  03:28:26 STEP  666 H= 666:00 +CPU= 79.864
  03:29:23 STEP  667 H= 667:00 +CPU= 55.259
  03:30:20 STEP  668 H= 668:00 +CPU= 56.875
  03:31:40 STEP  669 H= 669:00 +CPU= 78.519
  03:32:38 STEP  670 H= 670:00 +CPU= 56.901
  03:33:37 STEP  671 H= 671:00 +CPU= 57.724
  03:35:13 STEP  672 H= 672:00 +CPU= 93.870
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000672
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000672
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000672
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMGGhq0f+000648
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMSHhq0f+000648
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMUAhq0f+000648
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMGGhq0f+000660
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMSHhq0f+000660
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMUAhq0f+000660
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0269_1997050100_123_966_12182913_1_r433506247_27.zip
Uploading the intermediate file: upload_file_27.zip
  03:36:10 STEP  673 H= 673:00 +CPU= 55.797
  03:37:07 STEP  674 H= 674:00 +CPU= 56.026
  03:38:27 STEP  675 H= 675:00 +CPU= 78.617
  03:39:25 STEP  676 H= 676:00 +CPU= 57.148
  03:40:21 STEP  677 H= 677:00 +CPU= 55.253
  03:41:42 STEP  678 H= 678:00 +CPU= 79.815
  03:42:39 STEP  679 H= 679:00 +CPU= 55.446
  03:43:37 STEP  680 H= 680:00 +CPU= 57.092
  03:44:58 STEP  681 H= 681:00 +CPU= 79.598
  03:45:56 STEP  682 H= 682:00 +CPU= 56.827
  03:46:53 STEP  683 H= 683:00 +CPU= 56.004
  03:48:23 STEP  684 H= 684:00 +CPU= 89.104
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000684
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000684
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000684
  03:49:20 STEP  685 H= 685:00 +CPU= 55.785
  03:50:18 STEP  686 H= 686:00 +CPU= 56.799
  03:51:39 STEP  687 H= 687:00 +CPU= 79.155
  03:52:36 STEP  688 H= 688:00 +CPU= 56.119
  03:53:32 STEP  689 H= 689:00 +CPU= 54.914
  03:54:54 STEP  690 H= 690:00 +CPU= 81.105
  03:55:50 STEP  691 H= 691:00 +CPU= 54.672
  03:56:48 STEP  692 H= 692:00 +CPU= 57.152
  03:58:10 STEP  693 H= 693:00 +CPU= 80.315
  03:59:06 STEP  694 H= 694:00 +CPU= 55.348
  04:00:05 STEP  695 H= 695:00 +CPU= 57.890
  04:01:41 STEP  696 H= 696:00 +CPU= 93.500
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000696
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000696
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000696
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMGGhq0f+000672
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMSHhq0f+000672
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMUAhq0f+000672
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMGGhq0f+000684
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMSHhq0f+000684
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMUAhq0f+000684
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0269_1997050100_123_966_12182913_1_r433506247_28.zip
Uploading the intermediate file: upload_file_28.zip
  04:02:38 STEP  697 H= 697:00 +CPU= 55.711
  04:03:35 STEP  698 H= 698:00 +CPU= 56.463
  04:04:56 STEP  699 H= 699:00 +CPU= 79.286
  04:05:53 STEP  700 H= 700:00 +CPU= 56.232
  04:06:49 STEP  701 H= 701:00 +CPU= 55.563
  04:08:13 STEP  702 H= 702:00 +CPU= 81.780
  04:09:08 STEP  703 H= 703:00 +CPU= 54.052
  04:10:04 STEP  704 H= 704:00 +CPU= 55.979
  04:11:26 STEP  705 H= 705:00 +CPU= 80.380
  04:12:23 STEP  706 H= 706:00 +CPU= 55.916
  04:13:20 STEP  707 H= 707:00 +CPU= 55.982
  04:14:51 STEP  708 H= 708:00 +CPU= 89.150
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000708
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000708
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000708
  04:15:46 STEP  709 H= 709:00 +CPU= 53.938
  04:16:45 STEP  710 H= 710:00 +CPU= 57.810
  04:18:06 STEP  711 H= 711:00 +CPU= 80.307
  04:19:03 STEP  712 H= 712:00 +CPU= 55.762
  04:19:59 STEP  713 H= 713:00 +CPU= 55.244
  04:21:23 STEP  714 H= 714:00 +CPU= 81.865
  04:22:17 STEP  715 H= 715:00 +CPU= 53.576
  04:23:15 STEP  716 H= 716:00 +CPU= 57.422
  04:24:37 STEP  717 H= 717:00 +CPU= 80.546
  04:25:33 STEP  718 H= 718:00 +CPU= 54.613
  04:26:32 STEP  719 H= 719:00 +CPU= 58.342
  04:28:08 STEP  720 H= 720:00 +CPU= 93.644
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000720
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000720
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000720
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMGGhq0f+000696
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMSHhq0f+000696
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMUAhq0f+000696
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMGGhq0f+000708
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMSHhq0f+000708
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMUAhq0f+000708
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0269_1997050100_123_966_12182913_1_r433506247_29.zip
Uploading the intermediate file: upload_file_29.zip
  04:29:04 STEP  721 H= 721:00 +CPU= 54.385
  04:30:01 STEP  722 H= 722:00 +CPU= 56.634
  04:31:22 STEP  723 H= 723:00 +CPU= 79.711
  04:32:19 STEP  724 H= 724:00 +CPU= 55.294
  04:33:16 STEP  725 H= 725:00 +CPU= 55.656
  04:34:38 STEP  726 H= 726:00 +CPU= 81.113
  04:35:32 STEP  727 H= 727:00 +CPU= 53.016
  04:36:31 STEP  728 H= 728:00 +CPU= 58.413
  04:37:53 STEP  729 H= 729:00 +CPU= 79.828
  04:38:49 STEP  730 H= 730:00 +CPU= 54.975
  04:39:47 STEP  731 H= 731:00 +CPU= 57.170
  04:41:18 STEP  732 H= 732:00 +CPU= 89.474
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000732
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000732
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000732
  04:42:13 STEP  733 H= 733:00 +CPU= 53.480
  04:43:11 STEP  734 H= 734:00 +CPU= 57.303
  04:44:33 STEP  735 H= 735:00 +CPU= 80.361
  04:45:28 STEP  736 H= 736:00 +CPU= 54.076
  04:46:26 STEP  737 H= 737:00 +CPU= 56.948
  04:47:49 STEP  738 H= 738:00 +CPU= 82.189
  04:48:43 STEP  739 H= 739:00 +CPU= 52.273
  04:49:41 STEP  740 H= 740:00 +CPU= 57.905
  04:51:03 STEP  741 H= 741:00 +CPU= 80.564
  04:51:58 STEP  742 H= 742:00 +CPU= 54.089
  04:52:58 STEP  743 H= 743:00 +CPU= 58.799
  04:54:36 STEP  744 H= 744:00 +CPU= 94.799
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000744
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000744
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000744
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMGGhq0f+000720
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMSHhq0f+000720
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMUAhq0f+000720
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMGGhq0f+000732
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMSHhq0f+000732
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMUAhq0f+000732
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0269_1997050100_123_966_12182913_1_r433506247_30.zip
Uploading the intermediate file: upload_file_30.zip
  04:55:30 STEP  745 H= 745:00 +CPU= 53.134
  04:56:29 STEP  746 H= 746:00 +CPU= 58.326
  04:57:51 STEP  747 H= 747:00 +CPU= 79.869
  04:58:46 STEP  748 H= 748:00 +CPU= 54.483
  04:59:43 STEP  749 H= 749:00 +CPU= 56.143
  05:01:06 STEP  750 H= 750:00 +CPU= 81.612
  05:02:00 STEP  751 H= 751:00 +CPU= 52.783
  05:02:59 STEP  752 H= 752:00 +CPU= 58.066
  05:04:21 STEP  753 H= 753:00 +CPU= 81.181
  05:05:17 STEP  754 H= 754:00 +CPU= 54.155
  05:06:14 STEP  755 H= 755:00 +CPU= 56.446
  05:07:47 STEP  756 H= 756:00 +CPU= 90.445
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000756
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000756
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000756
  05:08:40 STEP  757 H= 757:00 +CPU= 52.561
  05:09:39 STEP  758 H= 758:00 +CPU= 57.947
  05:11:02 STEP  759 H= 759:00 +CPU= 81.259
  05:11:56 STEP  760 H= 760:00 +CPU= 53.699
  05:12:54 STEP  761 H= 761:00 +CPU= 56.336
  05:14:18 STEP  762 H= 762:00 +CPU= 83.090
  05:15:11 STEP  763 H= 763:00 +CPU= 51.667
  05:16:09 STEP  764 H= 764:00 +CPU= 57.604
  05:17:32 STEP  765 H= 765:00 +CPU= 81.559
  05:18:27 STEP  766 H= 766:00 +CPU= 53.132
  05:19:27 STEP  767 H= 767:00 +CPU= 58.867
  05:21:04 STEP  768 H= 768:00 +CPU= 95.181
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000768
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000768
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000768
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMGGhq0f+000744
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMSHhq0f+000744
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMUAhq0f+000744
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMGGhq0f+000756
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMSHhq0f+000756
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMUAhq0f+000756
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0269_1997050100_123_966_12182913_1_r433506247_31.zip
Uploading the intermediate file: upload_file_31.zip
Uploading trickle at timestep: 2764800
  05:21:58 STEP  769 H= 769:00 +CPU= 52.145
  05:22:56 STEP  770 H= 770:00 +CPU= 57.860
  05:24:18 STEP  771 H= 771:00 +CPU= 80.428
  05:25:12 STEP  772 H= 772:00 +CPU= 53.240
  05:26:09 STEP  773 H= 773:00 +CPU= 55.916
  05:27:34 STEP  774 H= 774:00 +CPU= 82.934
  05:28:26 STEP  775 H= 775:00 +CPU= 51.562
  05:29:25 STEP  776 H= 776:00 +CPU= 57.694
  05:30:47 STEP  777 H= 777:00 +CPU= 80.924
  05:31:41 STEP  778 H= 778:00 +CPU= 52.750
  05:32:37 STEP  779 H= 779:00 +CPU= 55.340
  05:34:10 STEP  780 H= 780:00 +CPU= 91.573
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000780
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000780
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000780
  05:35:03 STEP  781 H= 781:00 +CPU= 51.913
  05:36:02 STEP  782 H= 782:00 +CPU= 57.870
  05:37:26 STEP  783 H= 783:00 +CPU= 82.228
  05:38:20 STEP  784 H= 784:00 +CPU= 53.251
  05:39:16 STEP  785 H= 785:00 +CPU= 55.348
  05:40:42 STEP  786 H= 786:00 +CPU= 83.697
  05:41:35 STEP  787 H= 787:00 +CPU= 51.950
  05:42:32 STEP  788 H= 788:00 +CPU= 56.869
  05:43:55 STEP  789 H= 789:00 +CPU= 81.680
  05:44:50 STEP  790 H= 790:00 +CPU= 54.023
  05:45:48 STEP  791 H= 791:00 +CPU= 56.353
  05:47:27 STEP  792 H= 792:00 +CPU= 96.205
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000792
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000792
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000792
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMGGhq0f+000768
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMSHhq0f+000768
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMUAhq0f+000768
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMGGhq0f+000780
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMSHhq0f+000780
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMUAhq0f+000780
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0269_1997050100_123_966_12182913_1_r433506247_32.zip
Uploading the intermediate file: upload_file_32.zip
  05:48:20 STEP  793 H= 793:00 +CPU= 51.974
  05:49:18 STEP  794 H= 794:00 +CPU= 56.913
  05:50:41 STEP  795 H= 795:00 +CPU= 81.782
  05:51:35 STEP  796 H= 796:00 +CPU= 53.532
  05:52:31 STEP  797 H= 797:00 +CPU= 54.920
  05:53:56 STEP  798 H= 798:00 +CPU= 83.632
  05:54:50 STEP  799 H= 799:00 +CPU= 52.555
  05:55:47 STEP  800 H= 800:00 +CPU= 56.284
  05:57:10 STEP  801 H= 801:00 +CPU= 81.679
  05:58:05 STEP  802 H= 802:00 +CPU= 54.178
  05:59:01 STEP  803 H= 803:00 +CPU= 54.441
  06:00:34 STEP  804 H= 804:00 +CPU= 91.809
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000804
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000804
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000804
  06:01:28 STEP  805 H= 805:00 +CPU= 52.708
  06:02:25 STEP  806 H= 806:00 +CPU= 56.395
  06:03:49 STEP  807 H= 807:00 +CPU= 82.228
  06:04:44 STEP  808 H= 808:00 +CPU= 53.989
  06:05:40 STEP  809 H= 809:00 +CPU= 54.819
  06:07:06 STEP  810 H= 810:00 +CPU= 84.611
  06:07:59 STEP  811 H= 811:00 +CPU= 52.848
  06:08:57 STEP  812 H= 812:00 +CPU= 56.065
  06:10:22 STEP  813 H= 813:00 +CPU= 83.454
  06:11:17 STEP  814 H= 814:00 +CPU= 54.249
  06:12:14 STEP  815 H= 815:00 +CPU= 56.069
  06:13:53 STEP  816 H= 816:00 +CPU= 97.491
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000816
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000816
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000816
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMGGhq0f+000792
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMSHhq0f+000792
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMUAhq0f+000792
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMGGhq0f+000804
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMSHhq0f+000804
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMUAhq0f+000804
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0269_1997050100_123_966_12182913_1_r433506247_33.zip
Uploading the intermediate file: upload_file_33.zip
  06:14:47 STEP  817 H= 817:00 +CPU= 52.268
  06:15:44 STEP  818 H= 818:00 +CPU= 56.103
  06:17:08 STEP  819 H= 819:00 +CPU= 82.469
  06:18:03 STEP  820 H= 820:00 +CPU= 53.734
  06:18:57 STEP  821 H= 821:00 +CPU= 53.669
  06:20:22 STEP  822 H= 822:00 +CPU= 83.518
  06:21:16 STEP  823 H= 823:00 +CPU= 52.763
  06:22:12 STEP  824 H= 824:00 +CPU= 55.142
  06:23:37 STEP  825 H= 825:00 +CPU= 83.233
  06:24:32 STEP  826 H= 826:00 +CPU= 53.904
  06:25:26 STEP  827 H= 827:00 +CPU= 53.169
  06:27:00 STEP  828 H= 828:00 +CPU= 92.666
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000828
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000828
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000828
  06:27:53 STEP  829 H= 829:00 +CPU= 52.110
  06:28:48 STEP  830 H= 830:00 +CPU= 54.102
  06:30:13 STEP  831 H= 831:00 +CPU= 83.746
  06:31:08 STEP  832 H= 832:00 +CPU= 53.826
  06:32:01 STEP  833 H= 833:00 +CPU= 52.291
  06:33:29 STEP  834 H= 834:00 +CPU= 86.085
  06:34:22 STEP  835 H= 835:00 +CPU= 52.337
  06:35:16 STEP  836 H= 836:00 +CPU= 53.156
  06:36:43 STEP  837 H= 837:00 +CPU= 85.068
  06:37:38 STEP  838 H= 838:00 +CPU= 53.615
  06:38:29 STEP  839 H= 839:00 +CPU= 50.826
  06:40:00 STEP  840 H= 840:00 +CPU= 88.327
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000840
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000840
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000840
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMGGhq0f+000816
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMSHhq0f+000816
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMUAhq0f+000816
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMGGhq0f+000828
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMSHhq0f+000828
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMUAhq0f+000828
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0269_1997050100_123_966_12182913_1_r433506247_34.zip
Uploading the intermediate file: upload_file_34.zip
  06:40:49 STEP  841 H= 841:00 +CPU= 47.470
  06:41:39 STEP  842 H= 842:00 +CPU= 49.697
  06:42:55 STEP  843 H= 843:00 +CPU= 74.740
  06:43:45 STEP  844 H= 844:00 +CPU= 48.743
  06:44:34 STEP  845 H= 845:00 +CPU= 48.090
  06:45:51 STEP  846 H= 846:00 +CPU= 76.463
  06:46:40 STEP  847 H= 847:00 +CPU= 47.308
  06:47:30 STEP  848 H= 848:00 +CPU= 49.622
  06:48:46 STEP  849 H= 849:00 +CPU= 75.415
  06:49:36 STEP  850 H= 850:00 +CPU= 48.466
  06:50:26 STEP  851 H= 851:00 +CPU= 49.562
  06:51:56 STEP  852 H= 852:00 +CPU= 87.748
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000852
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000852
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000852
  06:52:43 STEP  853 H= 853:00 +CPU= 46.142
  06:53:34 STEP  854 H= 854:00 +CPU= 50.184
  06:54:49 STEP  855 H= 855:00 +CPU= 73.176
  06:55:36 STEP  856 H= 856:00 +CPU= 46.968
  06:56:24 STEP  857 H= 857:00 +CPU= 47.558
  06:57:37 STEP  858 H= 858:00 +CPU= 72.069
  06:58:24 STEP  859 H= 859:00 +CPU= 46.141
  06:59:12 STEP  860 H= 860:00 +CPU= 47.966
  07:00:25 STEP  861 H= 861:00 +CPU= 71.906
  07:01:12 STEP  862 H= 862:00 +CPU= 47.248
  07:02:06 STEP  863 H= 863:00 +CPU= 52.328
  07:03:35 STEP  864 H= 864:00 +CPU= 86.854
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000864
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000864
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000864
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMGGhq0f+000840
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMSHhq0f+000840
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMUAhq0f+000840
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMGGhq0f+000852
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMSHhq0f+000852
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMUAhq0f+000852
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0269_1997050100_123_966_12182913_1_r433506247_35.zip
Uploading the intermediate file: upload_file_35.zip
  07:04:22 STEP  865 H= 865:00 +CPU= 46.308
  07:05:12 STEP  866 H= 866:00 +CPU= 49.976
  07:06:23 STEP  867 H= 867:00 +CPU= 69.844
  07:07:11 STEP  868 H= 868:00 +CPU= 47.452
  07:08:00 STEP  869 H= 869:00 +CPU= 49.093
  07:09:13 STEP  870 H= 870:00 +CPU= 71.959
  07:10:00 STEP  871 H= 871:00 +CPU= 46.607
  07:10:52 STEP  872 H= 872:00 +CPU= 51.150
  07:12:03 STEP  873 H= 873:00 +CPU= 70.542
  07:12:51 STEP  874 H= 874:00 +CPU= 47.685
  07:13:41 STEP  875 H= 875:00 +CPU= 49.121
  07:15:01 STEP  876 H= 876:00 +CPU= 75.691
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000876
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000876
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000876
  07:15:50 STEP  877 H= 877:00 +CPU= 47.948
  07:16:41 STEP  878 H= 878:00 +CPU= 50.499
  07:17:50 STEP  879 H= 879:00 +CPU= 68.808
  07:18:39 STEP  880 H= 880:00 +CPU= 48.154
  07:19:29 STEP  881 H= 881:00 +CPU= 49.212
  07:20:39 STEP  882 H= 882:00 +CPU= 70.091
  07:21:28 STEP  883 H= 883:00 +CPU= 47.747
  07:22:22 STEP  884 H= 884:00 +CPU= 53.068
  07:23:33 STEP  885 H= 885:00 +CPU= 68.913
  07:24:27 STEP  886 H= 886:00 +CPU= 52.818
  07:25:28 STEP  887 H= 887:00 +CPU= 53.417
  07:26:59 STEP  888 H= 888:00 +CPU= 87.056
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000888
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000888
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000888
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMGGhq0f+000864
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMSHhq0f+000864
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMUAhq0f+000864
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMGGhq0f+000876
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMSHhq0f+000876
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMUAhq0f+000876
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0269_1997050100_123_966_12182913_1_r433506247_36.zip
Uploading the intermediate file: upload_file_36.zip
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 1997050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0269
(argv4) batchid: 966
(argv5) wuid: 12182913
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/4
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/4/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/4/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_2d1a65e9345c940e9cc5b0f67312eb4c to: /var/lib/boinc-client/slots/4/oifs_43r3_ps_0269_1997050100_123_966_12182913.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/4/oifs_43r3_ps_0269_1997050100_123_966_12182913.zip
ic_ancil_file: ic_ancil_12182913
ifsdata_file: ifsdata_12182913
climate_data_file: clim_data_12182913
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_bae4e80c3f46772533a2e95c7515ffa2 to: /var/lib/boinc-client/slots/4/ic_ancil_12182913.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/4/ic_ancil_12182913.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_58d882547242fb8ea5840ceb3f29633e to: /var/lib/boinc-client/slots/4/ifsdata/ifsdata_12182913.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/4/ifsdata/ifsdata_12182913.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/4/159l_2/clim_data_12182913.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/4/159l_2/clim_data_12182913.zip
Checking for progress XML file: /var/lib/boinc-client/slots/4/progress_file_12182913.xml
Opened progress file ok : /var/lib/boinc-client/slots/4/progress_file_12182913.xml
-- Model is restarting --
Adjusting last_iter, 889, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/4/progress_file_12182913.xml
last_cpu_time: 58694
upload_file_number: 37
last_iter: 887
last_upload: 3196800
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0269_1997050100_123_966_12182913_1_r433506247
The child process has been launched with process id: 341654
Executing the command: /var/lib/boinc-client/slots/4/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:Debian17:1:1:341654:341654] [20230122:073038:1674372638:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:Debian17:1:1:341654:341654] [20230122:073038:1674372638:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:Debian17:1:1:341654:341654] [20230122:073038:1674372638:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:Debian17:1:1:341654:341654] [20230122:073038:1674372638:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:Debian17:1:1:341654:341654] [20230122:073038:1674372638:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:Debian17:1:1:341654:341654] [20230122:073038:1674372638:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:Debian17:1:1:341654:341654] [20230122:073038:1674372638:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:Debian17:1:1:341654:341654] [20230122:073038:1674372638:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:Debian17:1:1:341654:341654] [20230122:073038:1674372638:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:Debian17:1:1:341654:341654] [20230122:073038:1674372638:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:Debian17:1:1:341654:341654] [20230122:073038:1674372638:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:Debian17:1:1:341654:341654] [20230122:073038:1674372638:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:Debian17:1:1:341654:341654] [20230122:073038:1674372638:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:Debian17:1:1:341654:341654] [20230122:073038:1674372638:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:Debian17:1:1:341654:341654] [20230122:073038:1674372638:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:Debian17:1:1:341654:341654] [20230122:073038:1674372638:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Debian17:1:1:341654:341654] [20230122:073038:1674372638:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:Debian17:1:1:341654:341654] [20230122:073038:1674372638:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Debian17:1:1:341654:341654] [20230122:073038:1674372638:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:Debian17:1:1:341654:341654] [20230122:073038:1674372638:0.001] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:Debian17:1:1:341654:341654] [20230122:073038:1674372638:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Debian17:1:1:341654:341654] [20230122:073038:1674372638:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Debian17:1:1:341654:341654] [20230122:073038:1674372638:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:Debian17:1:1:341654:341654] [20230122:073038:1674372638:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Debian17:1:1:341654:341654] [20230122:073038:1674372638:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Debian17:1:1:341654:341654] [20230122:073038:1674372638:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Debian17:1:1:341654:341654] [20230122:073038:1674372638:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:Debian17:1:1:341654:341654] [20230122:073038:1674372638:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:Debian17:1:1:341654:341654] [20230122:073038:1674372638:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:Debian17:1:1:341654:341654] [20230122:073038:1674372638:0.001] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  07:32:18 STEP  888 H= 888:00 +CPU= 85.609
  07:33:06 STEP  889 H= 889:00 +CPU= 48.197
  07:33:56 STEP  890 H= 890:00 +CPU= 48.992
  07:35:05 STEP  891 H= 891:00 +CPU= 68.355
  07:35:54 STEP  892 H= 892:00 +CPU= 48.945
  07:36:43 STEP  893 H= 893:00 +CPU= 48.134
  07:37:54 STEP  894 H= 894:00 +CPU= 69.927
  07:38:42 STEP  895 H= 895:00 +CPU= 48.085
  07:39:34 STEP  896 H= 896:00 +CPU= 50.687
  07:40:47 STEP  897 H= 897:00 +CPU= 72.148
  07:41:38 STEP  898 H= 898:00 +CPU= 49.809
  07:42:27 STEP  899 H= 899:00 +CPU= 48.486
  07:43:44 STEP  900 H= 900:00 +CPU= 75.633
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000900
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000900
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000900
  07:44:32 STEP  901 H= 901:00 +CPU= 48.191
  07:45:23 STEP  902 H= 902:00 +CPU= 49.531
  07:46:31 STEP  903 H= 903:00 +CPU= 67.796
  07:47:22 STEP  904 H= 904:00 +CPU= 50.335
  07:48:14 STEP  905 H= 905:00 +CPU= 50.761
  07:49:26 STEP  906 H= 906:00 +CPU= 71.261
  07:50:18 STEP  907 H= 907:00 +CPU= 51.059
  07:51:13 STEP  908 H= 908:00 +CPU= 51.096
  07:52:27 STEP  909 H= 909:00 +CPU= 72.510
  07:53:21 STEP  910 H= 910:00 +CPU= 53.453
  07:54:14 STEP  911 H= 911:00 +CPU= 51.916
  07:55:48 STEP  912 H= 912:00 +CPU= 86.499
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000912
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000912
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000912
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMGGhq0f+000888
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMSHhq0f+000888
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMUAhq0f+000888
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMGGhq0f+000900
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMSHhq0f+000900
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMUAhq0f+000900
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0269_1997050100_123_966_12182913_1_r433506247_37.zip
Uploading the intermediate file: upload_file_37.zip
  07:56:44 STEP  913 H= 913:00 +CPU= 54.440
  07:57:36 STEP  914 H= 914:00 +CPU= 51.755
  07:58:48 STEP  915 H= 915:00 +CPU= 70.617
  07:59:44 STEP  916 H= 916:00 +CPU= 54.808
  08:00:35 STEP  917 H= 917:00 +CPU= 50.427
  08:01:45 STEP  918 H= 918:00 +CPU= 68.858
  08:02:36 STEP  919 H= 919:00 +CPU= 50.791
  08:03:30 STEP  920 H= 920:00 +CPU= 52.813
  08:04:43 STEP  921 H= 921:00 +CPU= 72.039
  08:05:35 STEP  922 H= 922:00 +CPU= 51.289
  08:06:27 STEP  923 H= 923:00 +CPU= 51.342
  08:07:49 STEP  924 H= 924:00 +CPU= 80.120
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000924
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000924
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000924
  08:08:41 STEP  925 H= 925:00 +CPU= 51.590
  08:09:35 STEP  926 H= 926:00 +CPU= 53.330
  08:10:47 STEP  927 H= 927:00 +CPU= 71.090
  08:11:41 STEP  928 H= 928:00 +CPU= 52.728
  08:12:33 STEP  929 H= 929:00 +CPU= 51.558
  08:13:48 STEP  930 H= 930:00 +CPU= 73.238
  08:14:39 STEP  931 H= 931:00 +CPU= 50.900
  08:15:33 STEP  932 H= 932:00 +CPU= 52.742
  08:16:45 STEP  933 H= 933:00 +CPU= 71.197
  08:17:38 STEP  934 H= 934:00 +CPU= 51.682
  08:18:30 STEP  935 H= 935:00 +CPU= 51.142
  08:19:50 STEP  936 H= 936:00 +CPU= 77.768
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000936
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000936
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000936
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMGGhq0f+000912
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMSHhq0f+000912
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMUAhq0f+000912
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMGGhq0f+000924
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMSHhq0f+000924
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMUAhq0f+000924
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0269_1997050100_123_966_12182913_1_r433506247_38.zip
Uploading the intermediate file: upload_file_38.zip
  08:20:38 STEP  937 H= 937:00 +CPU= 46.790
  08:21:28 STEP  938 H= 938:00 +CPU= 50.080
  08:22:34 STEP  939 H= 939:00 +CPU= 64.213
  08:23:23 STEP  940 H= 940:00 +CPU= 49.063
  08:24:13 STEP  941 H= 941:00 +CPU= 48.735
  08:25:25 STEP  942 H= 942:00 +CPU= 70.739
  08:26:11 STEP  943 H= 943:00 +CPU= 45.195
  08:26:57 STEP  944 H= 944:00 +CPU= 46.473
  08:28:05 STEP  945 H= 945:00 +CPU= 66.926
  08:28:57 STEP  946 H= 946:00 +CPU= 50.412
  08:29:48 STEP  947 H= 947:00 +CPU= 50.469
  08:31:13 STEP  948 H= 948:00 +CPU= 82.776
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000948
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000948
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000948
  08:32:07 STEP  949 H= 949:00 +CPU= 53.117
  08:33:03 STEP  950 H= 950:00 +CPU= 55.662
  08:34:20 STEP  951 H= 951:00 +CPU= 75.681
  08:35:16 STEP  952 H= 952:00 +CPU= 54.833
  08:36:11 STEP  953 H= 953:00 +CPU= 53.983
  08:37:29 STEP  954 H= 954:00 +CPU= 76.498
  08:38:22 STEP  955 H= 955:00 +CPU= 52.849
  08:39:19 STEP  956 H= 956:00 +CPU= 55.561
  08:40:33 STEP  957 H= 957:00 +CPU= 72.251
  08:41:24 STEP  958 H= 958:00 +CPU= 50.595
  08:42:19 STEP  959 H= 959:00 +CPU= 53.053
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 1997050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0269
(argv4) batchid: 966
(argv5) wuid: 12182913
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/4
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/4/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/4/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_2d1a65e9345c940e9cc5b0f67312eb4c to: /var/lib/boinc-client/slots/4/oifs_43r3_ps_0269_1997050100_123_966_12182913.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/4/oifs_43r3_ps_0269_1997050100_123_966_12182913.zip
ic_ancil_file: ic_ancil_12182913
ifsdata_file: ifsdata_12182913
climate_data_file: clim_data_12182913
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_bae4e80c3f46772533a2e95c7515ffa2 to: /var/lib/boinc-client/slots/4/ic_ancil_12182913.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/4/ic_ancil_12182913.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_58d882547242fb8ea5840ceb3f29633e to: /var/lib/boinc-client/slots/4/ifsdata/ifsdata_12182913.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/4/ifsdata/ifsdata_12182913.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/4/159l_2/clim_data_12182913.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/4/159l_2/clim_data_12182913.zip
Checking for progress XML file: /var/lib/boinc-client/slots/4/progress_file_12182913.xml
Opened progress file ok : /var/lib/boinc-client/slots/4/progress_file_12182913.xml
-- Model is restarting --
Adjusting last_iter, 960, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/4/progress_file_12182913.xml
last_cpu_time: 62943
upload_file_number: 39
last_iter: 959
last_upload: 3369600
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0269_1997050100_123_966_12182913_1_r433506247
The child process has been launched with process id: 341877
Executing the command: /var/lib/boinc-client/slots/4/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:Debian17:1:1:341877:341877] [20230122:084909:1674377349:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:Debian17:1:1:341877:341877] [20230122:084909:1674377349:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:Debian17:1:1:341877:341877] [20230122:084909:1674377349:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:Debian17:1:1:341877:341877] [20230122:084909:1674377349:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:Debian17:1:1:341877:341877] [20230122:084909:1674377349:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:Debian17:1:1:341877:341877] [20230122:084909:1674377349:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:Debian17:1:1:341877:341877] [20230122:084909:1674377349:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:Debian17:1:1:341877:341877] [20230122:084909:1674377349:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:Debian17:1:1:341877:341877] [20230122:084909:1674377349:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:Debian17:1:1:341877:341877] [20230122:084909:1674377349:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:Debian17:1:1:341877:341877] [20230122:084909:1674377349:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:Debian17:1:1:341877:341877] [20230122:084909:1674377349:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:Debian17:1:1:341877:341877] [20230122:084909:1674377349:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:Debian17:1:1:341877:341877] [20230122:084909:1674377349:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:Debian17:1:1:341877:341877] [20230122:084909:1674377349:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:Debian17:1:1:341877:341877] [20230122:084909:1674377349:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Debian17:1:1:341877:341877] [20230122:084909:1674377349:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:Debian17:1:1:341877:341877] [20230122:084909:1674377349:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Debian17:1:1:341877:341877] [20230122:084909:1674377349:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:Debian17:1:1:341877:341877] [20230122:084909:1674377349:0.001] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:Debian17:1:1:341877:341877] [20230122:084909:1674377349:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Debian17:1:1:341877:341877] [20230122:084909:1674377349:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Debian17:1:1:341877:341877] [20230122:084909:1674377349:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:Debian17:1:1:341877:341877] [20230122:084909:1674377349:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Debian17:1:1:341877:341877] [20230122:084909:1674377349:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Debian17:1:1:341877:341877] [20230122:084909:1674377349:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Debian17:1:1:341877:341877] [20230122:084909:1674377349:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:Debian17:1:1:341877:341877] [20230122:084909:1674377349:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:Debian17:1:1:341877:341877] [20230122:084909:1674377349:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:Debian17:1:1:341877:341877] [20230122:084909:1674377349:0.001] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  08:50:37 STEP  960 H= 960:00 +CPU= 78.105
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000960
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000960
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000960
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMGGhq0f+000936
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMSHhq0f+000936
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMUAhq0f+000936
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMGGhq0f+000948
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMSHhq0f+000948
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMUAhq0f+000948
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0269_1997050100_123_966_12182913_1_r433506247_39.zip
Uploading the intermediate file: upload_file_39.zip
  08:51:22 STEP  961 H= 961:00 +CPU= 42.916
  08:52:08 STEP  962 H= 962:00 +CPU= 44.325
  08:53:08 STEP  963 H= 963:00 +CPU= 58.842
  08:53:48 STEP  964 H= 964:00 +CPU= 40.315
  08:54:33 STEP  965 H= 965:00 +CPU= 44.588
  08:55:38 STEP  966 H= 966:00 +CPU= 61.960
  08:56:20 STEP  967 H= 967:00 +CPU= 41.008
  08:57:07 STEP  968 H= 968:00 +CPU= 46.357
  08:58:11 STEP  969 H= 969:00 +CPU= 60.970
  08:58:53 STEP  970 H= 970:00 +CPU= 42.071
  08:59:38 STEP  971 H= 971:00 +CPU= 44.658
  09:00:50 STEP  972 H= 972:00 +CPU= 69.758
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000972
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000972
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000972
  09:01:28 STEP  973 H= 973:00 +CPU= 38.401
  09:02:13 STEP  974 H= 974:00 +CPU= 44.188
  09:03:21 STEP  975 H= 975:00 +CPU= 65.684
  09:04:05 STEP  976 H= 976:00 +CPU= 43.665
  09:04:54 STEP  977 H= 977:00 +CPU= 48.022
  09:06:05 STEP  978 H= 978:00 +CPU= 70.913
  09:06:51 STEP  979 H= 979:00 +CPU= 45.089
  09:07:40 STEP  980 H= 980:00 +CPU= 48.940
  09:08:51 STEP  981 H= 981:00 +CPU= 69.988
  09:09:38 STEP  982 H= 982:00 +CPU= 46.450
  09:10:28 STEP  983 H= 983:00 +CPU= 49.497
  09:11:56 STEP  984 H= 984:00 +CPU= 82.901
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000984
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000984
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000984
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMGGhq0f+000960
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMSHhq0f+000960
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMUAhq0f+000960
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMGGhq0f+000972
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMSHhq0f+000972
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182913/ICMUAhq0f+000972
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0269_1997050100_123_966_12182913_1_r433506247_40.zip
Uploading the intermediate file: upload_file_40.zip
  09:12:42 STEP  985 H= 985:00 +CPU= 45.161
Abort request received from BOINC client, ending the child process
..Failed, model did not complete successfully

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 May 2023 16:34:51 1491072 22301097 oifs_43r3_ps_0269_1997050100_123_966_12182913_1 2,764,800 51,271 0.0185
09 May 2023 16:34:26 1491072 22301097 oifs_43r3_ps_0269_1997050100_123_966_12182913_1 1,900,800 35,670 0.0188


©2024 climateprediction.net