climateprediction.net home page
Task 22449337

Task 22449337

Name oifs_43r3_bl_a1bp_2016092300_20_1018_12290044_2
Workunit 12290044
Created 16 Jun 2024, 1:00:24 UTC
Sent 16 Jun 2024, 1:00:25 UTC
Report deadline 15 Aug 2024, 1:00:25 UTC
Received 16 Jun 2024, 6:03:49 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 1 (0x00000001) Unknown error code
Computer ID 1549176
Run time 4 hours 58 min 1 sec
CPU time 4 hours 55 min 39 sec
Validate state Invalid
Credit 1,318.46
Device peak FLOPS 6.67 GFLOPS
Application version OpenIFS 43r3 Baroclinic Lifecycle v1.13
x86_64-pc-linux-gnu
Peak working set size 5,025.09 MB
Peak swap size 5,981.07 MB
Peak disk usage 1,286.57 MB
Stderr
<core_client_version>7.20.5</core_client_version>
<![CDATA[
<message>
process exited with code 1 (0x1, -255)</message>
<stderr_txt>
rediction.net/oifs_43r3_bl_12290044/ICMSHh7zg+000492
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMUAh7zg+000492
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMGGh7zg+000504
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMSHh7zg+000504
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMUAh7zg+000504
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMGGh7zg+000516
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMSHh7zg+000516
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMUAh7zg+000516
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMGGh7zg+000528
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMSHh7zg+000528
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMUAh7zg+000528
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMGGh7zg+000540
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMSHh7zg+000540
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMUAh7zg+000540
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMGGh7zg+000552
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMSHh7zg+000552
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMUAh7zg+000552
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMGGh7zg+000564
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMSHh7zg+000564
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMUAh7zg+000564
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a1bp_2016092300_20_1018_12290044_2_r1713910281_5.zip
Uploading the intermediate file: upload_file_5.zip
  05:56:55 STEP  577 H= 144:15 +CPU= 19.284
  05:57:12 STEP  578 H= 144:30 +CPU= 17.604
  05:57:31 STEP  579 H= 144:45 +CPU= 18.948
  05:57:51 STEP  580 H= 145:00 +CPU= 19.822
  05:58:10 STEP  581 H= 145:15 +CPU= 18.874
  05:58:28 STEP  582 H= 145:30 +CPU= 17.850
  05:58:47 STEP  583 H= 145:45 +CPU= 18.534
  05:59:06 STEP  584 H= 146:00 +CPU= 19.577
  05:59:25 STEP  585 H= 146:15 +CPU= 18.139
  05:59:41 STEP  586 H= 146:30 +CPU= 16.275
  06:00:00 STEP  587 H= 146:45 +CPU= 19.490
  06:00:24 STEP  588 H= 147:00 +CPU= 23.427
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGh7zg+000588
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHh7zg+000588
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAh7zg+000588
  06:00:42 STEP  589 H= 147:15 +CPU= 18.265
  06:01:01 STEP  590 H= 147:30 +CPU= 18.767
  06:01:20 STEP  591 H= 147:45 +CPU= 18.576
  06:01:38 STEP  592 H= 148:00 +CPU= 18.539
  06:01:58 STEP  593 H= 148:15 +CPU= 19.224
  06:02:17 STEP  594 H= 148:30 +CPU= 19.367
  06:02:36 STEP  595 H= 148:45 +CPU= 19.281
  06:02:56 STEP  596 H= 149:00 +CPU= 19.278
  06:03:15 STEP  597 H= 149:15 +CPU= 19.323
  06:03:33 STEP  598 H= 149:30 +CPU= 17.953
  06:03:51 STEP  599 H= 149:45 +CPU= 17.556
  06:04:14 STEP  600 H= 150:00 +CPU= 22.922
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGh7zg+000600
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHh7zg+000600
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAh7zg+000600
  06:04:33 STEP  601 H= 150:15 +CPU= 19.026
  06:04:52 STEP  602 H= 150:30 +CPU= 19.194
  06:05:11 STEP  603 H= 150:45 +CPU= 18.668
  06:05:28 STEP  604 H= 151:00 +CPU= 17.050
  06:05:47 STEP  605 H= 151:15 +CPU= 19.264
  06:06:03 STEP  606 H= 151:30 +CPU= 15.789
  06:06:22 STEP  607 H= 151:45 +CPU= 19.249
  06:06:42 STEP  608 H= 152:00 +CPU= 19.732
  06:07:01 STEP  609 H= 152:15 +CPU= 19.096
  06:07:21 STEP  610 H= 152:30 +CPU= 19.208
  06:07:39 STEP  611 H= 152:45 +CPU= 18.214
  06:08:02 STEP  612 H= 153:00 +CPU= 22.885
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGh7zg+000612
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHh7zg+000612
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAh7zg+000612
  06:08:20 STEP  613 H= 153:15 +CPU= 18.233
  06:08:39 STEP  614 H= 153:30 +CPU= 18.699
  06:08:58 STEP  615 H= 153:45 +CPU= 18.778
  06:09:18 STEP  616 H= 154:00 +CPU= 19.961
  06:09:37 STEP  617 H= 154:15 +CPU= 18.693
  06:09:55 STEP  618 H= 154:30 +CPU= 18.014
  06:10:14 STEP  619 H= 154:45 +CPU= 18.975
  06:10:34 STEP  620 H= 155:00 +CPU= 19.624
  06:10:53 STEP  621 H= 155:15 +CPU= 19.133
  06:11:12 STEP  622 H= 155:30 +CPU= 19.040
  06:11:30 STEP  623 H= 155:45 +CPU= 18.170
  06:11:56 STEP  624 H= 156:00 +CPU= 25.588
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGh7zg+000624
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHh7zg+000624
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAh7zg+000624
  06:12:13 STEP  625 H= 156:15 +CPU= 17.171
  06:12:32 STEP  626 H= 156:30 +CPU= 18.442
  06:12:49 STEP  627 H= 156:45 +CPU= 16.790
  06:13:07 STEP  628 H= 157:00 +CPU= 18.582
  06:13:26 STEP  629 H= 157:15 +CPU= 18.167
  06:13:44 STEP  630 H= 157:30 +CPU= 17.931
  06:14:03 STEP  631 H= 157:45 +CPU= 19.351
  06:14:23 STEP  632 H= 158:00 +CPU= 19.636
  06:14:41 STEP  633 H= 158:15 +CPU= 18.593
  06:14:59 STEP  634 H= 158:30 +CPU= 18.084
  06:15:18 STEP  635 H= 158:45 +CPU= 18.171
  06:15:41 STEP  636 H= 159:00 +CPU= 23.454
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGh7zg+000636
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHh7zg+000636
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAh7zg+000636
  06:16:00 STEP  637 H= 159:15 +CPU= 18.330
  06:16:18 STEP  638 H= 159:30 +CPU= 18.497
  06:16:37 STEP  639 H= 159:45 +CPU= 19.098
  06:16:57 STEP  640 H= 160:00 +CPU= 19.918
  06:17:15 STEP  641 H= 160:15 +CPU= 17.319
  06:17:34 STEP  642 H= 160:30 +CPU= 18.810
  06:17:51 STEP  643 H= 160:45 +CPU= 17.055
  06:18:09 STEP  644 H= 161:00 +CPU= 18.564
  06:18:28 STEP  645 H= 161:15 +CPU= 19.063
  06:18:48 STEP  646 H= 161:30 +CPU= 19.039
  06:19:06 STEP  647 H= 161:45 +CPU= 18.055
  06:19:27 STEP  648 H= 162:00 +CPU= 21.348
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGh7zg+000648
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHh7zg+000648
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAh7zg+000648
  06:19:43 STEP  649 H= 162:15 +CPU= 16.366
  06:20:02 STEP  650 H= 162:30 +CPU= 18.049
  06:20:21 STEP  651 H= 162:45 +CPU= 18.877
  06:20:37 STEP  652 H= 163:00 +CPU= 16.438
  06:20:55 STEP  653 H= 163:15 +CPU= 18.062
  06:21:14 STEP  654 H= 163:30 +CPU= 18.819
  06:21:31 STEP  655 H= 163:45 +CPU= 16.604
  06:21:50 STEP  656 H= 164:00 +CPU= 19.051
  06:22:07 STEP  657 H= 164:15 +CPU= 17.574
  06:22:27 STEP  658 H= 164:30 +CPU= 19.136
  06:22:41 STEP  659 H= 164:45 +CPU= 13.928
  06:23:04 STEP  660 H= 165:00 +CPU= 23.151
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGh7zg+000660
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHh7zg+000660
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAh7zg+000660
  06:23:23 STEP  661 H= 165:15 +CPU= 19.085
  06:23:42 STEP  662 H= 165:30 +CPU= 18.505
  06:24:00 STEP  663 H= 165:45 +CPU= 18.119
  06:24:19 STEP  664 H= 166:00 +CPU= 19.605
  06:24:39 STEP  665 H= 166:15 +CPU= 19.226
  06:24:58 STEP  666 H= 166:30 +CPU= 18.732
  06:25:16 STEP  667 H= 166:45 +CPU= 18.287
  06:25:35 STEP  668 H= 167:00 +CPU= 19.141
  06:25:54 STEP  669 H= 167:15 +CPU= 18.797
  06:26:11 STEP  670 H= 167:30 +CPU= 17.217
  06:26:29 STEP  671 H= 167:45 +CPU= 17.485
  06:26:53 STEP  672 H= 168:00 +CPU= 23.417
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGh7zg+000672
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHh7zg+000672
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAh7zg+000672
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMGGh7zg+000576
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMSHh7zg+000576
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMUAh7zg+000576
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMGGh7zg+000588
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMSHh7zg+000588
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMUAh7zg+000588
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMGGh7zg+000600
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMSHh7zg+000600
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMUAh7zg+000600
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMGGh7zg+000612
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMSHh7zg+000612
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMUAh7zg+000612
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMGGh7zg+000624
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMSHh7zg+000624
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMUAh7zg+000624
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMGGh7zg+000636
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMSHh7zg+000636
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMUAh7zg+000636
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMGGh7zg+000648
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMSHh7zg+000648
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMUAh7zg+000648
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMGGh7zg+000660
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMSHh7zg+000660
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMUAh7zg+000660
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a1bp_2016092300_20_1018_12290044_2_r1713910281_6.zip
Uploading the intermediate file: upload_file_6.zip
  06:27:12 STEP  673 H= 168:15 +CPU= 19.080
  06:27:30 STEP  674 H= 168:30 +CPU= 17.843
  06:27:49 STEP  675 H= 168:45 +CPU= 19.057
  06:28:09 STEP  676 H= 169:00 +CPU= 19.905
  06:28:27 STEP  677 H= 169:15 +CPU= 18.058
  06:28:46 STEP  678 H= 169:30 +CPU= 18.610
  06:29:03 STEP  679 H= 169:45 +CPU= 17.162
  06:29:21 STEP  680 H= 170:00 +CPU= 18.223
  06:29:39 STEP  681 H= 170:15 +CPU= 17.529
  06:29:57 STEP  682 H= 170:30 +CPU= 18.471
  06:30:16 STEP  683 H= 170:45 +CPU= 18.597
  06:30:39 STEP  684 H= 171:00 +CPU= 22.597
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGh7zg+000684
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHh7zg+000684
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAh7zg+000684
  06:30:57 STEP  685 H= 171:15 +CPU= 18.514
  06:31:16 STEP  686 H= 171:30 +CPU= 18.204
  06:31:33 STEP  687 H= 171:45 +CPU= 17.795
  06:31:53 STEP  688 H= 172:00 +CPU= 19.337
  06:32:11 STEP  689 H= 172:15 +CPU= 18.422
  06:32:29 STEP  690 H= 172:30 +CPU= 17.918
  06:32:47 STEP  691 H= 172:45 +CPU= 17.489
  06:33:06 STEP  692 H= 173:00 +CPU= 19.558
  06:33:22 STEP  693 H= 173:15 +CPU= 15.868
  06:33:39 STEP  694 H= 173:30 +CPU= 16.865
  06:33:56 STEP  695 H= 173:45 +CPU= 17.108
  06:34:20 STEP  696 H= 174:00 +CPU= 23.250
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGh7zg+000696
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHh7zg+000696
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAh7zg+000696
  06:34:39 STEP  697 H= 174:15 +CPU= 18.926
  06:34:58 STEP  698 H= 174:30 +CPU= 18.895
  06:35:15 STEP  699 H= 174:45 +CPU= 17.084
  06:35:32 STEP  700 H= 175:00 +CPU= 17.526
  06:35:50 STEP  701 H= 175:15 +CPU= 17.530
  06:36:09 STEP  702 H= 175:30 +CPU= 18.900
  06:36:27 STEP  703 H= 175:45 +CPU= 18.117
  06:36:47 STEP  704 H= 176:00 +CPU= 19.496
  06:37:04 STEP  705 H= 176:15 +CPU= 17.124
  06:37:22 STEP  706 H= 176:30 +CPU= 17.830
  06:37:41 STEP  707 H= 176:45 +CPU= 19.025
  06:38:04 STEP  708 H= 177:00 +CPU= 23.239
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGh7zg+000708
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHh7zg+000708
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAh7zg+000708
  06:38:23 STEP  709 H= 177:15 +CPU= 18.839
  06:38:38 STEP  710 H= 177:30 +CPU= 15.235
  06:38:57 STEP  711 H= 177:45 +CPU= 18.844
  06:39:17 STEP  712 H= 178:00 +CPU= 19.521
  06:39:35 STEP  713 H= 178:15 +CPU= 18.119
  06:39:53 STEP  714 H= 178:30 +CPU= 17.574
  06:40:11 STEP  715 H= 178:45 +CPU= 18.581
  06:40:30 STEP  716 H= 179:00 +CPU= 18.069
  06:40:48 STEP  717 H= 179:15 +CPU= 17.929
  06:41:04 STEP  718 H= 179:30 +CPU= 16.498
  06:41:23 STEP  719 H= 179:45 +CPU= 18.452
  06:41:47 STEP  720 H= 180:00 +CPU= 24.368
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGh7zg+000720
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHh7zg+000720
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAh7zg+000720
  06:42:05 STEP  721 H= 180:15 +CPU= 17.765
  06:42:24 STEP  722 H= 180:30 +CPU= 18.346
  06:42:42 STEP  723 H= 180:45 +CPU= 18.920
  06:43:01 STEP  724 H= 181:00 +CPU= 18.706
  06:43:19 STEP  725 H= 181:15 +CPU= 18.114
  06:43:38 STEP  726 H= 181:30 +CPU= 17.986
  06:43:57 STEP  727 H= 181:45 +CPU= 19.004
  06:44:15 STEP  728 H= 182:00 +CPU= 17.902
  06:44:33 STEP  729 H= 182:15 +CPU= 18.483
  06:44:52 STEP  730 H= 182:30 +CPU= 18.906
  06:45:11 STEP  731 H= 182:45 +CPU= 19.344
  06:45:35 STEP  732 H= 183:00 +CPU= 23.019
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGh7zg+000732
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHh7zg+000732
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAh7zg+000732
  06:45:53 STEP  733 H= 183:15 +CPU= 18.608
  06:46:12 STEP  734 H= 183:30 +CPU= 18.749
  06:46:31 STEP  735 H= 183:45 +CPU= 18.844
  06:46:50 STEP  736 H= 184:00 +CPU= 19.193
  06:47:08 STEP  737 H= 184:15 +CPU= 18.060
  06:47:27 STEP  738 H= 184:30 +CPU= 18.441
  06:47:44 STEP  739 H= 184:45 +CPU= 17.427
  06:48:04 STEP  740 H= 185:00 +CPU= 19.811
  06:48:23 STEP  741 H= 185:15 +CPU= 18.444
  06:48:39 STEP  742 H= 185:30 +CPU= 16.354
  06:48:57 STEP  743 H= 185:45 +CPU= 17.885
  06:49:21 STEP  744 H= 186:00 +CPU= 23.873
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGh7zg+000744
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHh7zg+000744
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAh7zg+000744
  06:49:38 STEP  745 H= 186:15 +CPU= 16.628
  06:49:55 STEP  746 H= 186:30 +CPU= 17.508
  06:50:13 STEP  747 H= 186:45 +CPU= 17.937
  06:50:29 STEP  748 H= 187:00 +CPU= 15.866
  06:50:46 STEP  749 H= 187:15 +CPU= 16.705
  06:51:04 STEP  750 H= 187:30 +CPU= 18.240
  06:51:22 STEP  751 H= 187:45 +CPU= 17.850
  06:51:40 STEP  752 H= 188:00 +CPU= 17.775
  06:51:59 STEP  753 H= 188:15 +CPU= 18.506
  06:52:16 STEP  754 H= 188:30 +CPU= 17.501
  06:52:35 STEP  755 H= 188:45 +CPU= 18.862
  06:52:58 STEP  756 H= 189:00 +CPU= 22.624
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGh7zg+000756
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHh7zg+000756
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAh7zg+000756
  06:53:15 STEP  757 H= 189:15 +CPU= 17.461
  06:53:34 STEP  758 H= 189:30 +CPU= 18.962
  06:53:52 STEP  759 H= 189:45 +CPU= 18.120
  06:54:11 STEP  760 H= 190:00 +CPU= 18.939
  06:54:29 STEP  761 H= 190:15 +CPU= 17.048
  06:54:47 STEP  762 H= 190:30 +CPU= 18.028
  06:55:05 STEP  763 H= 190:45 +CPU= 18.093
  06:55:22 STEP  764 H= 191:00 +CPU= 17.271
  06:55:41 STEP  765 H= 191:15 +CPU= 18.503
  06:56:00 STEP  766 H= 191:30 +CPU= 18.915
  06:56:18 STEP  767 H= 191:45 +CPU= 18.125
  06:56:42 STEP  768 H= 192:00 +CPU= 24.051
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGh7zg+000768
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHh7zg+000768
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAh7zg+000768
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMGGh7zg+000672
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMSHh7zg+000672
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMUAh7zg+000672
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMGGh7zg+000684
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMSHh7zg+000684
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMUAh7zg+000684
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMGGh7zg+000696
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMSHh7zg+000696
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMUAh7zg+000696
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMGGh7zg+000708
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMSHh7zg+000708
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMUAh7zg+000708
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMGGh7zg+000720
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMSHh7zg+000720
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMUAh7zg+000720
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMGGh7zg+000732
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMSHh7zg+000732
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMUAh7zg+000732
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMGGh7zg+000744
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMSHh7zg+000744
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMUAh7zg+000744
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMGGh7zg+000756
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMSHh7zg+000756
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMUAh7zg+000756
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a1bp_2016092300_20_1018_12290044_2_r1713910281_7.zip
Uploading the intermediate file: upload_file_7.zip
  06:57:00 STEP  769 H= 192:15 +CPU= 18.143
  06:57:18 STEP  770 H= 192:30 +CPU= 18.021
  06:57:37 STEP  771 H= 192:45 +CPU= 18.718
  06:57:55 STEP  772 H= 193:00 +CPU= 17.746
  06:58:12 STEP  773 H= 193:15 +CPU= 17.130
  06:58:28 STEP  774 H= 193:30 +CPU= 16.046
  06:58:45 STEP  775 H= 193:45 +CPU= 16.722
  06:59:02 STEP  776 H= 194:00 +CPU= 16.725
  06:59:18 STEP  777 H= 194:15 +CPU= 16.267
  06:59:34 STEP  778 H= 194:30 +CPU= 15.989
  06:59:48 STEP  779 H= 194:45 +CPU= 13.481
  07:00:09 STEP  780 H= 195:00 +CPU= 21.249
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGh7zg+000780
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHh7zg+000780
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAh7zg+000780
  07:00:28 STEP  781 H= 195:15 +CPU= 18.248
  07:00:44 STEP  782 H= 195:30 +CPU= 15.931
  07:00:59 STEP  783 H= 195:45 +CPU= 14.253
  07:01:14 STEP  784 H= 196:00 +CPU= 14.891
  07:01:29 STEP  785 H= 196:15 +CPU= 15.648
  07:01:46 STEP  786 H= 196:30 +CPU= 16.207
  07:02:01 STEP  787 H= 196:45 +CPU= 14.901
  07:02:17 STEP  788 H= 197:00 +CPU= 16.545
  07:02:32 STEP  789 H= 197:15 +CPU= 14.091
  07:02:46 STEP  790 H= 197:30 +CPU= 14.147
  07:03:00 STEP  791 H= 197:45 +CPU= 14.718
  07:03:19 STEP  792 H= 198:00 +CPU= 18.890
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGh7zg+000792
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHh7zg+000792
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAh7zg+000792
  07:03:34 STEP  793 H= 198:15 +CPU= 14.759
  07:03:48 STEP  794 H= 198:30 +CPU= 13.378
  07:04:04 STEP  795 H= 198:45 +CPU= 16.397
  07:04:19 STEP  796 H= 199:00 +CPU= 15.283
  07:04:33 STEP  797 H= 199:15 +CPU= 14.041
  07:04:47 STEP  798 H= 199:30 +CPU= 13.703
  07:05:01 STEP  799 H= 199:45 +CPU= 13.750
  07:05:16 STEP  800 H= 200:00 +CPU= 15.331
  07:05:32 STEP  801 H= 200:15 +CPU= 15.948
  07:05:46 STEP  802 H= 200:30 +CPU= 13.317
  07:06:02 STEP  803 H= 200:45 +CPU= 16.384
  07:06:23 STEP  804 H= 201:00 +CPU= 21.279
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGh7zg+000804
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHh7zg+000804
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAh7zg+000804
  07:06:42 STEP  805 H= 201:15 +CPU= 18.632
  07:06:57 STEP  806 H= 201:30 +CPU= 14.889
  07:07:11 STEP  807 H= 201:45 +CPU= 14.336
  07:07:28 STEP  808 H= 202:00 +CPU= 16.227
  07:07:41 STEP  809 H= 202:15 +CPU= 13.561
  07:07:56 STEP  810 H= 202:30 +CPU= 14.959
  07:08:10 STEP  811 H= 202:45 +CPU= 13.619
  07:08:25 STEP  812 H= 203:00 +CPU= 14.537
  07:08:39 STEP  813 H= 203:15 +CPU= 14.753
  07:08:53 STEP  814 H= 203:30 +CPU= 13.938
  07:09:07 STEP  815 H= 203:45 +CPU= 13.208
  07:09:26 STEP  816 H= 204:00 +CPU= 19.333
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGh7zg+000816
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHh7zg+000816
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAh7zg+000816
  07:09:40 STEP  817 H= 204:15 +CPU= 14.065
  07:09:53 STEP  818 H= 204:30 +CPU= 13.164
  07:10:09 STEP  819 H= 204:45 +CPU= 15.916
  07:10:25 STEP  820 H= 205:00 +CPU= 15.475
  07:10:42 STEP  821 H= 205:15 +CPU= 16.917
  07:10:57 STEP  822 H= 205:30 +CPU= 15.165
  07:11:14 STEP  823 H= 205:45 +CPU= 16.404
  07:11:31 STEP  824 H= 206:00 +CPU= 16.849
  07:11:49 STEP  825 H= 206:15 +CPU= 18.191
  07:12:03 STEP  826 H= 206:30 +CPU= 13.768
  07:12:16 STEP  827 H= 206:45 +CPU= 13.440
  07:12:34 STEP  828 H= 207:00 +CPU= 17.784
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGh7zg+000828
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHh7zg+000828
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAh7zg+000828
  07:12:50 STEP  829 H= 207:15 +CPU= 16.508
  07:13:05 STEP  830 H= 207:30 +CPU= 14.320
  07:13:19 STEP  831 H= 207:45 +CPU= 13.954
  07:13:33 STEP  832 H= 208:00 +CPU= 14.192
  07:13:50 STEP  833 H= 208:15 +CPU= 16.616
  07:14:07 STEP  834 H= 208:30 +CPU= 17.202
  07:14:24 STEP  835 H= 208:45 +CPU= 16.891
  07:14:43 STEP  836 H= 209:00 +CPU= 19.391
  07:15:02 STEP  837 H= 209:15 +CPU= 18.267
  07:15:21 STEP  838 H= 209:30 +CPU= 18.741
  07:15:39 STEP  839 H= 209:45 +CPU= 18.496
  07:16:02 STEP  840 H= 210:00 +CPU= 23.112
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGh7zg+000840
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHh7zg+000840
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAh7zg+000840
  07:16:19 STEP  841 H= 210:15 +CPU= 16.495
  07:16:38 STEP  842 H= 210:30 +CPU= 18.704
  07:16:56 STEP  843 H= 210:45 +CPU= 18.047
  07:17:14 STEP  844 H= 211:00 +CPU= 17.834
  07:17:32 STEP  845 H= 211:15 +CPU= 18.657
  07:17:50 STEP  846 H= 211:30 +CPU= 17.642
  07:18:08 STEP  847 H= 211:45 +CPU= 18.089
  07:18:25 STEP  848 H= 212:00 +CPU= 16.960
  07:18:44 STEP  849 H= 212:15 +CPU= 18.249
  07:19:02 STEP  850 H= 212:30 +CPU= 18.352
  07:19:20 STEP  851 H= 212:45 +CPU= 18.382
  07:19:43 STEP  852 H= 213:00 +CPU= 22.342
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGh7zg+000852
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHh7zg+000852
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAh7zg+000852
  07:20:00 STEP  853 H= 213:15 +CPU= 16.922
  07:20:17 STEP  854 H= 213:30 +CPU= 17.639
  07:20:36 STEP  855 H= 213:45 +CPU= 18.735
  07:20:55 STEP  856 H= 214:00 +CPU= 18.175
  07:21:13 STEP  857 H= 214:15 +CPU= 18.694
  07:21:30 STEP  858 H= 214:30 +CPU= 16.168
  07:21:48 STEP  859 H= 214:45 +CPU= 18.520
  07:22:08 STEP  860 H= 215:00 +CPU= 19.591
  07:22:27 STEP  861 H= 215:15 +CPU= 18.916
  07:22:44 STEP  862 H= 215:30 +CPU= 17.599
  07:23:02 STEP  863 H= 215:45 +CPU= 17.257
  07:23:26 STEP  864 H= 216:00 +CPU= 24.040
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGh7zg+000864
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHh7zg+000864
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAh7zg+000864
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMGGh7zg+000768
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMSHh7zg+000768
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMUAh7zg+000768
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMGGh7zg+000780
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMSHh7zg+000780
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMUAh7zg+000780
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMGGh7zg+000792
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMSHh7zg+000792
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMUAh7zg+000792
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMGGh7zg+000804
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMSHh7zg+000804
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMUAh7zg+000804
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMGGh7zg+000816
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMSHh7zg+000816
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMUAh7zg+000816
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMGGh7zg+000828
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMSHh7zg+000828
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMUAh7zg+000828
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMGGh7zg+000840
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMSHh7zg+000840
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMUAh7zg+000840
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMGGh7zg+000852
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMSHh7zg+000852
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMUAh7zg+000852
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a1bp_2016092300_20_1018_12290044_2_r1713910281_8.zip
Uploading the intermediate file: upload_file_8.zip
  07:23:43 STEP  865 H= 216:15 +CPU= 17.296
  07:24:00 STEP  866 H= 216:30 +CPU= 17.057
  07:24:19 STEP  867 H= 216:45 +CPU= 18.920
  07:24:39 STEP  868 H= 217:00 +CPU= 19.346
  07:24:57 STEP  869 H= 217:15 +CPU= 18.017
  07:25:16 STEP  870 H= 217:30 +CPU= 18.515
  07:25:33 STEP  871 H= 217:45 +CPU= 17.752
  07:25:51 STEP  872 H= 218:00 +CPU= 17.840
  07:26:09 STEP  873 H= 218:15 +CPU= 17.919
  07:26:28 STEP  874 H= 218:30 +CPU= 18.953
  07:26:48 STEP  875 H= 218:45 +CPU= 19.145
  07:27:11 STEP  876 H= 219:00 +CPU= 23.092
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGh7zg+000876
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHh7zg+000876
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAh7zg+000876
  07:27:28 STEP  877 H= 219:15 +CPU= 17.582
  07:27:46 STEP  878 H= 219:30 +CPU= 17.782
  07:28:05 STEP  879 H= 219:45 +CPU= 18.884
  07:28:24 STEP  880 H= 220:00 +CPU= 18.699
  07:28:40 STEP  881 H= 220:15 +CPU= 15.575
  07:28:59 STEP  882 H= 220:30 +CPU= 18.857
  07:29:16 STEP  883 H= 220:45 +CPU= 17.464
  07:29:34 STEP  884 H= 221:00 +CPU= 18.278
  07:29:50 STEP  885 H= 221:15 +CPU= 15.966
  07:30:09 STEP  886 H= 221:30 +CPU= 18.076
  07:30:26 STEP  887 H= 221:45 +CPU= 17.464
  07:30:50 STEP  888 H= 222:00 +CPU= 24.194
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGh7zg+000888
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHh7zg+000888
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAh7zg+000888
  07:31:08 STEP  889 H= 222:15 +CPU= 17.808
  07:31:26 STEP  890 H= 222:30 +CPU= 17.669
  07:31:45 STEP  891 H= 222:45 +CPU= 18.982
  07:32:04 STEP  892 H= 223:00 +CPU= 19.131
  07:32:22 STEP  893 H= 223:15 +CPU= 17.519
  07:32:41 STEP  894 H= 223:30 +CPU= 18.923
  07:32:59 STEP  895 H= 223:45 +CPU= 17.952
  07:33:18 STEP  896 H= 224:00 +CPU= 18.859
  07:33:36 STEP  897 H= 224:15 +CPU= 18.332
  07:33:55 STEP  898 H= 224:30 +CPU= 18.686
  07:34:13 STEP  899 H= 224:45 +CPU= 18.188
  07:34:31 STEP  900 H= 225:00 +CPU= 17.857
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGh7zg+000900
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHh7zg+000900
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAh7zg+000900
  07:34:45 STEP  901 H= 225:15 +CPU= 13.946
  07:34:59 STEP  902 H= 225:30 +CPU= 13.791
  07:35:12 STEP  903 H= 225:45 +CPU= 13.454
  07:35:27 STEP  904 H= 226:00 +CPU= 15.051
  07:35:40 STEP  905 H= 226:15 +CPU= 12.871
  07:35:57 STEP  906 H= 226:30 +CPU= 16.448
  07:36:15 STEP  907 H= 226:45 +CPU= 17.548
  07:36:32 STEP  908 H= 227:00 +CPU= 17.141
  07:36:47 STEP  909 H= 227:15 +CPU= 14.335
  07:37:03 STEP  910 H= 227:30 +CPU= 16.606
  07:37:20 STEP  911 H= 227:45 +CPU= 16.674
  07:37:43 STEP  912 H= 228:00 +CPU= 22.798
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGh7zg+000912
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHh7zg+000912
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAh7zg+000912
  07:38:00 STEP  913 H= 228:15 +CPU= 16.462
  07:38:15 STEP  914 H= 228:30 +CPU= 15.012
  07:38:29 STEP  915 H= 228:45 +CPU= 13.718
  07:38:42 STEP  916 H= 229:00 +CPU= 13.733
  07:38:58 STEP  917 H= 229:15 +CPU= 15.174
  07:39:11 STEP  918 H= 229:30 +CPU= 13.686
  07:39:25 STEP  919 H= 229:45 +CPU= 13.275
  07:39:38 STEP  920 H= 230:00 +CPU= 13.576
  07:39:54 STEP  921 H= 230:15 +CPU= 15.304
  07:40:08 STEP  922 H= 230:30 +CPU= 14.147
  07:40:25 STEP  923 H= 230:45 +CPU= 16.776
  07:40:47 STEP  924 H= 231:00 +CPU= 21.844
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGh7zg+000924
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHh7zg+000924
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAh7zg+000924
  07:41:02 STEP  925 H= 231:15 +CPU= 15.116
  07:41:18 STEP  926 H= 231:30 +CPU= 16.123
  07:41:34 STEP  927 H= 231:45 +CPU= 16.080
  07:41:53 STEP  928 H= 232:00 +CPU= 18.938
  07:42:10 STEP  929 H= 232:15 +CPU= 16.566
  07:42:26 STEP  930 H= 232:30 +CPU= 15.901
  07:42:42 STEP  931 H= 232:45 +CPU= 16.542
  07:42:56 STEP  932 H= 233:00 +CPU= 13.709
  07:43:09 STEP  933 H= 233:15 +CPU= 13.161
  07:43:25 STEP  934 H= 233:30 +CPU= 15.820
  07:43:41 STEP  935 H= 233:45 +CPU= 16.414
  07:44:05 STEP  936 H= 234:00 +CPU= 23.596
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGh7zg+000936
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHh7zg+000936
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAh7zg+000936
  07:44:20 STEP  937 H= 234:15 +CPU= 14.737
  07:44:35 STEP  938 H= 234:30 +CPU= 15.031
  07:44:49 STEP  939 H= 234:45 +CPU= 13.878
  07:45:03 STEP  940 H= 235:00 +CPU= 13.954
  07:45:18 STEP  941 H= 235:15 +CPU= 15.118
  07:45:33 STEP  942 H= 235:30 +CPU= 15.376
  07:45:50 STEP  943 H= 235:45 +CPU= 16.857
  07:46:05 STEP  944 H= 236:00 +CPU= 14.306
  07:46:18 STEP  945 H= 236:15 +CPU= 13.104
  07:46:34 STEP  946 H= 236:30 +CPU= 16.207
  07:46:49 STEP  947 H= 236:45 +CPU= 14.949
  07:47:07 STEP  948 H= 237:00 +CPU= 18.033
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGh7zg+000948
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHh7zg+000948
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAh7zg+000948
  07:47:24 STEP  949 H= 237:15 +CPU= 16.313
  07:47:42 STEP  950 H= 237:30 +CPU= 18.287
  07:47:58 STEP  951 H= 237:45 +CPU= 15.933
  07:48:14 STEP  952 H= 238:00 +CPU= 15.970
  07:48:28 STEP  953 H= 238:15 +CPU= 14.359
  07:48:47 STEP  954 H= 238:30 +CPU= 18.509
  07:49:03 STEP  955 H= 238:45 +CPU= 16.532
  07:49:19 STEP  956 H= 239:00 +CPU= 15.195
  07:49:37 STEP  957 H= 239:15 +CPU= 18.253
  07:49:55 STEP  958 H= 239:30 +CPU= 18.143
  07:50:12 STEP  959 H= 239:45 +CPU= 16.247
  07:50:38 STEP  960 H= 240:00 +CPU= 26.540
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGh7zg+000960
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHh7zg+000960
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAh7zg+000960
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMGGh7zg+000864
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMSHh7zg+000864
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMUAh7zg+000864
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMGGh7zg+000876
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMSHh7zg+000876
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMUAh7zg+000876
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMGGh7zg+000888
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMSHh7zg+000888
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMUAh7zg+000888
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMGGh7zg+000900
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMSHh7zg+000900
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMUAh7zg+000900
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMGGh7zg+000912
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMSHh7zg+000912
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMUAh7zg+000912
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMGGh7zg+000924
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMSHh7zg+000924
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMUAh7zg+000924
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMGGh7zg+000936
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMSHh7zg+000936
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMUAh7zg+000936
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMGGh7zg+000948
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMSHh7zg+000948
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290044/ICMUAh7zg+000948
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a1bp_2016092300_20_1018_12290044_2_r1713910281_9.zip
Uploading the intermediate file: upload_file_9.zip
  07:50:56 STEP  961 H= 240:15 +CPU= 17.687
Uploading trickle at timestep: 864000
  07:51:12 STEP  962 H= 240:30 +CPU= 15.787
  07:51:29 STEP  963 H= 240:45 +CPU= 16.720
  07:51:48 STEP  964 H= 241:00 +CPU= 19.435
  07:52:07 STEP  965 H= 241:15 +CPU= 18.594
  07:52:24 STEP  966 H= 241:30 +CPU= 16.612
  07:52:42 STEP  967 H= 241:45 +CPU= 18.833
  07:53:02 STEP  968 H= 242:00 +CPU= 19.052
  07:53:21 STEP  969 H= 242:15 +CPU= 19.059
  07:53:40 STEP  970 H= 242:30 +CPU= 19.048
  07:53:56 STEP  971 H= 242:45 +CPU= 16.164
  07:54:16 STEP  972 H= 243:00 +CPU= 20.407
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGh7zg+000972
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHh7zg+000972
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAh7zg+000972
  07:54:34 STEP  973 H= 243:15 +CPU= 17.674
  07:54:53 STEP  974 H= 243:30 +CPU= 18.705
  07:55:11 STEP  975 H= 243:45 +CPU= 18.432
  07:55:29 STEP  976 H= 244:00 +CPU= 17.927
  07:55:45 STEP  977 H= 244:15 +CPU= 15.040
  07:56:03 STEP  978 H= 244:30 +CPU= 18.080
  07:56:21 STEP  979 H= 244:45 +CPU= 18.112
  07:56:39 STEP  980 H= 245:00 +CPU= 18.333
  07:56:57 STEP  981 H= 245:15 +CPU= 17.503
  07:57:14 STEP  982 H= 245:30 +CPU= 16.687
[EC_DRHOOK:Ryze:1:1:3835597:3835597] [20240616:075726:1718517446:17810.597] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1538] Received signal#8 (SIGFPE) :: 4412MB (heap), 5025MB (maxrss), 0MB (maxstack), 0 (paging), nsigs = 1
[EC_DRHOOK:Ryze:1:1:3835597:3835597] [20240616:075726:1718517446:17810.597] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1542] Also activating Harakiri-alarm (SIGALRM=14) to expire after 500s elapsed to prevent hangs, nsigs = 1
[EC_DRHOOK:Ryze:1:1:3835597:3835597] [20240616:075726:1718517446:17810.598] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1544] Harakiri signal handler 'signal_harakiri' for signal#14 (SIGALRM) installed at 0x820890 (old at (nil))
[EC_DRHOOK:Ryze:1:1:3835597:3835597] [20240616:075726:1718517446:17810.598] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1617] Signal#8 was caused by floating-point overflow [memaddr=0x1cc6692], nsigs = 1
[EC_DRHOOK:Ryze:1:1:3835597:3835597] [20240616:075726:1718517446:17810.598] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1686] Starting DrHook backtrace for signal#8, nsigs = 1
[EC_DRHOOK:Ryze:1:1:3835597:3835597] [20240616:075726:1718517446:17810.598] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3843] 4412 MB (maxheap), 5025 MB (maxrss), 0 MB (maxstack)
[EC_DRHOOK:Ryze:1:1:3835597:3835597] [20240616:075726:1718517446:17810.598] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :  MASTER 
[EC_DRHOOK:Ryze:1:1:3835597:3835597] [20240616:075726:1718517446:17810.598] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :   CNT0 
[EC_DRHOOK:Ryze:1:1:3835597:3835597] [20240616:075726:1718517446:17810.598] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :    CNT1 
[EC_DRHOOK:Ryze:1:1:3835597:3835597] [20240616:075726:1718517446:17810.598] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :     CNT2 
[EC_DRHOOK:Ryze:1:1:3835597:3835597] [20240616:075726:1718517446:17810.598] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :      CNT3 
[EC_DRHOOK:Ryze:1:1:3835597:3835597] [20240616:075726:1718517446:17810.598] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :       CNT4 
[EC_DRHOOK:Ryze:1:1:3835597:3835597] [20240616:075726:1718517446:17810.598] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :        STEPO 
[EC_DRHOOK:Ryze:1:1:3835597:3835597] [20240616:075726:1718517446:17810.598] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :         SCAN2M 
[EC_DRHOOK:Ryze:1:1:3835597:3835597] [20240616:075726:1718517446:17810.598] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :          GP_MODEL_HEAP 
[EC_DRHOOK:Ryze:1:1:3835597:3835597] [20240616:075726:1718517446:17810.598] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :           GP_MODEL 
[EC_DRHOOK:Ryze:1:1:3835597:3835597] [20240616:075726:1718517446:17810.598] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :            EC_PHYS_DRV 
[EC_DRHOOK:Ryze:1:1:3835597:3835597] [20240616:075726:1718517446:17810.598] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :             >OMP-PHYSICS              (1001) 
[EC_DRHOOK:Ryze:1:1:3835597:3835597] [20240616:075726:1718517446:17810.598] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :              EC_PHYS 
[EC_DRHOOK:Ryze:1:1:3835597:3835597] [20240616:075726:1718517446:17810.598] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :               CALLPAR 
[EC_DRHOOK:Ryze:1:1:3835597:3835597] [20240616:075726:1718517446:17810.598] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :                TURBULENCE_LAYER 
[EC_DRHOOK:Ryze:1:1:3835597:3835597] [20240616:075726:1718517446:17810.598] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :                 VDFOUTER 
[EC_DRHOOK:Ryze:1:1:3835597:3835597] [20240616:075726:1718517446:17810.598] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :                  VDFMAIN 
[EC_DRHOOK:Ryze:1:1:3835597:3835597] [20240616:075727:1718517447:17811.598] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:240] Backtrace(s) for program '/var/lib/boinc-client/slots/5/oifs_43r3_model.exe' : sigcontextptr=0x7ffd89df5910
[EC_DRHOOK:Ryze:1:1:3835597:3835597] [20240616:075727:1718517447:17811.598] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:277] Backtrace (size = 24) with addr2line-cmd
[EC_DRHOOK:Ryze:1:1:3835597:3835597] [20240616:075727:1718517447:17811.598] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:302] /usr/bin/addr2line -fs -e '/var/lib/boinc-client/slots/5/oifs_43r3_model.exe' 0x83a8f3 0x83aef9 0x1dd5490 0x1cc6692 0x121fd8b 0x1207e29 0x11fdcb2 0xfac003 0xf5fa6b 0xf4dbc4 0xbd7131 0x12c5002 0xbacbae 0x688153 0x4cd1dc 0x4b5d85 0x4b4f52 0x49d638 0x406fbd 0x401e7f 0x401e12 0x1dcc34a 0x1dcdbe7 0x401cd5
[EC_DRHOOK:Ryze:1:1:3835597:3835597] [20240616:075727:1718517447:17811.598] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [0]: [0x83a8f3] : ??() at ??:0
[EC_DRHOOK:Ryze:1:1:3835597:3835597] [20240616:075727:1718517447:17811.598] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [1]: [0x83aef9] : ??() at ??:0
[EC_DRHOOK:Ryze:1:1:3835597:3835597] [20240616:075727:1718517447:17811.598] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [2]: [0x1dd5490] : ??() at ??:0
[EC_DRHOOK:Ryze:1:1:3835597:3835597] [20240616:075727:1718517447:17811.598] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [3]: [0x1cc6692] : ??() at ??:0
[EC_DRHOOK:Ryze:1:1:3835597:3835597] [20240616:075727:1718517447:17811.598] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [4]: [0x121fd8b] : ??() at ??:0
[EC_DRHOOK:Ryze:1:1:3835597:3835597] [20240616:075727:1718517447:17811.598] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [5]: [0x1207e29] : ??() at ??:0
[EC_DRHOOK:Ryze:1:1:3835597:3835597] [20240616:075727:1718517447:17811.598] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [6]: [0x11fdcb2] : ??() at ??:0
[EC_DRHOOK:Ryze:1:1:3835597:3835597] [20240616:075727:1718517447:17811.598] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [7]: [0xfac003] : ??() at ??:0
[EC_DRHOOK:Ryze:1:1:3835597:3835597] [20240616:075727:1718517447:17811.598] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [8]: [0xf5fa6b] : ??() at ??:0
[EC_DRHOOK:Ryze:1:1:3835597:3835597] [20240616:075727:1718517447:17811.598] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [9]: [0xf4dbc4] : ??() at ??:0
[EC_DRHOOK:Ryze:1:1:3835597:3835597] [20240616:075727:1718517447:17811.598] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [10]: [0xbd7131] : ??() at ??:0
[EC_DRHOOK:Ryze:1:1:3835597:3835597] [20240616:075727:1718517447:17811.598] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [11]: [0x12c5002] : ??() at ??:0
[EC_DRHOOK:Ryze:1:1:3835597:3835597] [20240616:075727:1718517447:17811.598] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [12]: [0xbacbae] : ??() at ??:0
[EC_DRHOOK:Ryze:1:1:3835597:3835597] [20240616:075727:1718517447:17811.598] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [13]: [0x688153] : ??() at ??:0
[EC_DRHOOK:Ryze:1:1:3835597:3835597] [20240616:075727:1718517447:17811.598] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [14]: [0x4cd1dc] : ??() at ??:0
[EC_DRHOOK:Ryze:1:1:3835597:3835597] [20240616:075727:1718517447:17811.598] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [15]: [0x4b5d85] : ??() at ??:0
[EC_DRHOOK:Ryze:1:1:3835597:3835597] [20240616:075727:1718517447:17811.598] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [16]: [0x4b4f52] : ??() at ??:0
[EC_DRHOOK:Ryze:1:1:3835597:3835597] [20240616:075727:1718517447:17811.598] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [17]: [0x49d638] : ??() at ??:0
[EC_DRHOOK:Ryze:1:1:3835597:3835597] [20240616:075727:1718517447:17811.598] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [18]: [0x406fbd] : ??() at ??:0
[EC_DRHOOK:Ryze:1:1:3835597:3835597] [20240616:075727:1718517447:17811.598] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [19]: [0x401e7f] : ??() at ??:0
[EC_DRHOOK:Ryze:1:1:3835597:3835597] [20240616:075727:1718517447:17811.598] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [20]: [0x401e12] : ??() at ??:0
[EC_DRHOOK:Ryze:1:1:3835597:3835597] [20240616:075727:1718517447:17811.598] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [21]: [0x1dcc34a] : ??() at ??:0
[EC_DRHOOK:Ryze:1:1:3835597:3835597] [20240616:075727:1718517447:17811.598] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [22]: [0x1dcdbe7] : ??() at ??:0
[EC_DRHOOK:Ryze:1:1:3835597:3835597] [20240616:075727:1718517447:17811.598] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [23]: [0x401cd5] : ??() at ??:0
[EC_DRHOOK:Ryze:1:1:3835597:3835597] [20240616:075727:1718517447:17811.598] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:370] End of backtrace(s)
 Process            1  thread            1  calling linux_trbk from intel_trbk()
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:240] Backtrace(s) for program '/var/lib/boinc-client/slots/5/oifs_43r3_model.exe' : sigcontextptr=0x7ffd89df5820
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:277] Backtrace (size = 25) with addr2line-cmd
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:302] /usr/bin/addr2line -fs -e '/var/lib/boinc-client/slots/5/oifs_43r3_model.exe' 0x83a8f3 0x83aef9 0x822540 0x1dd5490 0x1cc6692 0x121fd8b 0x1207e29 0x11fdcb2 0xfac003 0xf5fa6b 0xf4dbc4 0xbd7131 0x12c5002 0xbacbae 0x688153 0x4cd1dc 0x4b5d85 0x4b4f52 0x49d638 0x406fbd 0x401e7f 0x401e12 0x1dcc34a 0x1dcdbe7 0x401cd5
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [0]: [0x83a8f3] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [1]: [0x83aef9] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [2]: [0x822540] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [3]: [0x1dd5490] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [4]: [0x1cc6692] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [5]: [0x121fd8b] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [6]: [0x1207e29] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [7]: [0x11fdcb2] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [8]: [0xfac003] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [9]: [0xf5fa6b] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [10]: [0xf4dbc4] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [11]: [0xbd7131] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [12]: [0x12c5002] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [13]: [0xbacbae] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [14]: [0x688153] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [15]: [0x4cd1dc] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [16]: [0x4b5d85] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [17]: [0x4b4f52] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [18]: [0x49d638] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [19]: [0x406fbd] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [20]: [0x401e7f] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [21]: [0x401e12] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [22]: [0x1dcc34a] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [23]: [0x1dcdbe7] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [24]: [0x401cd5] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:370] End of backtrace(s)
[EC_DRHOOK:Ryze:1:1:3835597:3835597] [20240616:075727:1718517447:17811.617] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1734] DrHook backtrace done for signal#8, nsigs = 1
[EC_DRHOOK:Ryze:1:1:3835597:3835597] [20240616:075727:1718517447:17811.617] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1785] Calling previous signal handler at 0x1cf8da0 for signal#8, nsigs = 1
forrtl: error (72): floating overflow
Image              PC                Routine            Line        Source             
oifs_43r3_model.e  0000000001CF935B  Unknown               Unknown  Unknown
oifs_43r3_model.e  00000000008214C1  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001DD5490  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001CC6692  Unknown               Unknown  Unknown
oifs_43r3_model.e  000000000121FD8B  vdfmain_                  874  vdfmain.F90
oifs_43r3_model.e  0000000001207E29  vdfouter_                 739  vdfouter.F90
oifs_43r3_model.e  00000000011FDCB2  turbulence_layer_         158  turbulence_layer.F90
oifs_43r3_model.e  0000000000FAC003  callpar_                  869  callpar.F90
oifs_43r3_model.e  0000000000F5FA6B  ec_phys_                  670  ec_phys.F90
oifs_43r3_model.e  0000000000F4DBC4  ec_phys_drv_              599  ec_phys_drv.F90
oifs_43r3_model.e  0000000000BD7131  gp_model_                 613  gp_model.F90
oifs_43r3_model.e  00000000012C5002  gp_model_heap_             74  gp_model_heap.F90
oifs_43r3_model.e  0000000000BACBAE  scan2m_                   535  scan2m.F90
oifs_43r3_model.e  0000000000688153  stepo_                    327  stepo.F90
oifs_43r3_model.e  00000000004CD1DC  cnt4_                    1133  cnt4.F90
oifs_43r3_model.e  00000000004B5D85  cnt3_                     267  cnt3.F90
oifs_43r3_model.e  00000000004B4F52  cnt2_                      88  cnt2.F90
oifs_43r3_model.e  000000000049D638  cnt1_                      92  cnt1.F90
oifs_43r3_model.e  0000000000406FBD  cnt0_                     146  cnt0.F90
oifs_43r3_model.e  0000000000401E7F  MAIN__                     96  master.F90
oifs_43r3_model.e  0000000000401E12  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001DCC34A  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001DCDBE7  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000000401CD5  Unknown               Unknown  Unknown
..The child process has been killed with signal: 6
>>> Printing last 70 lines from file: NODE.001_01

         AVE   0.421951904661095E-05 0.000000000000000E+00 0.501205397135970E-02
 GPNORM CRFL                 AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.182770523930869E-05 0.000000000000000E+00 0.547444297152725E-02
 GPNORM CSFL                 AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.241541267855880E-06 0.000000000000000E+00 0.427147564136278E-02
 GPNORM PFRC                 AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.108207154512566E+00 0.000000000000000E+00 0.100000000000000E+01
 NSTEP =   972 STEPO   A00000000
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
 NSTEP =   972 SCAN2M_HPOS  P
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ICMGGh7zg+000972 MODE=w
 IO-STREAM CLOSED - ICMGGh7zg+000972
 MPI-TASK:   1 -    324535924 BYTES IN      6 RECORDS TRANSFERRED IN    0.0004 SECONDS811339.8 Mbytes/s, TOTAL TIME=    0.0075( 5.3%)
 NSTEP =   972 STEPO   0AA000000
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ./ICMSHh7zg+000972 MODE=w
 IO-STREAM CLOSED - ./ICMSHh7zg+000972
 MPI-TASK:   1 -    707399250 BYTES IN    202 RECORDS TRANSFERRED IN    0.0079 SECONDS 89544.2 Mbytes/s, TOTAL TIME=    0.2111( 3.7%)
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ./ICMUAh7zg+000972 MODE=w
 IO-STREAM CLOSED - ./ICMUAh7zg+000972
 MPI-TASK:   1 -   5192574784 BYTES IN    125 RECORDS TRANSFERRED IN    0.0049 SECONDS******** Mbytes/s, TOTAL TIME=    0.1586( 3.1%)
 STEPO_FPOS WAS : B00000F00E
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ./ICMUAh7zg+000972 MODE=a
 IO-STREAM CLOSED - ./ICMUAh7zg+000972
 MPI-TASK:   1 -           10 BYTES IN      2 RECORDS TRANSFERRED IN    0.0002 SECONDS     0.0 Mbytes/s, TOTAL TIME=    0.0030( 6.7%)
 STEPO_FPOS WAS : V00000000Y
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ./ICMUAh7zg+000972 MODE=a
 IO-STREAM CLOSED - ./ICMUAh7zg+000972
 MPI-TASK:   1 -    315956195 BYTES IN      4 RECORDS TRANSFERRED IN    0.0003 SECONDS******** Mbytes/s, TOTAL TIME=    0.0050( 6.0%)
 STEPO_FPOS WAS : T00000000M
 NSTEP =   972 STEPO   0AAA00AAA
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
 NSTEP =   972 SCAN2M_HPOS  P
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ICMGGh7zg+000972 MODE=a
 IO-STREAM CLOSED - ICMGGh7zg+000972
 MPI-TASK:   1 -    324535924 BYTES IN      7 RECORDS TRANSFERRED IN    0.0003 SECONDS******** Mbytes/s, TOTAL TIME=    0.0055( 5.5%)
 07:54:16 STEP  972 H= 243:00 +CPU= 20.407
 NSTEP =   973 STEPO   0AAA00AAA
 07:54:34 STEP  973 H= 243:15 +CPU= 17.674
 NSTEP =   974 STEPO   0AAA00AAA
 07:54:53 STEP  974 H= 243:30 +CPU= 18.705
ISTEP=     975 RSOLINC= 1361.4482 RI0= 1365.0000 ZI0= 1361.4482 RIP0= 1361.4482
ISTEP=     975 ZI0= 1361.4482 ZSEASON= 1.000000000 REA=149597870000.00 RDEASO=149597870000.00
 NSTEP =   975 STEPO   0AAA00AAA
 07:55:11 STEP  975 H= 243:45 +CPU= 18.432
 NSTEP =   976 STEPO   0AAA00AAA
 07:55:29 STEP  976 H= 244:00 +CPU= 17.927
 NSTEP =   977 STEPO   0AAA00AAA
 07:55:45 STEP  977 H= 244:15 +CPU= 15.040
 NSTEP =   978 STEPO   0AAA00AAA
 07:56:03 STEP  978 H= 244:30 +CPU= 18.080
 NSTEP =   979 STEPO   0AAA00AAA
 07:56:21 STEP  979 H= 244:45 +CPU= 18.112
ISTEP=     980 RSOLINC= 1361.4482 RI0= 1365.0000 ZI0= 1361.4482 RIP0= 1361.4482
ISTEP=     980 ZI0= 1361.4482 ZSEASON= 1.000000000 REA=149597870000.00 RDEASO=149597870000.00
 NSTEP =   980 STEPO   0AAA00AAA
 07:56:39 STEP  980 H= 245:00 +CPU= 18.333
 NSTEP =   981 STEPO   0AAA00AAA
 07:56:57 STEP  981 H= 245:15 +CPU= 17.503
 NSTEP =   982 STEPO   0AAA00AAA
 07:57:14 STEP  982 H= 245:30 +CPU= 16.687
 NSTEP =   983 STEPO   0AAA00AAA
------------------------------------------------
CNT0 not found; string returned was: 'STEPO'
>>> Printing last 8 lines from file: ifs.stat

 07:55:29 0AAA00AAA STEPO      977    17.928   17.928   18.005    293:41    294:54 0.19709459338951E-04       2GB       0MB
 07:55:45 0AAA00AAA STEPO      978    15.040   15.040   15.096    293:56    295:09 0.19709459338951E-04       2GB       0MB
 07:56:03 0AAA00AAA STEPO      979    18.080   18.080   18.139    294:14    295:27 0.19709459338951E-04       2GB       0MB
 07:56:21 0AAA00AAA STEPO      980    18.112   18.112   18.152    294:32    295:45 0.19709459338951E-04       2GB       0MB
 07:56:39 0AAA00AAA STEPO      981    18.333   18.333   18.389    294:51    296:04 0.19709459338951E-04       2GB       0MB
 07:56:57 0AAA00AAA STEPO      982    17.503   17.503   17.577    295:08    296:21 0.19709459338951E-04       2GB       0MB
 07:57:14 0AAA00AAA STEPO      983    16.687   16.687   16.740    295:25    296:38 0.19709459338951E-04       2GB       0MB
------------------------------------------------
>>> Printing last 11 lines from file: rcf

 CSTEP   = "     960",
 CTIME   = "00100000      ",
 NSTEPLPP        =         201,         202,          49,      228226,      228227, 2*0,         948, 2*0,
 IPRGPNSRES      =           1,
 IPRGPEWRES      =           1,
 IPRTRWRES       =           1,
 IPRTRVRES       =           1,
 GMASS0  =  0.000000000000000E+000,
 GMASSI  =  0.000000000000000E+000
 /
------------------------------------------------
>>> Printing last 8 lines from file: /var/lib/boinc-client/slots/5/progress_file_12290044.xml

<running_values>
  <last_cpu_time>17739.670000</last_cpu_time>
  <upload_file_number>10</upload_file_number>
  <last_iter>983</last_iter>
  <last_upload>864000</last_upload>
  <model_completed>0</model_completed>
</running_values>
------------------------------------------------
..Failed, model did not complete successfully

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
16 Jun 2024 06:04:21 1549176 22449337 oifs_43r3_bl_a1bp_2016092300_20_1018_12290044_2 864,000 17,334 0.0201


©2024 cpdn.org