climateprediction.net home page
Task 22449679

Task 22449679

Name oifs_43r3_bl_a25z_2016092300_20_1018_12291134_1
Workunit 12291134
Created 17 Jun 2024, 16:32:48 UTC
Sent 17 Jun 2024, 16:33:51 UTC
Report deadline 16 Aug 2024, 16:33:51 UTC
Received 18 Jun 2024, 10:23:15 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 1 (0x00000001) Unknown error code
Computer ID 1551419
Run time 16 hours 57 min 56 sec
CPU time 13 hours 57 min 19 sec
Validate state Invalid
Credit 0.00
Device peak FLOPS 3.07 GFLOPS
Application version OpenIFS 43r3 Baroclinic Lifecycle v1.13
x86_64-pc-linux-gnu
Peak working set size 5,425.21 MB
Peak swap size 5,981.07 MB
Peak disk usage 1,581.90 MB
Stderr
<core_client_version>8.0.2</core_client_version>
<![CDATA[
<message>
process exited with code 1 (0x1, -255)</message>
<stderr_txt>
var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMSHh7zg+000384
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMUAh7zg+000384
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMGGh7zg+000396
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMSHh7zg+000396
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMUAh7zg+000396
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMGGh7zg+000408
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMSHh7zg+000408
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMUAh7zg+000408
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMGGh7zg+000420
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMSHh7zg+000420
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMUAh7zg+000420
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMGGh7zg+000432
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMSHh7zg+000432
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMUAh7zg+000432
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMGGh7zg+000444
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMSHh7zg+000444
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMUAh7zg+000444
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMGGh7zg+000456
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMSHh7zg+000456
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMUAh7zg+000456
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMGGh7zg+000468
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMSHh7zg+000468
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMUAh7zg+000468
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a25z_2016092300_20_1018_12291134_1_r1843713102_4.zip
Uploading the intermediate file: upload_file_4.zip
  04:12:36 STEP  481 H= 120:15 +CPU= 53.673
  04:13:41 STEP  482 H= 120:30 +CPU= 53.680
  04:14:47 STEP  483 H= 120:45 +CPU= 53.749
  04:15:57 STEP  484 H= 121:00 +CPU= 56.227
  04:17:04 STEP  485 H= 121:15 +CPU= 54.403
  04:18:10 STEP  486 H= 121:30 +CPU= 54.317
  04:19:17 STEP  487 H= 121:45 +CPU= 53.942
  04:20:25 STEP  488 H= 122:00 +CPU= 55.029
  04:21:30 STEP  489 H= 122:15 +CPU= 54.027
  04:22:39 STEP  490 H= 122:30 +CPU= 54.094
  04:23:45 STEP  491 H= 122:45 +CPU= 53.109
  04:25:10 STEP  492 H= 123:00 +CPU= 66.794
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+000492
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+000492
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+000492
  04:26:17 STEP  493 H= 123:15 +CPU= 52.230
  04:27:21 STEP  494 H= 123:30 +CPU= 51.747
  04:28:25 STEP  495 H= 123:45 +CPU= 52.110
  04:29:31 STEP  496 H= 124:00 +CPU= 53.155
  04:30:35 STEP  497 H= 124:15 +CPU= 51.937
  04:31:39 STEP  498 H= 124:30 +CPU= 52.099
  04:32:43 STEP  499 H= 124:45 +CPU= 51.900
  04:33:48 STEP  500 H= 125:00 +CPU= 53.067
  04:34:54 STEP  501 H= 125:15 +CPU= 51.906
  04:36:00 STEP  502 H= 125:30 +CPU= 51.587
  04:37:12 STEP  503 H= 125:45 +CPU= 52.574
  04:38:46 STEP  504 H= 126:00 +CPU= 65.833
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+000504
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+000504
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+000504
  04:40:00 STEP  505 H= 126:15 +CPU= 51.858
  04:41:05 STEP  506 H= 126:30 +CPU= 52.285
  04:42:10 STEP  507 H= 126:45 +CPU= 52.805
  04:43:15 STEP  508 H= 127:00 +CPU= 54.188
  04:44:19 STEP  509 H= 127:15 +CPU= 52.803
  04:45:26 STEP  510 H= 127:30 +CPU= 53.247
  04:46:31 STEP  511 H= 127:45 +CPU= 53.690
  04:47:38 STEP  512 H= 128:00 +CPU= 54.993
  04:48:45 STEP  513 H= 128:15 +CPU= 53.977
  04:49:48 STEP  514 H= 128:30 +CPU= 53.725
  04:50:52 STEP  515 H= 128:45 +CPU= 53.897
  04:52:12 STEP  516 H= 129:00 +CPU= 67.439
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+000516
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+000516
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+000516
  04:53:16 STEP  517 H= 129:15 +CPU= 53.541
  04:54:22 STEP  518 H= 129:30 +CPU= 54.014
  04:55:26 STEP  519 H= 129:45 +CPU= 54.001
  04:56:36 STEP  520 H= 130:00 +CPU= 56.086
  04:57:38 STEP  521 H= 130:15 +CPU= 53.637
  04:58:42 STEP  522 H= 130:30 +CPU= 54.066
  04:59:47 STEP  523 H= 130:45 +CPU= 54.183
  05:00:55 STEP  524 H= 131:00 +CPU= 55.002
  05:01:57 STEP  525 H= 131:15 +CPU= 53.305
  05:03:01 STEP  526 H= 131:30 +CPU= 52.860
  05:04:06 STEP  527 H= 131:45 +CPU= 52.818
  05:05:35 STEP  528 H= 132:00 +CPU= 70.353
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+000528
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+000528
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+000528
  05:06:37 STEP  529 H= 132:15 +CPU= 52.150
  05:07:44 STEP  530 H= 132:30 +CPU= 52.959
  05:08:52 STEP  531 H= 132:45 +CPU= 52.099
  05:10:00 STEP  532 H= 133:00 +CPU= 53.393
  05:11:06 STEP  533 H= 133:15 +CPU= 52.319
  05:12:11 STEP  534 H= 133:30 +CPU= 52.176
  05:13:17 STEP  535 H= 133:45 +CPU= 52.247
  05:14:25 STEP  536 H= 134:00 +CPU= 55.042
  05:15:32 STEP  537 H= 134:15 +CPU= 53.372
  05:16:34 STEP  538 H= 134:30 +CPU= 52.161
  05:17:39 STEP  539 H= 134:45 +CPU= 52.556
  05:19:01 STEP  540 H= 135:00 +CPU= 66.665
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+000540
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+000540
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+000540
  05:20:04 STEP  541 H= 135:15 +CPU= 52.764
  05:21:08 STEP  542 H= 135:30 +CPU= 52.685
  05:22:12 STEP  543 H= 135:45 +CPU= 53.140
  05:23:16 STEP  544 H= 136:00 +CPU= 54.464
  05:24:20 STEP  545 H= 136:15 +CPU= 52.983
  05:25:25 STEP  546 H= 136:30 +CPU= 53.525
  05:26:30 STEP  547 H= 136:45 +CPU= 53.617
  05:27:37 STEP  548 H= 137:00 +CPU= 55.324
  05:28:43 STEP  549 H= 137:15 +CPU= 54.250
  05:29:46 STEP  550 H= 137:30 +CPU= 53.876
  05:30:52 STEP  551 H= 137:45 +CPU= 53.967
  05:32:16 STEP  552 H= 138:00 +CPU= 68.377
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+000552
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+000552
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+000552
  05:33:22 STEP  553 H= 138:15 +CPU= 53.878
  05:34:21 STEP  554 H= 138:30 +CPU= 53.308
  05:35:19 STEP  555 H= 138:45 +CPU= 52.854
  05:36:16 STEP  556 H= 139:00 +CPU= 54.539
  05:37:12 STEP  557 H= 139:15 +CPU= 53.259
  05:38:14 STEP  558 H= 139:30 +CPU= 52.802
  05:39:19 STEP  559 H= 139:45 +CPU= 53.354
  05:40:27 STEP  560 H= 140:00 +CPU= 53.745
  05:41:36 STEP  561 H= 140:15 +CPU= 52.759
  05:42:41 STEP  562 H= 140:30 +CPU= 51.811
  05:43:47 STEP  563 H= 140:45 +CPU= 51.894
  05:45:08 STEP  564 H= 141:00 +CPU= 65.110
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+000564
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+000564
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+000564
  05:46:14 STEP  565 H= 141:15 +CPU= 51.286
  05:47:20 STEP  566 H= 141:30 +CPU= 51.693
  05:48:24 STEP  567 H= 141:45 +CPU= 51.747
  05:49:30 STEP  568 H= 142:00 +CPU= 52.843
  05:50:37 STEP  569 H= 142:15 +CPU= 51.461
  05:51:39 STEP  570 H= 142:30 +CPU= 51.926
  05:52:44 STEP  571 H= 142:45 +CPU= 51.226
  05:53:48 STEP  572 H= 143:00 +CPU= 52.301
  05:54:55 STEP  573 H= 143:15 +CPU= 51.554
  05:55:59 STEP  574 H= 143:30 +CPU= 51.502
  05:57:05 STEP  575 H= 143:45 +CPU= 51.750
  05:58:29 STEP  576 H= 144:00 +CPU= 69.541
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+000576
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+000576
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+000576
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMGGh7zg+000480
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMSHh7zg+000480
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMUAh7zg+000480
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMGGh7zg+000492
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMSHh7zg+000492
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMUAh7zg+000492
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMGGh7zg+000504
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMSHh7zg+000504
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMUAh7zg+000504
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMGGh7zg+000516
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMSHh7zg+000516
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMUAh7zg+000516
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMGGh7zg+000528
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMSHh7zg+000528
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMUAh7zg+000528
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMGGh7zg+000540
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMSHh7zg+000540
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMUAh7zg+000540
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMGGh7zg+000552
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMSHh7zg+000552
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMUAh7zg+000552
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMGGh7zg+000564
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMSHh7zg+000564
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMUAh7zg+000564
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a25z_2016092300_20_1018_12291134_1_r1843713102_5.zip
Uploading the intermediate file: upload_file_5.zip
  05:59:36 STEP  577 H= 144:15 +CPU= 52.030
  06:00:39 STEP  578 H= 144:30 +CPU= 52.382
  06:01:44 STEP  579 H= 144:45 +CPU= 52.429
  06:02:50 STEP  580 H= 145:00 +CPU= 54.085
  06:03:53 STEP  581 H= 145:15 +CPU= 52.273
  06:04:55 STEP  582 H= 145:30 +CPU= 52.367
  06:06:00 STEP  583 H= 145:45 +CPU= 52.810
  06:07:04 STEP  584 H= 146:00 +CPU= 54.301
  06:08:07 STEP  585 H= 146:15 +CPU= 53.092
  06:09:12 STEP  586 H= 146:30 +CPU= 53.260
  06:10:20 STEP  587 H= 146:45 +CPU= 52.755
  06:11:41 STEP  588 H= 147:00 +CPU= 66.076
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+000588
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+000588
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+000588
  06:12:47 STEP  589 H= 147:15 +CPU= 52.929
  06:13:52 STEP  590 H= 147:30 +CPU= 53.622
  06:14:57 STEP  591 H= 147:45 +CPU= 52.725
  06:16:02 STEP  592 H= 148:00 +CPU= 54.105
  06:17:06 STEP  593 H= 148:15 +CPU= 52.589
  06:18:09 STEP  594 H= 148:30 +CPU= 52.284
  06:19:13 STEP  595 H= 148:45 +CPU= 52.098
  06:20:16 STEP  596 H= 149:00 +CPU= 52.978
  06:21:20 STEP  597 H= 149:15 +CPU= 52.083
  06:22:20 STEP  598 H= 149:30 +CPU= 51.551
  06:23:22 STEP  599 H= 149:45 +CPU= 51.531
  06:24:40 STEP  600 H= 150:00 +CPU= 64.809
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+000600
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+000600
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+000600
  06:25:42 STEP  601 H= 150:15 +CPU= 51.591
  06:26:44 STEP  602 H= 150:30 +CPU= 51.561
  06:27:46 STEP  603 H= 150:45 +CPU= 51.541
  06:28:45 STEP  604 H= 151:00 +CPU= 53.609
  06:29:40 STEP  605 H= 151:15 +CPU= 52.337
  06:30:34 STEP  606 H= 151:30 +CPU= 52.197
  06:31:28 STEP  607 H= 151:45 +CPU= 51.550
  06:32:24 STEP  608 H= 152:00 +CPU= 53.123
  06:33:19 STEP  609 H= 152:15 +CPU= 52.493
  06:34:13 STEP  610 H= 152:30 +CPU= 52.282
  06:35:08 STEP  611 H= 152:45 +CPU= 52.277
  06:36:17 STEP  612 H= 153:00 +CPU= 66.243
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+000612
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+000612
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+000612
  06:37:13 STEP  613 H= 153:15 +CPU= 52.694
  06:38:08 STEP  614 H= 153:30 +CPU= 52.962
  06:39:07 STEP  615 H= 153:45 +CPU= 51.816
  06:40:10 STEP  616 H= 154:00 +CPU= 54.437
  06:41:23 STEP  617 H= 154:15 +CPU= 53.064
  06:42:27 STEP  618 H= 154:30 +CPU= 53.202
  06:43:32 STEP  619 H= 154:45 +CPU= 53.126
  06:44:42 STEP  620 H= 155:00 +CPU= 55.137
  06:45:43 STEP  621 H= 155:15 +CPU= 53.465
  06:46:41 STEP  622 H= 155:30 +CPU= 53.428
  06:47:39 STEP  623 H= 155:45 +CPU= 53.657
  06:48:49 STEP  624 H= 156:00 +CPU= 67.585
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+000624
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+000624
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+000624
  06:49:44 STEP  625 H= 156:15 +CPU= 52.695
  06:50:40 STEP  626 H= 156:30 +CPU= 52.982
  06:51:38 STEP  627 H= 156:45 +CPU= 53.429
  06:52:36 STEP  628 H= 157:00 +CPU= 54.806
  06:53:34 STEP  629 H= 157:15 +CPU= 54.041
  06:54:38 STEP  630 H= 157:30 +CPU= 53.382
  06:55:46 STEP  631 H= 157:45 +CPU= 53.917
  06:56:56 STEP  632 H= 158:00 +CPU= 54.661
  06:58:01 STEP  633 H= 158:15 +CPU= 52.803
  06:59:08 STEP  634 H= 158:30 +CPU= 53.289
  07:00:12 STEP  635 H= 158:45 +CPU= 52.993
  07:01:33 STEP  636 H= 159:00 +CPU= 66.762
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+000636
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+000636
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+000636
  07:02:38 STEP  637 H= 159:15 +CPU= 53.475
  07:03:47 STEP  638 H= 159:30 +CPU= 53.764
  07:04:51 STEP  639 H= 159:45 +CPU= 53.671
  07:06:00 STEP  640 H= 160:00 +CPU= 55.452
  07:07:05 STEP  641 H= 160:15 +CPU= 53.825
  07:08:10 STEP  642 H= 160:30 +CPU= 53.716
  07:09:17 STEP  643 H= 160:45 +CPU= 54.225
  07:10:26 STEP  644 H= 161:00 +CPU= 54.831
  07:11:30 STEP  645 H= 161:15 +CPU= 53.782
  07:12:38 STEP  646 H= 161:30 +CPU= 53.886
  07:13:39 STEP  647 H= 161:45 +CPU= 52.315
  07:15:00 STEP  648 H= 162:00 +CPU= 66.102
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+000648
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+000648
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+000648
  07:16:06 STEP  649 H= 162:15 +CPU= 52.732
  07:17:12 STEP  650 H= 162:30 +CPU= 53.121
  07:18:19 STEP  651 H= 162:45 +CPU= 53.162
  07:19:23 STEP  652 H= 163:00 +CPU= 53.683
  07:20:26 STEP  653 H= 163:15 +CPU= 52.171
  07:21:30 STEP  654 H= 163:30 +CPU= 52.736
  07:22:41 STEP  655 H= 163:45 +CPU= 53.273
  07:23:45 STEP  656 H= 164:00 +CPU= 53.666
  07:24:52 STEP  657 H= 164:15 +CPU= 53.252
  07:26:00 STEP  658 H= 164:30 +CPU= 52.693
  07:27:03 STEP  659 H= 164:45 +CPU= 52.883
  07:28:25 STEP  660 H= 165:00 +CPU= 66.532
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+000660
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+000660
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+000660
  07:29:31 STEP  661 H= 165:15 +CPU= 53.050
  07:30:33 STEP  662 H= 165:30 +CPU= 53.003
  07:31:39 STEP  663 H= 165:45 +CPU= 53.721
  07:32:44 STEP  664 H= 166:00 +CPU= 54.488
  07:33:49 STEP  665 H= 166:15 +CPU= 53.778
  07:34:58 STEP  666 H= 166:30 +CPU= 54.507
  07:36:03 STEP  667 H= 166:45 +CPU= 54.451
  07:37:15 STEP  668 H= 167:00 +CPU= 55.777
  07:38:21 STEP  669 H= 167:15 +CPU= 54.387
  07:39:24 STEP  670 H= 167:30 +CPU= 54.166
  07:40:27 STEP  671 H= 167:45 +CPU= 53.955
  07:41:53 STEP  672 H= 168:00 +CPU= 71.757
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+000672
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+000672
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+000672
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMGGh7zg+000576
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMSHh7zg+000576
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMUAh7zg+000576
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMGGh7zg+000588
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMSHh7zg+000588
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMUAh7zg+000588
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMGGh7zg+000600
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMSHh7zg+000600
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMUAh7zg+000600
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMGGh7zg+000612
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMSHh7zg+000612
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMUAh7zg+000612
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMGGh7zg+000624
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMSHh7zg+000624
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMUAh7zg+000624
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMGGh7zg+000636
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMSHh7zg+000636
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMUAh7zg+000636
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMGGh7zg+000648
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMSHh7zg+000648
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMUAh7zg+000648
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMGGh7zg+000660
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMSHh7zg+000660
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMUAh7zg+000660
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a25z_2016092300_20_1018_12291134_1_r1843713102_6.zip
Uploading the intermediate file: upload_file_6.zip
  07:43:01 STEP  673 H= 168:15 +CPU= 53.873
  07:44:05 STEP  674 H= 168:30 +CPU= 53.052
  07:45:11 STEP  675 H= 168:45 +CPU= 53.617
  07:46:17 STEP  676 H= 169:00 +CPU= 54.475
  07:47:21 STEP  677 H= 169:15 +CPU= 53.029
  07:48:24 STEP  678 H= 169:30 +CPU= 53.035
  07:49:27 STEP  679 H= 169:45 +CPU= 52.796
  07:50:32 STEP  680 H= 170:00 +CPU= 54.261
  07:51:32 STEP  681 H= 170:15 +CPU= 52.316
  07:52:36 STEP  682 H= 170:30 +CPU= 52.340
  07:53:36 STEP  683 H= 170:45 +CPU= 52.327
  07:54:55 STEP  684 H= 171:00 +CPU= 66.549
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+000684
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+000684
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+000684
  07:55:57 STEP  685 H= 171:15 +CPU= 53.363
  07:57:00 STEP  686 H= 171:30 +CPU= 53.596
  07:58:02 STEP  687 H= 171:45 +CPU= 53.868
  07:59:05 STEP  688 H= 172:00 +CPU= 55.352
  08:00:10 STEP  689 H= 172:15 +CPU= 54.214
  08:01:15 STEP  690 H= 172:30 +CPU= 54.229
  08:02:24 STEP  691 H= 172:45 +CPU= 54.985
  08:03:32 STEP  692 H= 173:00 +CPU= 57.007
  08:04:36 STEP  693 H= 173:15 +CPU= 55.335
  08:05:44 STEP  694 H= 173:30 +CPU= 56.259
  08:06:52 STEP  695 H= 173:45 +CPU= 56.398
  08:08:17 STEP  696 H= 174:00 +CPU= 70.377
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+000696
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+000696
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+000696
  08:09:28 STEP  697 H= 174:15 +CPU= 56.379
  08:10:40 STEP  698 H= 174:30 +CPU= 56.997
  08:11:52 STEP  699 H= 174:45 +CPU= 56.854
  08:13:05 STEP  700 H= 175:00 +CPU= 57.638
  08:14:11 STEP  701 H= 175:15 +CPU= 56.409
  08:15:18 STEP  702 H= 175:30 +CPU= 56.470
  08:16:23 STEP  703 H= 175:45 +CPU= 56.377
  08:17:29 STEP  704 H= 176:00 +CPU= 57.313
  08:18:37 STEP  705 H= 176:15 +CPU= 56.858
  08:19:44 STEP  706 H= 176:30 +CPU= 56.858
  08:20:51 STEP  707 H= 176:45 +CPU= 57.087
  08:22:14 STEP  708 H= 177:00 +CPU= 70.383
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+000708
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+000708
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+000708
  08:23:21 STEP  709 H= 177:15 +CPU= 56.870
  08:24:29 STEP  710 H= 177:30 +CPU= 57.045
  08:25:36 STEP  711 H= 177:45 +CPU= 56.680
  08:26:45 STEP  712 H= 178:00 +CPU= 58.160
  08:27:51 STEP  713 H= 178:15 +CPU= 56.385
  08:28:58 STEP  714 H= 178:30 +CPU= 56.966
  08:30:05 STEP  715 H= 178:45 +CPU= 56.873
  08:31:14 STEP  716 H= 179:00 +CPU= 58.635
  08:32:22 STEP  717 H= 179:15 +CPU= 57.523
  08:33:28 STEP  718 H= 179:30 +CPU= 56.848
  08:34:35 STEP  719 H= 179:45 +CPU= 57.230
  08:36:04 STEP  720 H= 180:00 +CPU= 75.821
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+000720
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+000720
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+000720
  08:37:10 STEP  721 H= 180:15 +CPU= 57.380
  08:38:17 STEP  722 H= 180:30 +CPU= 57.617
  08:39:24 STEP  723 H= 180:45 +CPU= 57.105
  08:40:34 STEP  724 H= 181:00 +CPU= 58.469
  08:41:44 STEP  725 H= 181:15 +CPU= 57.819
  08:42:53 STEP  726 H= 181:30 +CPU= 57.309
  08:44:00 STEP  727 H= 181:45 +CPU= 57.370
  08:45:08 STEP  728 H= 182:00 +CPU= 58.448
  08:46:15 STEP  729 H= 182:15 +CPU= 57.497
  08:47:21 STEP  730 H= 182:30 +CPU= 57.222
  08:48:28 STEP  731 H= 182:45 +CPU= 56.693
  08:49:51 STEP  732 H= 183:00 +CPU= 70.038
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+000732
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+000732
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+000732
  08:51:03 STEP  733 H= 183:15 +CPU= 56.759
  08:52:10 STEP  734 H= 183:30 +CPU= 56.414
  08:53:16 STEP  735 H= 183:45 +CPU= 56.503
  08:54:23 STEP  736 H= 184:00 +CPU= 57.523
  08:55:29 STEP  737 H= 184:15 +CPU= 56.707
  08:56:35 STEP  738 H= 184:30 +CPU= 57.212
  08:57:40 STEP  739 H= 184:45 +CPU= 56.650
  08:58:48 STEP  740 H= 185:00 +CPU= 58.460
  08:59:54 STEP  741 H= 185:15 +CPU= 56.806
  09:01:02 STEP  742 H= 185:30 +CPU= 57.217
  09:02:08 STEP  743 H= 185:45 +CPU= 56.697
  09:03:31 STEP  744 H= 186:00 +CPU= 70.953
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+000744
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+000744
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+000744
  09:04:40 STEP  745 H= 186:15 +CPU= 56.849
  09:05:48 STEP  746 H= 186:30 +CPU= 56.740
  09:06:58 STEP  747 H= 186:45 +CPU= 57.036
  09:08:11 STEP  748 H= 187:00 +CPU= 58.669
  09:09:20 STEP  749 H= 187:15 +CPU= 56.463
  09:10:32 STEP  750 H= 187:30 +CPU= 57.287
  09:11:42 STEP  751 H= 187:45 +CPU= 57.082
  09:12:52 STEP  752 H= 188:00 +CPU= 58.624
  09:14:02 STEP  753 H= 188:15 +CPU= 57.092
  09:15:08 STEP  754 H= 188:30 +CPU= 56.853
  09:16:15 STEP  755 H= 188:45 +CPU= 56.882
  09:17:34 STEP  756 H= 189:00 +CPU= 69.900
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+000756
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+000756
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+000756
  09:18:47 STEP  757 H= 189:15 +CPU= 57.468
  09:19:53 STEP  758 H= 189:30 +CPU= 56.787
  09:20:59 STEP  759 H= 189:45 +CPU= 57.003
  09:22:08 STEP  760 H= 190:00 +CPU= 58.917
  09:23:15 STEP  761 H= 190:15 +CPU= 56.942
  09:24:22 STEP  762 H= 190:30 +CPU= 57.243
  09:25:29 STEP  763 H= 190:45 +CPU= 57.744
  09:26:37 STEP  764 H= 191:00 +CPU= 58.689
  09:27:43 STEP  765 H= 191:15 +CPU= 57.249
  09:28:49 STEP  766 H= 191:30 +CPU= 57.027
  09:29:59 STEP  767 H= 191:45 +CPU= 56.883
  09:31:27 STEP  768 H= 192:00 +CPU= 75.517
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+000768
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+000768
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+000768
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMGGh7zg+000672
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMSHh7zg+000672
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMUAh7zg+000672
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMGGh7zg+000684
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMSHh7zg+000684
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMUAh7zg+000684
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMGGh7zg+000696
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMSHh7zg+000696
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMUAh7zg+000696
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMGGh7zg+000708
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMSHh7zg+000708
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMUAh7zg+000708
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMGGh7zg+000720
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMSHh7zg+000720
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMUAh7zg+000720
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMGGh7zg+000732
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMSHh7zg+000732
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMUAh7zg+000732
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMGGh7zg+000744
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMSHh7zg+000744
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMUAh7zg+000744
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMGGh7zg+000756
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMSHh7zg+000756
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMUAh7zg+000756
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a25z_2016092300_20_1018_12291134_1_r1843713102_7.zip
Uploading the intermediate file: upload_file_7.zip
  09:32:34 STEP  769 H= 192:15 +CPU= 56.755
  09:33:42 STEP  770 H= 192:30 +CPU= 57.409
  09:34:49 STEP  771 H= 192:45 +CPU= 57.001
  09:35:59 STEP  772 H= 193:00 +CPU= 59.167
  09:37:06 STEP  773 H= 193:15 +CPU= 56.975
  09:38:13 STEP  774 H= 193:30 +CPU= 57.451
  09:39:21 STEP  775 H= 193:45 +CPU= 57.939
  09:40:31 STEP  776 H= 194:00 +CPU= 59.316
  09:41:38 STEP  777 H= 194:15 +CPU= 57.193
  09:42:46 STEP  778 H= 194:30 +CPU= 57.216
  09:43:51 STEP  779 H= 194:45 +CPU= 56.974
  09:45:16 STEP  780 H= 195:00 +CPU= 71.605
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+000780
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+000780
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+000780
  09:46:26 STEP  781 H= 195:15 +CPU= 57.130
  09:47:35 STEP  782 H= 195:30 +CPU= 57.283
  09:48:46 STEP  783 H= 195:45 +CPU= 57.504
  09:50:00 STEP  784 H= 196:00 +CPU= 58.590
  09:51:08 STEP  785 H= 196:15 +CPU= 57.054
  09:52:13 STEP  786 H= 196:30 +CPU= 57.047
  09:53:22 STEP  787 H= 196:45 +CPU= 57.168
  09:54:34 STEP  788 H= 197:00 +CPU= 58.451
  09:55:42 STEP  789 H= 197:15 +CPU= 57.305
  09:56:53 STEP  790 H= 197:30 +CPU= 57.769
  09:58:02 STEP  791 H= 197:45 +CPU= 57.102
  09:59:28 STEP  792 H= 198:00 +CPU= 70.467
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+000792
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+000792
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+000792
  10:00:37 STEP  793 H= 198:15 +CPU= 56.853
  10:01:48 STEP  794 H= 198:30 +CPU= 57.436
  10:02:55 STEP  795 H= 198:45 +CPU= 57.519
  10:04:04 STEP  796 H= 199:00 +CPU= 58.849
  10:05:08 STEP  797 H= 199:15 +CPU= 56.471
  10:06:16 STEP  798 H= 199:30 +CPU= 57.747
  10:07:24 STEP  799 H= 199:45 +CPU= 57.337
  10:08:33 STEP  800 H= 200:00 +CPU= 58.635
  10:09:43 STEP  801 H= 200:15 +CPU= 57.323
  10:10:50 STEP  802 H= 200:30 +CPU= 56.837
  10:11:58 STEP  803 H= 200:45 +CPU= 57.276
  10:13:21 STEP  804 H= 201:00 +CPU= 70.964
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+000804
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+000804
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+000804
  10:14:27 STEP  805 H= 201:15 +CPU= 57.009
  10:15:30 STEP  806 H= 201:30 +CPU= 56.029
  10:16:36 STEP  807 H= 201:45 +CPU= 57.105
  10:17:44 STEP  808 H= 202:00 +CPU= 58.790
  10:18:50 STEP  809 H= 202:15 +CPU= 57.368
  10:19:58 STEP  810 H= 202:30 +CPU= 57.609
  10:21:06 STEP  811 H= 202:45 +CPU= 57.240
  10:22:16 STEP  812 H= 203:00 +CPU= 59.072
  10:23:28 STEP  813 H= 203:15 +CPU= 57.715
  10:24:39 STEP  814 H= 203:30 +CPU= 57.542
  10:25:50 STEP  815 H= 203:45 +CPU= 57.636
  10:27:20 STEP  816 H= 204:00 +CPU= 76.563
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+000816
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+000816
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+000816
  10:28:28 STEP  817 H= 204:15 +CPU= 57.611
  10:29:37 STEP  818 H= 204:30 +CPU= 58.153
  10:30:46 STEP  819 H= 204:45 +CPU= 57.798
  10:31:56 STEP  820 H= 205:00 +CPU= 59.396
  10:33:03 STEP  821 H= 205:15 +CPU= 57.778
  10:34:11 STEP  822 H= 205:30 +CPU= 57.546
  10:35:21 STEP  823 H= 205:45 +CPU= 58.663
  10:36:31 STEP  824 H= 206:00 +CPU= 59.021
  10:37:40 STEP  825 H= 206:15 +CPU= 57.943
  10:38:49 STEP  826 H= 206:30 +CPU= 56.773
  10:39:58 STEP  827 H= 206:45 +CPU= 57.334
  10:41:27 STEP  828 H= 207:00 +CPU= 71.322
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+000828
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+000828
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+000828
  10:42:37 STEP  829 H= 207:15 +CPU= 57.115
  10:43:43 STEP  830 H= 207:30 +CPU= 56.870
  10:44:52 STEP  831 H= 207:45 +CPU= 57.527
  10:46:01 STEP  832 H= 208:00 +CPU= 59.040
  10:47:08 STEP  833 H= 208:15 +CPU= 57.490
  10:48:16 STEP  834 H= 208:30 +CPU= 57.760
  10:49:22 STEP  835 H= 208:45 +CPU= 57.256
  10:50:30 STEP  836 H= 209:00 +CPU= 58.957
  10:51:38 STEP  837 H= 209:15 +CPU= 57.391
  10:52:45 STEP  838 H= 209:30 +CPU= 57.854
  10:53:53 STEP  839 H= 209:45 +CPU= 57.728
  10:55:19 STEP  840 H= 210:00 +CPU= 72.552
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+000840
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+000840
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+000840
  10:56:26 STEP  841 H= 210:15 +CPU= 57.865
  10:57:34 STEP  842 H= 210:30 +CPU= 57.454
  10:58:40 STEP  843 H= 210:45 +CPU= 57.694
  10:59:49 STEP  844 H= 211:00 +CPU= 59.207
  11:00:56 STEP  845 H= 211:15 +CPU= 58.199
  11:02:06 STEP  846 H= 211:30 +CPU= 57.346
  11:03:14 STEP  847 H= 211:45 +CPU= 57.163
  11:04:25 STEP  848 H= 212:00 +CPU= 59.199
  11:05:35 STEP  849 H= 212:15 +CPU= 57.414
  11:06:44 STEP  850 H= 212:30 +CPU= 57.372
  11:07:53 STEP  851 H= 212:45 +CPU= 57.491
  11:09:18 STEP  852 H= 213:00 +CPU= 71.069
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+000852
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+000852
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+000852
  11:10:30 STEP  853 H= 213:15 +CPU= 58.190
  11:11:43 STEP  854 H= 213:30 +CPU= 57.992
  11:12:55 STEP  855 H= 213:45 +CPU= 57.924
  11:14:07 STEP  856 H= 214:00 +CPU= 59.144
  11:15:18 STEP  857 H= 214:15 +CPU= 57.987
  11:16:27 STEP  858 H= 214:30 +CPU= 57.957
  11:17:38 STEP  859 H= 214:45 +CPU= 58.269
  11:18:48 STEP  860 H= 215:00 +CPU= 59.278
  11:19:54 STEP  861 H= 215:15 +CPU= 57.728
  11:21:01 STEP  862 H= 215:30 +CPU= 58.020
  11:22:06 STEP  863 H= 215:45 +CPU= 57.178
  11:23:34 STEP  864 H= 216:00 +CPU= 77.038
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+000864
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+000864
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+000864
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMGGh7zg+000768
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMSHh7zg+000768
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMUAh7zg+000768
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMGGh7zg+000780
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMSHh7zg+000780
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMUAh7zg+000780
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMGGh7zg+000792
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMSHh7zg+000792
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMUAh7zg+000792
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMGGh7zg+000804
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMSHh7zg+000804
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMUAh7zg+000804
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMGGh7zg+000816
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMSHh7zg+000816
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMUAh7zg+000816
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMGGh7zg+000828
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMSHh7zg+000828
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMUAh7zg+000828
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMGGh7zg+000840
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMSHh7zg+000840
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMUAh7zg+000840
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMGGh7zg+000852
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMSHh7zg+000852
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12291134/ICMUAh7zg+000852
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a25z_2016092300_20_1018_12291134_1_r1843713102_8.zip
Uploading the intermediate file: upload_file_8.zip
  11:24:44 STEP  865 H= 216:15 +CPU= 58.568
  11:25:51 STEP  866 H= 216:30 +CPU= 58.110
  11:26:59 STEP  867 H= 216:45 +CPU= 58.166
  11:28:09 STEP  868 H= 217:00 +CPU= 59.885
  11:29:15 STEP  869 H= 217:15 +CPU= 58.119
  11:30:27 STEP  870 H= 217:30 +CPU= 58.208
[EC_DRHOOK:testing:1:1:130989:130989] [20240618:113112:1718703072:61005.900] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1538] Received signal#8 (SIGFPE) :: 4412MB (heap), 5447MB (maxrss), 0MB (maxstack), 0 (paging), nsigs = 1
[EC_DRHOOK:testing:1:1:130989:130989] [20240618:113112:1718703072:61005.908] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1542] Also activating Harakiri-alarm (SIGALRM=14) to expire after 500s elapsed to prevent hangs, nsigs = 1
[EC_DRHOOK:testing:1:1:130989:130989] [20240618:113112:1718703072:61005.908] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1544] Harakiri signal handler 'signal_harakiri' for signal#14 (SIGALRM) installed at 0x820890 (old at (nil))
[EC_DRHOOK:testing:1:1:130989:130989] [20240618:113112:1718703072:61005.908] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1617] Signal#8 was caused by floating-point overflow [memaddr=0x1cc6692], nsigs = 1
[EC_DRHOOK:testing:1:1:130989:130989] [20240618:113112:1718703072:61005.908] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1686] Starting DrHook backtrace for signal#8, nsigs = 1
[EC_DRHOOK:testing:1:1:130989:130989] [20240618:113112:1718703072:61005.908] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3843] 4412 MB (maxheap), 5447 MB (maxrss), 0 MB (maxstack)
[EC_DRHOOK:testing:1:1:130989:130989] [20240618:113112:1718703072:61005.916] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :  MASTER 
[EC_DRHOOK:testing:1:1:130989:130989] [20240618:113112:1718703072:61005.916] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :   CNT0 
[EC_DRHOOK:testing:1:1:130989:130989] [20240618:113112:1718703072:61005.916] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :    CNT1 
[EC_DRHOOK:testing:1:1:130989:130989] [20240618:113112:1718703072:61005.916] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :     CNT2 
[EC_DRHOOK:testing:1:1:130989:130989] [20240618:113112:1718703072:61005.916] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :      CNT3 
[EC_DRHOOK:testing:1:1:130989:130989] [20240618:113112:1718703072:61005.916] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :       CNT4 
[EC_DRHOOK:testing:1:1:130989:130989] [20240618:113112:1718703072:61005.916] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :        STEPO 
[EC_DRHOOK:testing:1:1:130989:130989] [20240618:113112:1718703072:61005.916] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :         SCAN2M 
[EC_DRHOOK:testing:1:1:130989:130989] [20240618:113112:1718703072:61005.916] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :          GP_MODEL_HEAP 
[EC_DRHOOK:testing:1:1:130989:130989] [20240618:113112:1718703072:61005.916] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :           GP_MODEL 
[EC_DRHOOK:testing:1:1:130989:130989] [20240618:113112:1718703072:61005.916] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :            EC_PHYS_DRV 
[EC_DRHOOK:testing:1:1:130989:130989] [20240618:113112:1718703072:61005.916] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :             >OMP-PHYSICS              (1001) 
[EC_DRHOOK:testing:1:1:130989:130989] [20240618:113112:1718703072:61005.916] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :              EC_PHYS 
[EC_DRHOOK:testing:1:1:130989:130989] [20240618:113112:1718703072:61005.916] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :               CALLPAR 
[EC_DRHOOK:testing:1:1:130989:130989] [20240618:113112:1718703072:61005.916] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :                TURBULENCE_LAYER 
[EC_DRHOOK:testing:1:1:130989:130989] [20240618:113112:1718703072:61005.916] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :                 VDFOUTER 
[EC_DRHOOK:testing:1:1:130989:130989] [20240618:113112:1718703072:61005.916] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :                  VDFMAIN 
[EC_DRHOOK:testing:1:1:130989:130989] [20240618:113113:1718703073:61006.922] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:240] Backtrace(s) for program '/var/lib/boinc-client/slots/7/oifs_43r3_model.exe' : sigcontextptr=0x7fffb640e810
[EC_DRHOOK:testing:1:1:130989:130989] [20240618:113113:1718703073:61006.922] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:277] Backtrace (size = 24) with addr2line-cmd
[EC_DRHOOK:testing:1:1:130989:130989] [20240618:113113:1718703073:61006.922] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:302] /usr/bin/addr2line -fs -e '/var/lib/boinc-client/slots/7/oifs_43r3_model.exe' 0x83a8f3 0x83aef9 0x1dd5490 0x1cc6692 0x121fd8b 0x1207e29 0x11fdcb2 0xfac003 0xf5fa6b 0xf4dbc4 0xbd7131 0x12c5002 0xbacbae 0x688153 0x4cd1dc 0x4b5d85 0x4b4f52 0x49d638 0x406fbd 0x401e7f 0x401e12 0x1dcc34a 0x1dcdbe7 0x401cd5
[EC_DRHOOK:testing:1:1:130989:130989] [20240618:113113:1718703073:61006.922] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [0]: [0x83a8f3] : ??() at ??:0
[EC_DRHOOK:testing:1:1:130989:130989] [20240618:113113:1718703073:61006.922] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [1]: [0x83aef9] : ??() at ??:0
[EC_DRHOOK:testing:1:1:130989:130989] [20240618:113113:1718703073:61006.922] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [2]: [0x1dd5490] : ??() at ??:0
[EC_DRHOOK:testing:1:1:130989:130989] [20240618:113113:1718703073:61006.922] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [3]: [0x1cc6692] : ??() at ??:0
[EC_DRHOOK:testing:1:1:130989:130989] [20240618:113113:1718703073:61006.922] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [4]: [0x121fd8b] : ??() at ??:0
[EC_DRHOOK:testing:1:1:130989:130989] [20240618:113113:1718703073:61006.922] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [5]: [0x1207e29] : ??() at ??:0
[EC_DRHOOK:testing:1:1:130989:130989] [20240618:113113:1718703073:61006.922] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [6]: [0x11fdcb2] : ??() at ??:0
[EC_DRHOOK:testing:1:1:130989:130989] [20240618:113113:1718703073:61006.922] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [7]: [0xfac003] : ??() at ??:0
[EC_DRHOOK:testing:1:1:130989:130989] [20240618:113113:1718703073:61006.922] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [8]: [0xf5fa6b] : ??() at ??:0
[EC_DRHOOK:testing:1:1:130989:130989] [20240618:113113:1718703073:61006.922] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [9]: [0xf4dbc4] : ??() at ??:0
[EC_DRHOOK:testing:1:1:130989:130989] [20240618:113113:1718703073:61006.922] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [10]: [0xbd7131] : ??() at ??:0
[EC_DRHOOK:testing:1:1:130989:130989] [20240618:113113:1718703073:61006.922] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [11]: [0x12c5002] : ??() at ??:0
[EC_DRHOOK:testing:1:1:130989:130989] [20240618:113113:1718703073:61006.922] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [12]: [0xbacbae] : ??() at ??:0
[EC_DRHOOK:testing:1:1:130989:130989] [20240618:113113:1718703073:61006.922] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [13]: [0x688153] : ??() at ??:0
[EC_DRHOOK:testing:1:1:130989:130989] [20240618:113113:1718703073:61006.922] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [14]: [0x4cd1dc] : ??() at ??:0
[EC_DRHOOK:testing:1:1:130989:130989] [20240618:113113:1718703073:61006.922] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [15]: [0x4b5d85] : ??() at ??:0
[EC_DRHOOK:testing:1:1:130989:130989] [20240618:113113:1718703073:61006.922] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [16]: [0x4b4f52] : ??() at ??:0
[EC_DRHOOK:testing:1:1:130989:130989] [20240618:113113:1718703073:61006.922] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [17]: [0x49d638] : ??() at ??:0
[EC_DRHOOK:testing:1:1:130989:130989] [20240618:113113:1718703073:61006.922] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [18]: [0x406fbd] : ??() at ??:0
[EC_DRHOOK:testing:1:1:130989:130989] [20240618:113113:1718703073:61006.922] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [19]: [0x401e7f] : ??() at ??:0
[EC_DRHOOK:testing:1:1:130989:130989] [20240618:113113:1718703073:61006.922] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [20]: [0x401e12] : ??() at ??:0
[EC_DRHOOK:testing:1:1:130989:130989] [20240618:113113:1718703073:61006.922] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [21]: [0x1dcc34a] : ??() at ??:0
[EC_DRHOOK:testing:1:1:130989:130989] [20240618:113113:1718703073:61006.922] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [22]: [0x1dcdbe7] : ??() at ??:0
[EC_DRHOOK:testing:1:1:130989:130989] [20240618:113113:1718703073:61006.922] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [23]: [0x401cd5] : ??() at ??:0
[EC_DRHOOK:testing:1:1:130989:130989] [20240618:113113:1718703073:61006.922] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:370] End of backtrace(s)
 Process            1  thread            1  calling linux_trbk from intel_trbk()
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:240] Backtrace(s) for program '/var/lib/boinc-client/slots/7/oifs_43r3_model.exe' : sigcontextptr=0x7fffb640e720
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:277] Backtrace (size = 25) with addr2line-cmd
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:302] /usr/bin/addr2line -fs -e '/var/lib/boinc-client/slots/7/oifs_43r3_model.exe' 0x83a8f3 0x83aef9 0x822540 0x1dd5490 0x1cc6692 0x121fd8b 0x1207e29 0x11fdcb2 0xfac003 0xf5fa6b 0xf4dbc4 0xbd7131 0x12c5002 0xbacbae 0x688153 0x4cd1dc 0x4b5d85 0x4b4f52 0x49d638 0x406fbd 0x401e7f 0x401e12 0x1dcc34a 0x1dcdbe7 0x401cd5
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [0]: [0x83a8f3] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [1]: [0x83aef9] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [2]: [0x822540] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [3]: [0x1dd5490] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [4]: [0x1cc6692] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [5]: [0x121fd8b] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [6]: [0x1207e29] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [7]: [0x11fdcb2] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [8]: [0xfac003] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [9]: [0xf5fa6b] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [10]: [0xf4dbc4] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [11]: [0xbd7131] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [12]: [0x12c5002] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [13]: [0xbacbae] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [14]: [0x688153] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [15]: [0x4cd1dc] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [16]: [0x4b5d85] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [17]: [0x4b4f52] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [18]: [0x49d638] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [19]: [0x406fbd] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [20]: [0x401e7f] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [21]: [0x401e12] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [22]: [0x1dcc34a] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [23]: [0x1dcdbe7] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [24]: [0x401cd5] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:370] End of backtrace(s)
[EC_DRHOOK:testing:1:1:130989:130989] [20240618:113113:1718703073:61007.100] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1734] DrHook backtrace done for signal#8, nsigs = 1
[EC_DRHOOK:testing:1:1:130989:130989] [20240618:113113:1718703073:61007.100] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1785] Calling previous signal handler at 0x1cf8da0 for signal#8, nsigs = 1
forrtl: error (72): floating overflow
Image              PC                Routine            Line        Source             
oifs_43r3_model.e  0000000001CF935B  Unknown               Unknown  Unknown
oifs_43r3_model.e  00000000008214C1  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001DD5490  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001CC6692  Unknown               Unknown  Unknown
oifs_43r3_model.e  000000000121FD8B  vdfmain_                  874  vdfmain.F90
oifs_43r3_model.e  0000000001207E29  vdfouter_                 739  vdfouter.F90
oifs_43r3_model.e  00000000011FDCB2  turbulence_layer_         158  turbulence_layer.F90
oifs_43r3_model.e  0000000000FAC003  callpar_                  869  callpar.F90
oifs_43r3_model.e  0000000000F5FA6B  ec_phys_                  670  ec_phys.F90
oifs_43r3_model.e  0000000000F4DBC4  ec_phys_drv_              599  ec_phys_drv.F90
oifs_43r3_model.e  0000000000BD7131  gp_model_                 613  gp_model.F90
oifs_43r3_model.e  00000000012C5002  gp_model_heap_             74  gp_model_heap.F90
oifs_43r3_model.e  0000000000BACBAE  scan2m_                   535  scan2m.F90
oifs_43r3_model.e  0000000000688153  stepo_                    327  stepo.F90
oifs_43r3_model.e  00000000004CD1DC  cnt4_                    1133  cnt4.F90
oifs_43r3_model.e  00000000004B5D85  cnt3_                     267  cnt3.F90
oifs_43r3_model.e  00000000004B4F52  cnt2_                      88  cnt2.F90
oifs_43r3_model.e  000000000049D638  cnt1_                      92  cnt1.F90
oifs_43r3_model.e  0000000000406FBD  cnt0_                     146  cnt0.F90
oifs_43r3_model.e  0000000000401E7F  MAIN__                     96  master.F90
oifs_43r3_model.e  0000000000401E12  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001DCC34A  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001DCDBE7  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000000401CD5  Unknown               Unknown  Unknown
..The child process has been killed with signal: 6
>>> Printing last 70 lines from file: NODE.001_01

 GPNORM SFL                  AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.156411949504646E-04 0.000000000000000E+00 0.711654682807596E-02
 GPNORM CRFL                 AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.353377679212986E-05 0.000000000000000E+00 0.167464156329575E-01
 GPNORM CSFL                 AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.141831379301686E-05 0.000000000000000E+00 0.104785964383841E-01
 GPNORM PFRC                 AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.146273914744178E+00 0.000000000000000E+00 0.100000000000000E+01
 NSTEP =   864 STEPO   F00000000
 CREATE RESTART FILE ./srf00090000.0001                                                                                                                                                                                                                                              
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ./srf00090000.0001 MODE=w
 IO-STREAM CLOSED - ./srf00090000.0001
 MPI-TASK:   1 - ************ BYTES IN     46 RECORDS TRANSFERRED IN    4.2930 SECONDS******** Mbytes/s, TOTAL TIME=    4.9290(87.1%)
 CREATE RESTART CONTROL FILE ./rcf                                                                                                                   
 WRRESF: DELETED PREVIOUS RESTART FILE OK: ./srf00081200.0001
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
 NSTEP =   864 SCAN2M_HPOS  P
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ICMGGh7zg+000864 MODE=w
 IO-STREAM CLOSED - ICMGGh7zg+000864
 MPI-TASK:   1 -    314615396 BYTES IN      6 RECORDS TRANSFERRED IN    0.0006 SECONDS524359.0 Mbytes/s, TOTAL TIME=    0.0208( 2.9%)
 NSTEP =   864 STEPO   0AA000000
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ./ICMSHh7zg+000864 MODE=w
 IO-STREAM CLOSED - ./ICMSHh7zg+000864
 MPI-TASK:   1 -    707399250 BYTES IN    202 RECORDS TRANSFERRED IN    0.0211 SECONDS 33526.0 Mbytes/s, TOTAL TIME=    0.8788( 2.4%)
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ./ICMUAh7zg+000864 MODE=w
 IO-STREAM CLOSED - ./ICMUAh7zg+000864
 MPI-TASK:   1 -   5033846336 BYTES IN    125 RECORDS TRANSFERRED IN    0.0085 SECONDS592217.2 Mbytes/s, TOTAL TIME=    0.5427( 1.6%)
 STEPO_FPOS WAS : B00000F00E
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ./ICMUAh7zg+000864 MODE=a
 IO-STREAM CLOSED - ./ICMUAh7zg+000864
 MPI-TASK:   1 -           10 BYTES IN      2 RECORDS TRANSFERRED IN    0.0002 SECONDS     0.1 Mbytes/s, TOTAL TIME=    0.0069( 2.9%)
 STEPO_FPOS WAS : V00000000Y
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ./ICMUAh7zg+000864 MODE=a
 IO-STREAM CLOSED - ./ICMUAh7zg+000864
 MPI-TASK:   1 -    306035667 BYTES IN      4 RECORDS TRANSFERRED IN    0.0005 SECONDS612071.3 Mbytes/s, TOTAL TIME=    0.0118( 4.2%)
 STEPO_FPOS WAS : T00000000M
 NSTEP =   864 STEPO   0AAA00AAA
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
 NSTEP =   864 SCAN2M_HPOS  P
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ICMGGh7zg+000864 MODE=a
 IO-STREAM CLOSED - ICMGGh7zg+000864
 MPI-TASK:   1 -    314615396 BYTES IN      7 RECORDS TRANSFERRED IN    0.0007 SECONDS449450.6 Mbytes/s, TOTAL TIME=    0.0202( 3.5%)
 11:23:34 STEP  864 H= 216:00 +CPU= 77.038
ISTEP=     865 RSOLINC= 1361.4482 RI0= 1365.0000 ZI0= 1361.4482 RIP0= 1361.4482
ISTEP=     865 ZI0= 1361.4482 ZSEASON= 1.000000000 REA=149597870000.00 RDEASO=149597870000.00
 NSTEP =   865 STEPO   0AAA00AAA
 11:24:44 STEP  865 H= 216:15 +CPU= 58.568
 NSTEP =   866 STEPO   0AAA00AAA
 11:25:51 STEP  866 H= 216:30 +CPU= 58.110
 NSTEP =   867 STEPO   0AAA00AAA
 11:26:59 STEP  867 H= 216:45 +CPU= 58.166
 NSTEP =   868 STEPO   0AAA00AAA
 11:28:09 STEP  868 H= 217:00 +CPU= 59.885
 NSTEP =   869 STEPO   0AAA00AAA
 11:29:15 STEP  869 H= 217:15 +CPU= 58.119
ISTEP=     870 RSOLINC= 1361.4482 RI0= 1365.0000 ZI0= 1361.4482 RIP0= 1361.4482
ISTEP=     870 ZI0= 1361.4482 ZSEASON= 1.000000000 REA=149597870000.00 RDEASO=149597870000.00
 NSTEP =   870 STEPO   0AAA00AAA
 11:30:27 STEP  870 H= 217:30 +CPU= 58.208
 NSTEP =   871 STEPO   0AAA00AAA
------------------------------------------------
CNT0 not found; string returned was: 'STEPO'
>>> Printing last 8 lines from file: ifs.stat

 11:23:34 0AAA00AAA STEPO      865    59.961   59.961   68.365    830:50   1009:07 0.37482344802001E-04       2GB       0MB
 11:24:44 0AAA00AAA STEPO      866    58.568   58.568   69.599    831:49   1010:17 0.37482344802001E-04       2GB       0MB
 11:25:51 0AAA00AAA STEPO      867    58.110   58.110   66.957    832:47   1011:24 0.37482344802001E-04       2GB       0MB
 11:26:59 0AAA00AAA STEPO      868    58.166   58.166   67.932    833:45   1012:32 0.37482344802001E-04       2GB       0MB
 11:28:09 0AAA00AAA STEPO      869    59.885   59.885   70.091    834:45   1013:42 0.37482344802001E-04       2GB       0MB
 11:29:15 0AAA00AAA STEPO      870    58.119   58.119   66.775    835:43   1014:49 0.37482344802001E-04       2GB       0MB
 11:30:27 0AAA00AAA STEPO      871    58.208   58.208   71.204    836:41   1016:00 0.37482344802001E-04       2GB       0MB
------------------------------------------------
>>> Printing last 11 lines from file: rcf

 CSTEP   = "     864",
 CTIME   = "00090000      ",
 NSTEPLPP        =         201,         202,          49,      228226,      228227, 2*0,         852, 2*0,
 IPRGPNSRES      =           1,
 IPRGPEWRES      =           1,
 IPRTRWRES       =           1,
 IPRTRVRES       =           1,
 GMASS0  =  0.000000000000000E+000,
 GMASSI  =  0.000000000000000E+000
 /
------------------------------------------------
>>> Printing last 8 lines from file: /var/lib/boinc-client/slots/7/progress_file_12291134.xml

<running_values>
  <last_cpu_time>50234.190000</last_cpu_time>
  <upload_file_number>9</upload_file_number>
  <last_iter>871</last_iter>
  <last_upload>777600</last_upload>
  <model_completed>0</model_completed>
</running_values>
------------------------------------------------
..Failed, model did not complete successfully

</stderr_txt>
]]>
No trickles!


©2024 cpdn.org