climateprediction.net home page
Task 22314149

Task 22314149

Name oifs_43r3_bl_a17v_2016092300_15_991_12208366_1
Workunit 12208366
Created 17 Feb 2023, 4:50:30 UTC
Sent 17 Feb 2023, 4:51:20 UTC
Report deadline 18 Apr 2023, 4:51:20 UTC
Received 18 Feb 2023, 15:14:30 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 1 (0x00000001) Unknown error code
Computer ID 1538533
Run time 1 hours 48 min 10 sec
CPU time 1 hours 46 min 59 sec
Validate state Invalid
Credit 0.00
Device peak FLOPS 4.82 GFLOPS
Application version OpenIFS 43r3 Baroclinic Lifecycle v1.11
x86_64-pc-linux-gnu
Peak working set size 5,075.20 MB
Peak swap size 5,930.79 MB
Peak disk usage 1,293.80 MB
Stderr
<core_client_version>7.20.5</core_client_version>
<![CDATA[
<message>
process exited with code 1 (0x1, -255)</message>
<stderr_txt>
20 (old at (nil))
[EC_DRHOOK:atlas7l1:1:1:2214789:2214789] [20230218:134834:1676724514:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:atlas7l1:1:1:2214789:2214789] [20230218:134834:1676724514:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:atlas7l1:1:1:2214789:2214789] [20230218:134834:1676724514:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:atlas7l1:1:1:2214789:2214789] [20230218:134834:1676724514:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f520 (old at (nil))
[EC_DRHOOK:atlas7l1:1:1:2214789:2214789] [20230218:134834:1676724514:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f520 (old at (nil))
[EC_DRHOOK:atlas7l1:1:1:2214789:2214789] [20230218:134834:1676724514:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f520 (old at (nil))
[EC_DRHOOK:atlas7l1:1:1:2214789:2214789] [20230218:134834:1676724514:0.000] [catch_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  13:48:56 STEP    0 H=   0:00 +CPU= 19.659
Moving to projects directory: /var/lib/boinc_data/boinc-02/slots/2/ICMGGh7zg+000000
Moving to projects directory: /var/lib/boinc_data/boinc-02/slots/2/ICMSHh7zg+000000
Moving to projects directory: /var/lib/boinc_data/boinc-02/slots/2/ICMUAh7zg+000000
  13:49:12 STEP    1 H=   0:15 +CPU= 16.120
  13:49:28 STEP    2 H=   0:30 +CPU= 15.995
  13:49:45 STEP    3 H=   0:45 +CPU= 16.358
  13:50:03 STEP    4 H=   1:00 +CPU= 18.180
  13:50:19 STEP    5 H=   1:15 +CPU= 16.127
  13:50:35 STEP    6 H=   1:30 +CPU= 16.323
  13:50:52 STEP    7 H=   1:45 +CPU= 16.473
  13:51:10 STEP    8 H=   2:00 +CPU= 17.555
  13:51:26 STEP    9 H=   2:15 +CPU= 16.735
  13:51:43 STEP   10 H=   2:30 +CPU= 16.475
  13:51:59 STEP   11 H=   2:45 +CPU= 16.073
  13:52:20 STEP   12 H=   3:00 +CPU= 20.536
Moving to projects directory: /var/lib/boinc_data/boinc-02/slots/2/ICMGGh7zg+000012
Moving to projects directory: /var/lib/boinc_data/boinc-02/slots/2/ICMSHh7zg+000012
Moving to projects directory: /var/lib/boinc_data/boinc-02/slots/2/ICMUAh7zg+000012
  13:52:37 STEP   13 H=   3:15 +CPU= 17.129
  13:52:54 STEP   14 H=   3:30 +CPU= 16.686
  13:53:10 STEP   15 H=   3:45 +CPU= 16.285
  13:53:27 STEP   16 H=   4:00 +CPU= 17.024
  13:53:44 STEP   17 H=   4:15 +CPU= 16.498
  13:54:01 STEP   18 H=   4:30 +CPU= 16.735
  13:54:18 STEP   19 H=   4:45 +CPU= 16.943
  13:54:35 STEP   20 H=   5:00 +CPU= 16.865
  13:54:51 STEP   21 H=   5:15 +CPU= 16.671
  13:55:08 STEP   22 H=   5:30 +CPU= 16.584
  13:55:24 STEP   23 H=   5:45 +CPU= 16.447
  13:55:45 STEP   24 H=   6:00 +CPU= 20.556
Moving to projects directory: /var/lib/boinc_data/boinc-02/slots/2/ICMGGh7zg+000024
Moving to projects directory: /var/lib/boinc_data/boinc-02/slots/2/ICMSHh7zg+000024
Moving to projects directory: /var/lib/boinc_data/boinc-02/slots/2/ICMUAh7zg+000024
  13:56:02 STEP   25 H=   6:15 +CPU= 16.730
  13:56:18 STEP   26 H=   6:30 +CPU= 16.457
  13:56:35 STEP   27 H=   6:45 +CPU= 16.585
  13:56:52 STEP   28 H=   7:00 +CPU= 16.568
  13:57:08 STEP   29 H=   7:15 +CPU= 16.358
  13:57:24 STEP   30 H=   7:30 +CPU= 16.409
  13:57:41 STEP   31 H=   7:45 +CPU= 16.233
  13:57:57 STEP   32 H=   8:00 +CPU= 16.794
  13:58:14 STEP   33 H=   8:15 +CPU= 16.432
  13:58:31 STEP   34 H=   8:30 +CPU= 16.932
  13:58:47 STEP   35 H=   8:45 +CPU= 16.527
  13:59:07 STEP   36 H=   9:00 +CPU= 20.032
Moving to projects directory: /var/lib/boinc_data/boinc-02/slots/2/ICMGGh7zg+000036
Moving to projects directory: /var/lib/boinc_data/boinc-02/slots/2/ICMSHh7zg+000036
Moving to projects directory: /var/lib/boinc_data/boinc-02/slots/2/ICMUAh7zg+000036
  13:59:24 STEP   37 H=   9:15 +CPU= 16.484
  13:59:40 STEP   38 H=   9:30 +CPU= 16.462
  13:59:57 STEP   39 H=   9:45 +CPU= 16.635
  14:00:14 STEP   40 H=  10:00 +CPU= 16.494
  14:00:30 STEP   41 H=  10:15 +CPU= 16.727
  14:00:47 STEP   42 H=  10:30 +CPU= 16.780
  14:01:04 STEP   43 H=  10:45 +CPU= 16.530
  14:01:21 STEP   44 H=  11:00 +CPU= 17.049
  14:01:37 STEP   45 H=  11:15 +CPU= 16.210
  14:01:53 STEP   46 H=  11:30 +CPU= 16.216
  14:02:09 STEP   47 H=  11:45 +CPU= 16.268
  14:02:32 STEP   48 H=  12:00 +CPU= 22.804
Moving to projects directory: /var/lib/boinc_data/boinc-02/slots/2/ICMGGh7zg+000048
Moving to projects directory: /var/lib/boinc_data/boinc-02/slots/2/ICMSHh7zg+000048
Moving to projects directory: /var/lib/boinc_data/boinc-02/slots/2/ICMUAh7zg+000048
  14:02:49 STEP   49 H=  12:15 +CPU= 16.260
  14:03:05 STEP   50 H=  12:30 +CPU= 16.444
  14:03:22 STEP   51 H=  12:45 +CPU= 17.293
  14:03:39 STEP   52 H=  13:00 +CPU= 16.540
  14:03:56 STEP   53 H=  13:15 +CPU= 17.056
  14:04:13 STEP   54 H=  13:30 +CPU= 16.919
  14:04:29 STEP   55 H=  13:45 +CPU= 16.191
  14:04:47 STEP   56 H=  14:00 +CPU= 17.877
  14:05:04 STEP   57 H=  14:15 +CPU= 16.763
  14:05:20 STEP   58 H=  14:30 +CPU= 16.232
  14:05:37 STEP   59 H=  14:45 +CPU= 17.283
  14:05:58 STEP   60 H=  15:00 +CPU= 20.341
Moving to projects directory: /var/lib/boinc_data/boinc-02/slots/2/ICMGGh7zg+000060
Moving to projects directory: /var/lib/boinc_data/boinc-02/slots/2/ICMSHh7zg+000060
Moving to projects directory: /var/lib/boinc_data/boinc-02/slots/2/ICMUAh7zg+000060
  14:06:14 STEP   61 H=  15:15 +CPU= 16.701
  14:06:31 STEP   62 H=  15:30 +CPU= 16.321
  14:06:47 STEP   63 H=  15:45 +CPU= 16.233
  14:07:04 STEP   64 H=  16:00 +CPU= 16.536
  14:07:20 STEP   65 H=  16:15 +CPU= 16.028
  14:07:36 STEP   66 H=  16:30 +CPU= 16.084
  14:07:52 STEP   67 H=  16:45 +CPU= 16.219
  14:08:08 STEP   68 H=  17:00 +CPU= 16.460
  14:08:25 STEP   69 H=  17:15 +CPU= 16.320
  14:08:41 STEP   70 H=  17:30 +CPU= 15.937
  14:08:57 STEP   71 H=  17:45 +CPU= 16.621
  14:09:17 STEP   72 H=  18:00 +CPU= 19.431
Moving to projects directory: /var/lib/boinc_data/boinc-02/slots/2/ICMGGh7zg+000072
Moving to projects directory: /var/lib/boinc_data/boinc-02/slots/2/ICMSHh7zg+000072
Moving to projects directory: /var/lib/boinc_data/boinc-02/slots/2/ICMUAh7zg+000072
  14:09:33 STEP   73 H=  18:15 +CPU= 15.841
  14:09:49 STEP   74 H=  18:30 +CPU= 16.302
  14:10:05 STEP   75 H=  18:45 +CPU= 15.784
  14:10:21 STEP   76 H=  19:00 +CPU= 16.013
  14:10:36 STEP   77 H=  19:15 +CPU= 15.801
  14:10:53 STEP   78 H=  19:30 +CPU= 16.610
  14:11:09 STEP   79 H=  19:45 +CPU= 15.886
  14:11:26 STEP   80 H=  20:00 +CPU= 16.567
  14:11:42 STEP   81 H=  20:15 +CPU= 16.377
  14:11:58 STEP   82 H=  20:30 +CPU= 16.152
  14:12:14 STEP   83 H=  20:45 +CPU= 16.212
  14:12:34 STEP   84 H=  21:00 +CPU= 19.869
Moving to projects directory: /var/lib/boinc_data/boinc-02/slots/2/ICMGGh7zg+000084
Moving to projects directory: /var/lib/boinc_data/boinc-02/slots/2/ICMSHh7zg+000084
Moving to projects directory: /var/lib/boinc_data/boinc-02/slots/2/ICMUAh7zg+000084
  14:12:50 STEP   85 H=  21:15 +CPU= 16.013
  14:13:06 STEP   86 H=  21:30 +CPU= 15.782
  14:13:23 STEP   87 H=  21:45 +CPU= 16.607
  14:13:40 STEP   88 H=  22:00 +CPU= 17.284
  14:13:56 STEP   89 H=  22:15 +CPU= 16.357
  14:14:12 STEP   90 H=  22:30 +CPU= 15.806
  14:14:28 STEP   91 H=  22:45 +CPU= 15.982
  14:14:45 STEP   92 H=  23:00 +CPU= 16.500
  14:15:01 STEP   93 H=  23:15 +CPU= 16.509
  14:15:17 STEP   94 H=  23:30 +CPU= 15.926
  14:15:33 STEP   95 H=  23:45 +CPU= 15.721
  14:15:57 STEP   96 H=  24:00 +CPU= 24.327
Moving to projects directory: /var/lib/boinc_data/boinc-02/slots/2/ICMGGh7zg+000096
Moving to projects directory: /var/lib/boinc_data/boinc-02/slots/2/ICMSHh7zg+000096
Moving to projects directory: /var/lib/boinc_data/boinc-02/slots/2/ICMUAh7zg+000096
Adding to the zip: /var/lib/boinc_data/boinc-02/projects/climateprediction.net/oifs_43r3_bl_12208366/ICMGGh7zg+000000
Adding to the zip: /var/lib/boinc_data/boinc-02/projects/climateprediction.net/oifs_43r3_bl_12208366/ICMSHh7zg+000000
Adding to the zip: /var/lib/boinc_data/boinc-02/projects/climateprediction.net/oifs_43r3_bl_12208366/ICMUAh7zg+000000
Adding to the zip: /var/lib/boinc_data/boinc-02/projects/climateprediction.net/oifs_43r3_bl_12208366/ICMGGh7zg+000012
Adding to the zip: /var/lib/boinc_data/boinc-02/projects/climateprediction.net/oifs_43r3_bl_12208366/ICMSHh7zg+000012
Adding to the zip: /var/lib/boinc_data/boinc-02/projects/climateprediction.net/oifs_43r3_bl_12208366/ICMUAh7zg+000012
Adding to the zip: /var/lib/boinc_data/boinc-02/projects/climateprediction.net/oifs_43r3_bl_12208366/ICMGGh7zg+000024
Adding to the zip: /var/lib/boinc_data/boinc-02/projects/climateprediction.net/oifs_43r3_bl_12208366/ICMSHh7zg+000024
Adding to the zip: /var/lib/boinc_data/boinc-02/projects/climateprediction.net/oifs_43r3_bl_12208366/ICMUAh7zg+000024
Adding to the zip: /var/lib/boinc_data/boinc-02/projects/climateprediction.net/oifs_43r3_bl_12208366/ICMGGh7zg+000036
Adding to the zip: /var/lib/boinc_data/boinc-02/projects/climateprediction.net/oifs_43r3_bl_12208366/ICMSHh7zg+000036
Adding to the zip: /var/lib/boinc_data/boinc-02/projects/climateprediction.net/oifs_43r3_bl_12208366/ICMUAh7zg+000036
Adding to the zip: /var/lib/boinc_data/boinc-02/projects/climateprediction.net/oifs_43r3_bl_12208366/ICMGGh7zg+000048
Adding to the zip: /var/lib/boinc_data/boinc-02/projects/climateprediction.net/oifs_43r3_bl_12208366/ICMSHh7zg+000048
Adding to the zip: /var/lib/boinc_data/boinc-02/projects/climateprediction.net/oifs_43r3_bl_12208366/ICMUAh7zg+000048
Adding to the zip: /var/lib/boinc_data/boinc-02/projects/climateprediction.net/oifs_43r3_bl_12208366/ICMGGh7zg+000060
Adding to the zip: /var/lib/boinc_data/boinc-02/projects/climateprediction.net/oifs_43r3_bl_12208366/ICMSHh7zg+000060
Adding to the zip: /var/lib/boinc_data/boinc-02/projects/climateprediction.net/oifs_43r3_bl_12208366/ICMUAh7zg+000060
Adding to the zip: /var/lib/boinc_data/boinc-02/projects/climateprediction.net/oifs_43r3_bl_12208366/ICMGGh7zg+000072
Adding to the zip: /var/lib/boinc_data/boinc-02/projects/climateprediction.net/oifs_43r3_bl_12208366/ICMSHh7zg+000072
Adding to the zip: /var/lib/boinc_data/boinc-02/projects/climateprediction.net/oifs_43r3_bl_12208366/ICMUAh7zg+000072
Adding to the zip: /var/lib/boinc_data/boinc-02/projects/climateprediction.net/oifs_43r3_bl_12208366/ICMGGh7zg+000084
Adding to the zip: /var/lib/boinc_data/boinc-02/projects/climateprediction.net/oifs_43r3_bl_12208366/ICMSHh7zg+000084
Adding to the zip: /var/lib/boinc_data/boinc-02/projects/climateprediction.net/oifs_43r3_bl_12208366/ICMUAh7zg+000084
Zipping up the intermediate file: /var/lib/boinc_data/boinc-02/projects/climateprediction.net/oifs_43r3_bl_a17v_2016092300_15_991_12208366_1_r270612951_0.zip
  14:16:13 STEP   97 H=  24:15 +CPU= 15.989
Uploading the intermediate file: upload_file_0.zip
  14:16:30 STEP   98 H=  24:30 +CPU= 16.219
  14:16:46 STEP   99 H=  24:45 +CPU= 15.947
  14:17:02 STEP  100 H=  25:00 +CPU= 16.356
  14:17:19 STEP  101 H=  25:15 +CPU= 16.762
  14:17:36 STEP  102 H=  25:30 +CPU= 17.113
  14:17:52 STEP  103 H=  25:45 +CPU= 15.990
  14:18:08 STEP  104 H=  26:00 +CPU= 16.030
  14:18:24 STEP  105 H=  26:15 +CPU= 15.695
  14:18:40 STEP  106 H=  26:30 +CPU= 15.782
  14:18:56 STEP  107 H=  26:45 +CPU= 16.080
  14:19:15 STEP  108 H=  27:00 +CPU= 19.500
Moving to projects directory: /var/lib/boinc_data/boinc-02/slots/2/ICMGGh7zg+000108
Moving to projects directory: /var/lib/boinc_data/boinc-02/slots/2/ICMSHh7zg+000108
Moving to projects directory: /var/lib/boinc_data/boinc-02/slots/2/ICMUAh7zg+000108
  14:19:32 STEP  109 H=  27:15 +CPU= 16.348
  14:19:48 STEP  110 H=  27:30 +CPU= 16.057
  14:20:03 STEP  111 H=  27:45 +CPU= 15.640
  14:20:19 STEP  112 H=  28:00 +CPU= 15.878
  14:20:35 STEP  113 H=  28:15 +CPU= 15.576
  14:20:50 STEP  114 H=  28:30 +CPU= 15.590
  14:21:06 STEP  115 H=  28:45 +CPU= 15.583
  14:21:22 STEP  116 H=  29:00 +CPU= 15.922
  14:21:38 STEP  117 H=  29:15 +CPU= 16.150
  14:21:54 STEP  118 H=  29:30 +CPU= 16.087
  14:22:10 STEP  119 H=  29:45 +CPU= 15.643
  14:22:29 STEP  120 H=  30:00 +CPU= 19.527
Moving to projects directory: /var/lib/boinc_data/boinc-02/slots/2/ICMGGh7zg+000120
Moving to projects directory: /var/lib/boinc_data/boinc-02/slots/2/ICMSHh7zg+000120
Moving to projects directory: /var/lib/boinc_data/boinc-02/slots/2/ICMUAh7zg+000120
  14:22:45 STEP  121 H=  30:15 +CPU= 15.749
  14:23:02 STEP  122 H=  30:30 +CPU= 16.945
  14:23:18 STEP  123 H=  30:45 +CPU= 15.836
  14:23:34 STEP  124 H=  31:00 +CPU= 16.099
  14:23:50 STEP  125 H=  31:15 +CPU= 15.709
  14:24:05 STEP  126 H=  31:30 +CPU= 15.821
  14:24:21 STEP  127 H=  31:45 +CPU= 15.777
  14:24:38 STEP  128 H=  32:00 +CPU= 16.463
  14:24:53 STEP  129 H=  32:15 +CPU= 15.741
  14:25:09 STEP  130 H=  32:30 +CPU= 15.850
  14:25:25 STEP  131 H=  32:45 +CPU= 15.662
  14:25:44 STEP  132 H=  33:00 +CPU= 19.486
Moving to projects directory: /var/lib/boinc_data/boinc-02/slots/2/ICMGGh7zg+000132
Moving to projects directory: /var/lib/boinc_data/boinc-02/slots/2/ICMSHh7zg+000132
Moving to projects directory: /var/lib/boinc_data/boinc-02/slots/2/ICMUAh7zg+000132
  14:26:00 STEP  133 H=  33:15 +CPU= 15.753
  14:26:16 STEP  134 H=  33:30 +CPU= 16.070
  14:26:32 STEP  135 H=  33:45 +CPU= 15.965
  14:26:48 STEP  136 H=  34:00 +CPU= 15.828
  14:27:04 STEP  137 H=  34:15 +CPU= 15.851
  14:27:20 STEP  138 H=  34:30 +CPU= 15.808
  14:27:36 STEP  139 H=  34:45 +CPU= 15.723
  14:27:51 STEP  140 H=  35:00 +CPU= 15.820
  14:28:07 STEP  141 H=  35:15 +CPU= 15.368
  14:28:23 STEP  142 H=  35:30 +CPU= 15.932
  14:28:38 STEP  143 H=  35:45 +CPU= 15.793
  14:29:01 STEP  144 H=  36:00 +CPU= 22.226
Moving to projects directory: /var/lib/boinc_data/boinc-02/slots/2/ICMGGh7zg+000144
Moving to projects directory: /var/lib/boinc_data/boinc-02/slots/2/ICMSHh7zg+000144
Moving to projects directory: /var/lib/boinc_data/boinc-02/slots/2/ICMUAh7zg+000144
  14:29:16 STEP  145 H=  36:15 +CPU= 15.769
  14:29:32 STEP  146 H=  36:30 +CPU= 15.823
  14:29:48 STEP  147 H=  36:45 +CPU= 16.064
  14:30:04 STEP  148 H=  37:00 +CPU= 15.682
  14:30:19 STEP  149 H=  37:15 +CPU= 15.274
  14:30:35 STEP  150 H=  37:30 +CPU= 15.218
  14:30:50 STEP  151 H=  37:45 +CPU= 15.531
  14:31:06 STEP  152 H=  38:00 +CPU= 15.734
  14:31:22 STEP  153 H=  38:15 +CPU= 15.781
  14:31:37 STEP  154 H=  38:30 +CPU= 15.556
  14:31:53 STEP  155 H=  38:45 +CPU= 16.159
  14:32:13 STEP  156 H=  39:00 +CPU= 19.679
Moving to projects directory: /var/lib/boinc_data/boinc-02/slots/2/ICMGGh7zg+000156
Moving to projects directory: /var/lib/boinc_data/boinc-02/slots/2/ICMSHh7zg+000156
Moving to projects directory: /var/lib/boinc_data/boinc-02/slots/2/ICMUAh7zg+000156
  14:32:29 STEP  157 H=  39:15 +CPU= 16.005
  14:32:45 STEP  158 H=  39:30 +CPU= 15.662
  14:33:01 STEP  159 H=  39:45 +CPU= 15.962
  14:33:17 STEP  160 H=  40:00 +CPU= 15.809
  14:33:32 STEP  161 H=  40:15 +CPU= 15.553
  14:33:48 STEP  162 H=  40:30 +CPU= 15.571
  14:34:04 STEP  163 H=  40:45 +CPU= 16.182
  14:34:20 STEP  164 H=  41:00 +CPU= 16.262
  14:34:36 STEP  165 H=  41:15 +CPU= 15.579
  14:34:52 STEP  166 H=  41:30 +CPU= 16.069
  14:35:07 STEP  167 H=  41:45 +CPU= 15.672
  14:35:27 STEP  168 H=  42:00 +CPU= 19.315
Moving to projects directory: /var/lib/boinc_data/boinc-02/slots/2/ICMGGh7zg+000168
Moving to projects directory: /var/lib/boinc_data/boinc-02/slots/2/ICMSHh7zg+000168
Moving to projects directory: /var/lib/boinc_data/boinc-02/slots/2/ICMUAh7zg+000168
  14:35:43 STEP  169 H=  42:15 +CPU= 16.506
  14:35:59 STEP  170 H=  42:30 +CPU= 15.469
  14:36:15 STEP  171 H=  42:45 +CPU= 16.369
  14:36:32 STEP  172 H=  43:00 +CPU= 16.311
  14:36:48 STEP  173 H=  43:15 +CPU= 16.488
  14:37:03 STEP  174 H=  43:30 +CPU= 15.410
  14:37:19 STEP  175 H=  43:45 +CPU= 15.712
  14:37:35 STEP  176 H=  44:00 +CPU= 16.257
  14:37:51 STEP  177 H=  44:15 +CPU= 15.866
  14:38:07 STEP  178 H=  44:30 +CPU= 15.523
  14:38:22 STEP  179 H=  44:45 +CPU= 15.337
  14:38:42 STEP  180 H=  45:00 +CPU= 19.761
Moving to projects directory: /var/lib/boinc_data/boinc-02/slots/2/ICMGGh7zg+000180
Moving to projects directory: /var/lib/boinc_data/boinc-02/slots/2/ICMSHh7zg+000180
Moving to projects directory: /var/lib/boinc_data/boinc-02/slots/2/ICMUAh7zg+000180
  14:38:58 STEP  181 H=  45:15 +CPU= 15.655
  14:39:13 STEP  182 H=  45:30 +CPU= 15.623
  14:39:29 STEP  183 H=  45:45 +CPU= 15.523
  14:39:45 STEP  184 H=  46:00 +CPU= 15.825
  14:40:00 STEP  185 H=  46:15 +CPU= 15.407
  14:40:16 STEP  186 H=  46:30 +CPU= 15.467
  14:40:31 STEP  187 H=  46:45 +CPU= 15.635
  14:40:47 STEP  188 H=  47:00 +CPU= 15.604
  14:41:02 STEP  189 H=  47:15 +CPU= 15.612
  14:41:18 STEP  190 H=  47:30 +CPU= 15.347
  14:41:33 STEP  191 H=  47:45 +CPU= 15.405
  14:41:56 STEP  192 H=  48:00 +CPU= 23.048
Moving to projects directory: /var/lib/boinc_data/boinc-02/slots/2/ICMGGh7zg+000192
Moving to projects directory: /var/lib/boinc_data/boinc-02/slots/2/ICMSHh7zg+000192
Moving to projects directory: /var/lib/boinc_data/boinc-02/slots/2/ICMUAh7zg+000192
Adding to the zip: /var/lib/boinc_data/boinc-02/projects/climateprediction.net/oifs_43r3_bl_12208366/ICMGGh7zg+000096
Adding to the zip: /var/lib/boinc_data/boinc-02/projects/climateprediction.net/oifs_43r3_bl_12208366/ICMSHh7zg+000096
Adding to the zip: /var/lib/boinc_data/boinc-02/projects/climateprediction.net/oifs_43r3_bl_12208366/ICMUAh7zg+000096
Adding to the zip: /var/lib/boinc_data/boinc-02/projects/climateprediction.net/oifs_43r3_bl_12208366/ICMGGh7zg+000108
Adding to the zip: /var/lib/boinc_data/boinc-02/projects/climateprediction.net/oifs_43r3_bl_12208366/ICMSHh7zg+000108
Adding to the zip: /var/lib/boinc_data/boinc-02/projects/climateprediction.net/oifs_43r3_bl_12208366/ICMUAh7zg+000108
Adding to the zip: /var/lib/boinc_data/boinc-02/projects/climateprediction.net/oifs_43r3_bl_12208366/ICMGGh7zg+000120
Adding to the zip: /var/lib/boinc_data/boinc-02/projects/climateprediction.net/oifs_43r3_bl_12208366/ICMSHh7zg+000120
Adding to the zip: /var/lib/boinc_data/boinc-02/projects/climateprediction.net/oifs_43r3_bl_12208366/ICMUAh7zg+000120
Adding to the zip: /var/lib/boinc_data/boinc-02/projects/climateprediction.net/oifs_43r3_bl_12208366/ICMGGh7zg+000132
Adding to the zip: /var/lib/boinc_data/boinc-02/projects/climateprediction.net/oifs_43r3_bl_12208366/ICMSHh7zg+000132
Adding to the zip: /var/lib/boinc_data/boinc-02/projects/climateprediction.net/oifs_43r3_bl_12208366/ICMUAh7zg+000132
Adding to the zip: /var/lib/boinc_data/boinc-02/projects/climateprediction.net/oifs_43r3_bl_12208366/ICMGGh7zg+000144
Adding to the zip: /var/lib/boinc_data/boinc-02/projects/climateprediction.net/oifs_43r3_bl_12208366/ICMSHh7zg+000144
Adding to the zip: /var/lib/boinc_data/boinc-02/projects/climateprediction.net/oifs_43r3_bl_12208366/ICMUAh7zg+000144
Adding to the zip: /var/lib/boinc_data/boinc-02/projects/climateprediction.net/oifs_43r3_bl_12208366/ICMGGh7zg+000156
Adding to the zip: /var/lib/boinc_data/boinc-02/projects/climateprediction.net/oifs_43r3_bl_12208366/ICMSHh7zg+000156
Adding to the zip: /var/lib/boinc_data/boinc-02/projects/climateprediction.net/oifs_43r3_bl_12208366/ICMUAh7zg+000156
Adding to the zip: /var/lib/boinc_data/boinc-02/projects/climateprediction.net/oifs_43r3_bl_12208366/ICMGGh7zg+000168
Adding to the zip: /var/lib/boinc_data/boinc-02/projects/climateprediction.net/oifs_43r3_bl_12208366/ICMSHh7zg+000168
Adding to the zip: /var/lib/boinc_data/boinc-02/projects/climateprediction.net/oifs_43r3_bl_12208366/ICMUAh7zg+000168
Adding to the zip: /var/lib/boinc_data/boinc-02/projects/climateprediction.net/oifs_43r3_bl_12208366/ICMGGh7zg+000180
Adding to the zip: /var/lib/boinc_data/boinc-02/projects/climateprediction.net/oifs_43r3_bl_12208366/ICMSHh7zg+000180
Adding to the zip: /var/lib/boinc_data/boinc-02/projects/climateprediction.net/oifs_43r3_bl_12208366/ICMUAh7zg+000180
Zipping up the intermediate file: /var/lib/boinc_data/boinc-02/projects/climateprediction.net/oifs_43r3_bl_a17v_2016092300_15_991_12208366_1_r270612951_1.zip
Uploading the intermediate file: upload_file_1.zip
  14:42:12 STEP  193 H=  48:15 +CPU= 15.351
  14:42:27 STEP  194 H=  48:30 +CPU= 15.434
  14:42:42 STEP  195 H=  48:45 +CPU= 15.397
  14:42:59 STEP  196 H=  49:00 +CPU= 16.473
  14:43:15 STEP  197 H=  49:15 +CPU= 15.623
  14:43:30 STEP  198 H=  49:30 +CPU= 15.403
  14:43:46 STEP  199 H=  49:45 +CPU= 15.733
  14:44:02 STEP  200 H=  50:00 +CPU= 16.064
  14:44:17 STEP  201 H=  50:15 +CPU= 15.246
  14:44:32 STEP  202 H=  50:30 +CPU= 15.234
  14:44:48 STEP  203 H=  50:45 +CPU= 15.288
  14:45:07 STEP  204 H=  51:00 +CPU= 19.513
Moving to projects directory: /var/lib/boinc_data/boinc-02/slots/2/ICMGGh7zg+000204
Moving to projects directory: /var/lib/boinc_data/boinc-02/slots/2/ICMSHh7zg+000204
Moving to projects directory: /var/lib/boinc_data/boinc-02/slots/2/ICMUAh7zg+000204
  14:45:22 STEP  205 H=  51:15 +CPU= 15.253
  14:45:38 STEP  206 H=  51:30 +CPU= 15.472
  14:45:54 STEP  207 H=  51:45 +CPU= 15.625
  14:46:09 STEP  208 H=  52:00 +CPU= 15.480
  14:46:24 STEP  209 H=  52:15 +CPU= 15.202
  14:46:39 STEP  210 H=  52:30 +CPU= 15.170
  14:46:55 STEP  211 H=  52:45 +CPU= 15.428
  14:47:11 STEP  212 H=  53:00 +CPU= 15.612
  14:47:26 STEP  213 H=  53:15 +CPU= 15.287
  14:47:42 STEP  214 H=  53:30 +CPU= 15.789
  14:47:57 STEP  215 H=  53:45 +CPU= 15.625
  14:48:17 STEP  216 H=  54:00 +CPU= 19.309
Moving to projects directory: /var/lib/boinc_data/boinc-02/slots/2/ICMGGh7zg+000216
Moving to projects directory: /var/lib/boinc_data/boinc-02/slots/2/ICMSHh7zg+000216
Moving to projects directory: /var/lib/boinc_data/boinc-02/slots/2/ICMUAh7zg+000216
  14:48:32 STEP  217 H=  54:15 +CPU= 15.421
  14:48:48 STEP  218 H=  54:30 +CPU= 15.625
  14:49:03 STEP  219 H=  54:45 +CPU= 15.691
  14:49:20 STEP  220 H=  55:00 +CPU= 16.345
  14:49:35 STEP  221 H=  55:15 +CPU= 15.316
  14:49:51 STEP  222 H=  55:30 +CPU= 16.053
  14:50:08 STEP  223 H=  55:45 +CPU= 16.340
  14:50:24 STEP  224 H=  56:00 +CPU= 16.167
  14:50:40 STEP  225 H=  56:15 +CPU= 16.130
  14:50:55 STEP  226 H=  56:30 +CPU= 15.245
  14:51:10 STEP  227 H=  56:45 +CPU= 15.267
  14:51:30 STEP  228 H=  57:00 +CPU= 19.416
Moving to projects directory: /var/lib/boinc_data/boinc-02/slots/2/ICMGGh7zg+000228
Moving to projects directory: /var/lib/boinc_data/boinc-02/slots/2/ICMSHh7zg+000228
Moving to projects directory: /var/lib/boinc_data/boinc-02/slots/2/ICMUAh7zg+000228
  14:51:45 STEP  229 H=  57:15 +CPU= 15.382
  14:52:01 STEP  230 H=  57:30 +CPU= 15.610
  14:52:16 STEP  231 H=  57:45 +CPU= 15.277
  14:52:32 STEP  232 H=  58:00 +CPU= 15.524
  14:52:47 STEP  233 H=  58:15 +CPU= 15.429
  14:53:03 STEP  234 H=  58:30 +CPU= 15.407
  14:53:18 STEP  235 H=  58:45 +CPU= 15.423
  14:53:34 STEP  236 H=  59:00 +CPU= 15.813
  14:53:50 STEP  237 H=  59:15 +CPU= 15.989
  14:54:06 STEP  238 H=  59:30 +CPU= 15.451
  14:54:21 STEP  239 H=  59:45 +CPU= 15.607
  14:54:44 STEP  240 H=  60:00 +CPU= 22.595
Moving to projects directory: /var/lib/boinc_data/boinc-02/slots/2/ICMGGh7zg+000240
Moving to projects directory: /var/lib/boinc_data/boinc-02/slots/2/ICMSHh7zg+000240
Moving to projects directory: /var/lib/boinc_data/boinc-02/slots/2/ICMUAh7zg+000240
  14:54:59 STEP  241 H=  60:15 +CPU= 15.406
  14:55:15 STEP  242 H=  60:30 +CPU= 15.531
  14:55:30 STEP  243 H=  60:45 +CPU= 15.558
  14:55:46 STEP  244 H=  61:00 +CPU= 15.885
  14:56:01 STEP  245 H=  61:15 +CPU= 15.139
  14:56:17 STEP  246 H=  61:30 +CPU= 15.379
  14:56:32 STEP  247 H=  61:45 +CPU= 15.420
  14:56:48 STEP  248 H=  62:00 +CPU= 15.858
  14:57:04 STEP  249 H=  62:15 +CPU= 15.665
  14:57:19 STEP  250 H=  62:30 +CPU= 15.367
  14:57:35 STEP  251 H=  62:45 +CPU= 15.774
  14:57:54 STEP  252 H=  63:00 +CPU= 19.567
Moving to projects directory: /var/lib/boinc_data/boinc-02/slots/2/ICMGGh7zg+000252
Moving to projects directory: /var/lib/boinc_data/boinc-02/slots/2/ICMSHh7zg+000252
Moving to projects directory: /var/lib/boinc_data/boinc-02/slots/2/ICMUAh7zg+000252
  14:58:10 STEP  253 H=  63:15 +CPU= 15.830
  14:58:26 STEP  254 H=  63:30 +CPU= 15.562
  14:58:41 STEP  255 H=  63:45 +CPU= 15.424
  14:58:57 STEP  256 H=  64:00 +CPU= 15.858
  14:59:13 STEP  257 H=  64:15 +CPU= 15.803
  14:59:28 STEP  258 H=  64:30 +CPU= 15.339
  14:59:43 STEP  259 H=  64:45 +CPU= 15.140
  14:59:59 STEP  260 H=  65:00 +CPU= 15.980
  15:00:15 STEP  261 H=  65:15 +CPU= 15.362
  15:00:30 STEP  262 H=  65:30 +CPU= 15.395
  15:00:45 STEP  263 H=  65:45 +CPU= 15.329
  15:01:05 STEP  264 H=  66:00 +CPU= 19.109
Moving to projects directory: /var/lib/boinc_data/boinc-02/slots/2/ICMGGh7zg+000264
Moving to projects directory: /var/lib/boinc_data/boinc-02/slots/2/ICMSHh7zg+000264
Moving to projects directory: /var/lib/boinc_data/boinc-02/slots/2/ICMUAh7zg+000264
  15:01:20 STEP  265 H=  66:15 +CPU= 15.193
  15:01:35 STEP  266 H=  66:30 +CPU= 15.677
  15:01:51 STEP  267 H=  66:45 +CPU= 15.525
  15:02:07 STEP  268 H=  67:00 +CPU= 15.703
  15:02:22 STEP  269 H=  67:15 +CPU= 15.069
  15:02:37 STEP  270 H=  67:30 +CPU= 15.182
  15:02:53 STEP  271 H=  67:45 +CPU= 16.018
  15:03:09 STEP  272 H=  68:00 +CPU= 15.500
  15:03:24 STEP  273 H=  68:15 +CPU= 15.646
  15:03:40 STEP  274 H=  68:30 +CPU= 15.482
  15:03:55 STEP  275 H=  68:45 +CPU= 15.139
  15:04:14 STEP  276 H=  69:00 +CPU= 19.138
Moving to projects directory: /var/lib/boinc_data/boinc-02/slots/2/ICMGGh7zg+000276
Moving to projects directory: /var/lib/boinc_data/boinc-02/slots/2/ICMSHh7zg+000276
Moving to projects directory: /var/lib/boinc_data/boinc-02/slots/2/ICMUAh7zg+000276
  15:04:29 STEP  277 H=  69:15 +CPU= 15.086
  15:04:45 STEP  278 H=  69:30 +CPU= 15.730
  15:05:01 STEP  279 H=  69:45 +CPU= 16.225
  15:05:17 STEP  280 H=  70:00 +CPU= 15.845
  15:05:34 STEP  281 H=  70:15 +CPU= 17.245
  15:05:50 STEP  282 H=  70:30 +CPU= 15.719
  15:06:06 STEP  283 H=  70:45 +CPU= 15.847
  15:06:22 STEP  284 H=  71:00 +CPU= 15.572
  15:06:37 STEP  285 H=  71:15 +CPU= 15.785
  15:06:53 STEP  286 H=  71:30 +CPU= 15.295
  15:07:08 STEP  287 H=  71:45 +CPU= 15.534
  15:07:30 STEP  288 H=  72:00 +CPU= 21.939
Moving to projects directory: /var/lib/boinc_data/boinc-02/slots/2/ICMGGh7zg+000288
Moving to projects directory: /var/lib/boinc_data/boinc-02/slots/2/ICMSHh7zg+000288
Moving to projects directory: /var/lib/boinc_data/boinc-02/slots/2/ICMUAh7zg+000288
Adding to the zip: /var/lib/boinc_data/boinc-02/projects/climateprediction.net/oifs_43r3_bl_12208366/ICMGGh7zg+000192
Adding to the zip: /var/lib/boinc_data/boinc-02/projects/climateprediction.net/oifs_43r3_bl_12208366/ICMSHh7zg+000192
Adding to the zip: /var/lib/boinc_data/boinc-02/projects/climateprediction.net/oifs_43r3_bl_12208366/ICMUAh7zg+000192
Adding to the zip: /var/lib/boinc_data/boinc-02/projects/climateprediction.net/oifs_43r3_bl_12208366/ICMGGh7zg+000204
Adding to the zip: /var/lib/boinc_data/boinc-02/projects/climateprediction.net/oifs_43r3_bl_12208366/ICMSHh7zg+000204
Adding to the zip: /var/lib/boinc_data/boinc-02/projects/climateprediction.net/oifs_43r3_bl_12208366/ICMUAh7zg+000204
Adding to the zip: /var/lib/boinc_data/boinc-02/projects/climateprediction.net/oifs_43r3_bl_12208366/ICMGGh7zg+000216
Adding to the zip: /var/lib/boinc_data/boinc-02/projects/climateprediction.net/oifs_43r3_bl_12208366/ICMSHh7zg+000216
Adding to the zip: /var/lib/boinc_data/boinc-02/projects/climateprediction.net/oifs_43r3_bl_12208366/ICMUAh7zg+000216
Adding to the zip: /var/lib/boinc_data/boinc-02/projects/climateprediction.net/oifs_43r3_bl_12208366/ICMGGh7zg+000228
Adding to the zip: /var/lib/boinc_data/boinc-02/projects/climateprediction.net/oifs_43r3_bl_12208366/ICMSHh7zg+000228
Adding to the zip: /var/lib/boinc_data/boinc-02/projects/climateprediction.net/oifs_43r3_bl_12208366/ICMUAh7zg+000228
Adding to the zip: /var/lib/boinc_data/boinc-02/projects/climateprediction.net/oifs_43r3_bl_12208366/ICMGGh7zg+000240
Adding to the zip: /var/lib/boinc_data/boinc-02/projects/climateprediction.net/oifs_43r3_bl_12208366/ICMSHh7zg+000240
Adding to the zip: /var/lib/boinc_data/boinc-02/projects/climateprediction.net/oifs_43r3_bl_12208366/ICMUAh7zg+000240
Adding to the zip: /var/lib/boinc_data/boinc-02/projects/climateprediction.net/oifs_43r3_bl_12208366/ICMGGh7zg+000252
Adding to the zip: /var/lib/boinc_data/boinc-02/projects/climateprediction.net/oifs_43r3_bl_12208366/ICMSHh7zg+000252
Adding to the zip: /var/lib/boinc_data/boinc-02/projects/climateprediction.net/oifs_43r3_bl_12208366/ICMUAh7zg+000252
Adding to the zip: /var/lib/boinc_data/boinc-02/projects/climateprediction.net/oifs_43r3_bl_12208366/ICMGGh7zg+000264
Adding to the zip: /var/lib/boinc_data/boinc-02/projects/climateprediction.net/oifs_43r3_bl_12208366/ICMSHh7zg+000264
Adding to the zip: /var/lib/boinc_data/boinc-02/projects/climateprediction.net/oifs_43r3_bl_12208366/ICMUAh7zg+000264
Adding to the zip: /var/lib/boinc_data/boinc-02/projects/climateprediction.net/oifs_43r3_bl_12208366/ICMGGh7zg+000276
Adding to the zip: /var/lib/boinc_data/boinc-02/projects/climateprediction.net/oifs_43r3_bl_12208366/ICMSHh7zg+000276
Adding to the zip: /var/lib/boinc_data/boinc-02/projects/climateprediction.net/oifs_43r3_bl_12208366/ICMUAh7zg+000276
Zipping up the intermediate file: /var/lib/boinc_data/boinc-02/projects/climateprediction.net/oifs_43r3_bl_a17v_2016092300_15_991_12208366_1_r270612951_2.zip
Uploading the intermediate file: upload_file_2.zip
  15:07:46 STEP  289 H=  72:15 +CPU= 16.211
  15:08:02 STEP  290 H=  72:30 +CPU= 15.327
  15:08:17 STEP  291 H=  72:45 +CPU= 15.220
  15:08:33 STEP  292 H=  73:00 +CPU= 15.733
  15:08:49 STEP  293 H=  73:15 +CPU= 15.702
  15:09:04 STEP  294 H=  73:30 +CPU= 15.730
  15:09:20 STEP  295 H=  73:45 +CPU= 15.725
  15:09:36 STEP  296 H=  74:00 +CPU= 15.701
  15:09:52 STEP  297 H=  74:15 +CPU= 16.694
  15:10:08 STEP  298 H=  74:30 +CPU= 15.552
  15:10:23 STEP  299 H=  74:45 +CPU= 15.207
  15:10:43 STEP  300 H=  75:00 +CPU= 19.473
Moving to projects directory: /var/lib/boinc_data/boinc-02/slots/2/ICMGGh7zg+000300
Moving to projects directory: /var/lib/boinc_data/boinc-02/slots/2/ICMSHh7zg+000300
Moving to projects directory: /var/lib/boinc_data/boinc-02/slots/2/ICMUAh7zg+000300
  15:10:58 STEP  301 H=  75:15 +CPU= 15.638
  15:11:13 STEP  302 H=  75:30 +CPU= 15.138
  15:11:29 STEP  303 H=  75:45 +CPU= 15.178
  15:11:44 STEP  304 H=  76:00 +CPU= 15.466
  15:12:00 STEP  305 H=  76:15 +CPU= 15.605
  15:12:15 STEP  306 H=  76:30 +CPU= 15.489
  15:12:31 STEP  307 H=  76:45 +CPU= 15.298
  15:12:46 STEP  308 H=  77:00 +CPU= 15.636
  15:13:02 STEP  309 H=  77:15 +CPU= 16.156
  15:13:18 STEP  310 H=  77:30 +CPU= 15.882
  15:13:34 STEP  311 H=  77:45 +CPU= 15.579
  15:13:53 STEP  312 H=  78:00 +CPU= 19.126
Moving to projects directory: /var/lib/boinc_data/boinc-02/slots/2/ICMGGh7zg+000312
Moving to projects directory: /var/lib/boinc_data/boinc-02/slots/2/ICMSHh7zg+000312
Moving to projects directory: /var/lib/boinc_data/boinc-02/slots/2/ICMUAh7zg+000312
  15:14:08 STEP  313 H=  78:15 +CPU= 15.371
  15:14:24 STEP  314 H=  78:30 +CPU= 15.322
  15:14:39 STEP  315 H=  78:45 +CPU= 15.258
  15:14:55 STEP  316 H=  79:00 +CPU= 16.181
  15:15:11 STEP  317 H=  79:15 +CPU= 15.343
  15:15:26 STEP  318 H=  79:30 +CPU= 15.347
  15:15:42 STEP  319 H=  79:45 +CPU= 15.710
  15:15:57 STEP  320 H=  80:00 +CPU= 15.734
  15:16:13 STEP  321 H=  80:15 +CPU= 15.426
  15:16:29 STEP  322 H=  80:30 +CPU= 15.748
  15:16:44 STEP  323 H=  80:45 +CPU= 15.467
  15:17:04 STEP  324 H=  81:00 +CPU= 19.709
Moving to projects directory: /var/lib/boinc_data/boinc-02/slots/2/ICMGGh7zg+000324
Moving to projects directory: /var/lib/boinc_data/boinc-02/slots/2/ICMSHh7zg+000324
Moving to projects directory: /var/lib/boinc_data/boinc-02/slots/2/ICMUAh7zg+000324
  15:17:19 STEP  325 H=  81:15 +CPU= 15.411
  15:17:35 STEP  326 H=  81:30 +CPU= 15.649
  15:17:51 STEP  327 H=  81:45 +CPU= 15.625
  15:18:07 STEP  328 H=  82:00 +CPU= 16.306
  15:18:22 STEP  329 H=  82:15 +CPU= 15.392
  15:18:38 STEP  330 H=  82:30 +CPU= 15.387
  15:18:53 STEP  331 H=  82:45 +CPU= 15.390
  15:19:09 STEP  332 H=  83:00 +CPU= 16.274
  15:19:25 STEP  333 H=  83:15 +CPU= 15.564
  15:19:40 STEP  334 H=  83:30 +CPU= 15.307
  15:19:56 STEP  335 H=  83:45 +CPU= 15.312
  15:20:18 STEP  336 H=  84:00 +CPU= 22.798
Moving to projects directory: /var/lib/boinc_data/boinc-02/slots/2/ICMGGh7zg+000336
Moving to projects directory: /var/lib/boinc_data/boinc-02/slots/2/ICMSHh7zg+000336
Moving to projects directory: /var/lib/boinc_data/boinc-02/slots/2/ICMUAh7zg+000336
  15:20:34 STEP  337 H=  84:15 +CPU= 15.407
  15:20:49 STEP  338 H=  84:30 +CPU= 15.573
  15:21:05 STEP  339 H=  84:45 +CPU= 15.753
  15:21:21 STEP  340 H=  85:00 +CPU= 15.568
  15:21:37 STEP  341 H=  85:15 +CPU= 15.845
  15:21:52 STEP  342 H=  85:30 +CPU= 15.548
  15:22:08 STEP  343 H=  85:45 +CPU= 15.417
  15:22:24 STEP  344 H=  86:00 +CPU= 16.854
  15:22:40 STEP  345 H=  86:15 +CPU= 15.288
  15:22:55 STEP  346 H=  86:30 +CPU= 15.358
  15:23:11 STEP  347 H=  86:45 +CPU= 16.363
  15:23:31 STEP  348 H=  87:00 +CPU= 19.297
Moving to projects directory: /var/lib/boinc_data/boinc-02/slots/2/ICMGGh7zg+000348
Moving to projects directory: /var/lib/boinc_data/boinc-02/slots/2/ICMSHh7zg+000348
Moving to projects directory: /var/lib/boinc_data/boinc-02/slots/2/ICMUAh7zg+000348
  15:23:46 STEP  349 H=  87:15 +CPU= 15.547
  15:24:02 STEP  350 H=  87:30 +CPU= 15.562
  15:24:18 STEP  351 H=  87:45 +CPU= 16.212
  15:24:34 STEP  352 H=  88:00 +CPU= 15.837
  15:24:50 STEP  353 H=  88:15 +CPU= 15.578
  15:25:05 STEP  354 H=  88:30 +CPU= 15.794
  15:25:21 STEP  355 H=  88:45 +CPU= 16.147
  15:25:37 STEP  356 H=  89:00 +CPU= 15.723
  15:25:53 STEP  357 H=  89:15 +CPU= 15.441
  15:26:09 STEP  358 H=  89:30 +CPU= 15.907
  15:26:25 STEP  359 H=  89:45 +CPU= 16.074
  15:26:44 STEP  360 H=  90:00 +CPU= 19.532
Moving to projects directory: /var/lib/boinc_data/boinc-02/slots/2/ICMGGh7zg+000360
Moving to projects directory: /var/lib/boinc_data/boinc-02/slots/2/ICMSHh7zg+000360
Moving to projects directory: /var/lib/boinc_data/boinc-02/slots/2/ICMUAh7zg+000360
  15:27:00 STEP  361 H=  90:15 +CPU= 15.591
  15:27:15 STEP  362 H=  90:30 +CPU= 15.548
  15:27:31 STEP  363 H=  90:45 +CPU= 15.503
  15:27:47 STEP  364 H=  91:00 +CPU= 15.831
  15:28:02 STEP  365 H=  91:15 +CPU= 15.631
  15:28:18 STEP  366 H=  91:30 +CPU= 15.650
  15:28:34 STEP  367 H=  91:45 +CPU= 15.554
  15:28:50 STEP  368 H=  92:00 +CPU= 16.343
  15:29:05 STEP  369 H=  92:15 +CPU= 15.467
  15:29:21 STEP  370 H=  92:30 +CPU= 15.273
  15:29:36 STEP  371 H=  92:45 +CPU= 15.474
  15:29:55 STEP  372 H=  93:00 +CPU= 19.257
Moving to projects directory: /var/lib/boinc_data/boinc-02/slots/2/ICMGGh7zg+000372
Moving to projects directory: /var/lib/boinc_data/boinc-02/slots/2/ICMSHh7zg+000372
Moving to projects directory: /var/lib/boinc_data/boinc-02/slots/2/ICMUAh7zg+000372
  15:30:11 STEP  373 H=  93:15 +CPU= 15.512
  15:30:27 STEP  374 H=  93:30 +CPU= 15.655
  15:30:42 STEP  375 H=  93:45 +CPU= 15.408
  15:30:59 STEP  376 H=  94:00 +CPU= 17.064
  15:31:15 STEP  377 H=  94:15 +CPU= 15.615
  15:31:30 STEP  378 H=  94:30 +CPU= 15.289
  15:31:46 STEP  379 H=  94:45 +CPU= 16.040
  15:32:02 STEP  380 H=  95:00 +CPU= 16.325
  15:32:18 STEP  381 H=  95:15 +CPU= 15.270
  15:32:33 STEP  382 H=  95:30 +CPU= 15.548
  15:32:50 STEP  383 H=  95:45 +CPU= 16.260
  15:33:12 STEP  384 H=  96:00 +CPU= 22.165
Moving to projects directory: /var/lib/boinc_data/boinc-02/slots/2/ICMGGh7zg+000384
Moving to projects directory: /var/lib/boinc_data/boinc-02/slots/2/ICMSHh7zg+000384
Moving to projects directory: /var/lib/boinc_data/boinc-02/slots/2/ICMUAh7zg+000384
Adding to the zip: /var/lib/boinc_data/boinc-02/projects/climateprediction.net/oifs_43r3_bl_12208366/ICMGGh7zg+000288
Adding to the zip: /var/lib/boinc_data/boinc-02/projects/climateprediction.net/oifs_43r3_bl_12208366/ICMSHh7zg+000288
Adding to the zip: /var/lib/boinc_data/boinc-02/projects/climateprediction.net/oifs_43r3_bl_12208366/ICMUAh7zg+000288
Adding to the zip: /var/lib/boinc_data/boinc-02/projects/climateprediction.net/oifs_43r3_bl_12208366/ICMGGh7zg+000300
Adding to the zip: /var/lib/boinc_data/boinc-02/projects/climateprediction.net/oifs_43r3_bl_12208366/ICMSHh7zg+000300
Adding to the zip: /var/lib/boinc_data/boinc-02/projects/climateprediction.net/oifs_43r3_bl_12208366/ICMUAh7zg+000300
Adding to the zip: /var/lib/boinc_data/boinc-02/projects/climateprediction.net/oifs_43r3_bl_12208366/ICMGGh7zg+000312
Adding to the zip: /var/lib/boinc_data/boinc-02/projects/climateprediction.net/oifs_43r3_bl_12208366/ICMSHh7zg+000312
Adding to the zip: /var/lib/boinc_data/boinc-02/projects/climateprediction.net/oifs_43r3_bl_12208366/ICMUAh7zg+000312
Adding to the zip: /var/lib/boinc_data/boinc-02/projects/climateprediction.net/oifs_43r3_bl_12208366/ICMGGh7zg+000324
Adding to the zip: /var/lib/boinc_data/boinc-02/projects/climateprediction.net/oifs_43r3_bl_12208366/ICMSHh7zg+000324
Adding to the zip: /var/lib/boinc_data/boinc-02/projects/climateprediction.net/oifs_43r3_bl_12208366/ICMUAh7zg+000324
Adding to the zip: /var/lib/boinc_data/boinc-02/projects/climateprediction.net/oifs_43r3_bl_12208366/ICMGGh7zg+000336
Adding to the zip: /var/lib/boinc_data/boinc-02/projects/climateprediction.net/oifs_43r3_bl_12208366/ICMSHh7zg+000336
Adding to the zip: /var/lib/boinc_data/boinc-02/projects/climateprediction.net/oifs_43r3_bl_12208366/ICMUAh7zg+000336
Adding to the zip: /var/lib/boinc_data/boinc-02/projects/climateprediction.net/oifs_43r3_bl_12208366/ICMGGh7zg+000348
Adding to the zip: /var/lib/boinc_data/boinc-02/projects/climateprediction.net/oifs_43r3_bl_12208366/ICMSHh7zg+000348
Adding to the zip: /var/lib/boinc_data/boinc-02/projects/climateprediction.net/oifs_43r3_bl_12208366/ICMUAh7zg+000348
Adding to the zip: /var/lib/boinc_data/boinc-02/projects/climateprediction.net/oifs_43r3_bl_12208366/ICMGGh7zg+000360
Adding to the zip: /var/lib/boinc_data/boinc-02/projects/climateprediction.net/oifs_43r3_bl_12208366/ICMSHh7zg+000360
Adding to the zip: /var/lib/boinc_data/boinc-02/projects/climateprediction.net/oifs_43r3_bl_12208366/ICMUAh7zg+000360
Adding to the zip: /var/lib/boinc_data/boinc-02/projects/climateprediction.net/oifs_43r3_bl_12208366/ICMGGh7zg+000372
Adding to the zip: /var/lib/boinc_data/boinc-02/projects/climateprediction.net/oifs_43r3_bl_12208366/ICMSHh7zg+000372
Adding to the zip: /var/lib/boinc_data/boinc-02/projects/climateprediction.net/oifs_43r3_bl_12208366/ICMUAh7zg+000372
Zipping up the intermediate file: /var/lib/boinc_data/boinc-02/projects/climateprediction.net/oifs_43r3_bl_a17v_2016092300_15_991_12208366_1_r270612951_3.zip
Uploading the intermediate file: upload_file_3.zip
  15:33:27 STEP  385 H=  96:15 +CPU= 15.330
  15:33:42 STEP  386 H=  96:30 +CPU= 15.182
  15:33:57 STEP  387 H=  96:45 +CPU= 15.163
  15:34:13 STEP  388 H=  97:00 +CPU= 15.632
  15:34:28 STEP  389 H=  97:15 +CPU= 15.302
  15:34:44 STEP  390 H=  97:30 +CPU= 15.264
  15:34:59 STEP  391 H=  97:45 +CPU= 15.329
  15:35:15 STEP  392 H=  98:00 +CPU= 15.752
  15:35:30 STEP  393 H=  98:15 +CPU= 15.261
[EC_DRHOOK:atlas7l1:1:1:2214789:2214789] [20230218:153540:1676730940:6426.097] [signal_drhook@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1538] Received signal#8 (SIGFPE) :: 4362MB (heap), 5076MB (maxrss), 0MB (maxstack), 0 (paging), nsigs = 1
[EC_DRHOOK:atlas7l1:1:1:2214789:2214789] [20230218:153540:1676730940:6426.097] [signal_drhook@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1542] Also activating Harakiri-alarm (SIGALRM=14) to expire after 500s elapsed to prevent hangs, nsigs = 1
[EC_DRHOOK:atlas7l1:1:1:2214789:2214789] [20230218:153540:1676730940:6426.097] [signal_drhook@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1544] Harakiri signal handler 'signal_harakiri' for signal#14 (SIGALRM) installed at 0x81f0c0 (old at (nil))
[EC_DRHOOK:atlas7l1:1:1:2214789:2214789] [20230218:153540:1676730940:6426.097] [signal_drhook@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1617] Signal#8 was caused by floating-point overflow [memaddr=0x1cc4a8f], nsigs = 1
[EC_DRHOOK:atlas7l1:1:1:2214789:2214789] [20230218:153540:1676730940:6426.097] [signal_drhook@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1686] Starting DrHook backtrace for signal#8, nsigs = 1
[EC_DRHOOK:atlas7l1:1:1:2214789:2214789] [20230218:153540:1676730940:6426.097] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3843] 4362 MB (maxheap), 5076 MB (maxrss), 0 MB (maxstack)
[EC_DRHOOK:atlas7l1:1:1:2214789:2214789] [20230218:153540:1676730940:6426.097] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :  MASTER 
[EC_DRHOOK:atlas7l1:1:1:2214789:2214789] [20230218:153540:1676730940:6426.097] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :   CNT0 
[EC_DRHOOK:atlas7l1:1:1:2214789:2214789] [20230218:153540:1676730940:6426.097] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :    CNT1 
[EC_DRHOOK:atlas7l1:1:1:2214789:2214789] [20230218:153540:1676730940:6426.097] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :     CNT2 
[EC_DRHOOK:atlas7l1:1:1:2214789:2214789] [20230218:153540:1676730940:6426.097] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :      CNT3 
[EC_DRHOOK:atlas7l1:1:1:2214789:2214789] [20230218:153540:1676730940:6426.097] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :       CNT4 
[EC_DRHOOK:atlas7l1:1:1:2214789:2214789] [20230218:153540:1676730940:6426.097] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :        STEPO 
[EC_DRHOOK:atlas7l1:1:1:2214789:2214789] [20230218:153540:1676730940:6426.097] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :         SCAN2M 
[EC_DRHOOK:atlas7l1:1:1:2214789:2214789] [20230218:153540:1676730940:6426.097] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :          GP_MODEL_HEAP 
[EC_DRHOOK:atlas7l1:1:1:2214789:2214789] [20230218:153540:1676730940:6426.097] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :           GP_MODEL 
[EC_DRHOOK:atlas7l1:1:1:2214789:2214789] [20230218:153540:1676730940:6426.097] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :            EC_PHYS_DRV 
[EC_DRHOOK:atlas7l1:1:1:2214789:2214789] [20230218:153540:1676730940:6426.097] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :             >OMP-PHYSICS              (1001) 
[EC_DRHOOK:atlas7l1:1:1:2214789:2214789] [20230218:153540:1676730940:6426.097] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :              EC_PHYS 
[EC_DRHOOK:atlas7l1:1:1:2214789:2214789] [20230218:153540:1676730940:6426.097] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :               CALLPAR 
[EC_DRHOOK:atlas7l1:1:1:2214789:2214789] [20230218:153540:1676730940:6426.097] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :                TURBULENCE_LAYER 
[EC_DRHOOK:atlas7l1:1:1:2214789:2214789] [20230218:153540:1676730940:6426.097] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :                 VDFOUTER 
[EC_DRHOOK:atlas7l1:1:1:2214789:2214789] [20230218:153540:1676730940:6426.097] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :                  VDFMAIN 
[EC_DRHOOK:atlas7l1:1:1:2214789:2214789] [20230218:153541:1676730941:6427.097] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:240] Backtrace(s) for program '/var/lib/boinc_data/boinc-02/slots/2/oifs_43r3_model.exe' : sigcontextptr=0x7ffe76ad2090
[EC_DRHOOK:atlas7l1:1:1:2214789:2214789] [20230218:153541:1676730941:6427.097] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:277] Backtrace (size = 24) with plain hex-dump
[EC_DRHOOK:atlas7l1:1:1:2214789:2214789] [20230218:153541:1676730941:6427.097] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:340] [0]: [0x838803] : 0x838803
[EC_DRHOOK:atlas7l1:1:1:2214789:2214789] [20230218:153541:1676730941:6427.097] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:340] [1]: [0x838e09] : 0x838e09
[EC_DRHOOK:atlas7l1:1:1:2214789:2214789] [20230218:153541:1676730941:6427.097] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:340] [2]: [0x1d8bb20] : 0x1d8bb20
[EC_DRHOOK:atlas7l1:1:1:2214789:2214789] [20230218:153541:1676730941:6427.097] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:340] [3]: [0x1cc4a8f] : 0x1cc4a8f
[EC_DRHOOK:atlas7l1:1:1:2214789:2214789] [20230218:153541:1676730941:6427.097] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:340] [4]: [0x12d283b] : 0x12d283b
[EC_DRHOOK:atlas7l1:1:1:2214789:2214789] [20230218:153541:1676730941:6427.097] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:340] [5]: [0x12ba8d9] : 0x12ba8d9
[EC_DRHOOK:atlas7l1:1:1:2214789:2214789] [20230218:153541:1676730941:6427.097] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:340] [6]: [0x12b0762] : 0x12b0762
[EC_DRHOOK:atlas7l1:1:1:2214789:2214789] [20230218:153541:1676730941:6427.097] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:340] [7]: [0x1163fc3] : 0x1163fc3
[EC_DRHOOK:atlas7l1:1:1:2214789:2214789] [20230218:153541:1676730941:6427.097] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:340] [8]: [0x10b682f] : 0x10b682f
[EC_DRHOOK:atlas7l1:1:1:2214789:2214789] [20230218:153541:1676730941:6427.097] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:340] [9]: [0x10a915c] : 0x10a915c
[EC_DRHOOK:atlas7l1:1:1:2214789:2214789] [20230218:153541:1676730941:6427.097] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:340] [10]: [0xe680c3] : 0xe680c3
[EC_DRHOOK:atlas7l1:1:1:2214789:2214789] [20230218:153541:1676730941:6427.097] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:340] [11]: [0x15c1e62] : 0x15c1e62
[EC_DRHOOK:atlas7l1:1:1:2214789:2214789] [20230218:153541:1676730941:6427.097] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:340] [12]: [0x15ac40e] : 0x15ac40e
[EC_DRHOOK:atlas7l1:1:1:2214789:2214789] [20230218:153541:1676730941:6427.097] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:340] [13]: [0x6620b4] : 0x6620b4
[EC_DRHOOK:atlas7l1:1:1:2214789:2214789] [20230218:153541:1676730941:6427.097] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:340] [14]: [0x4a0e7c] : 0x4a0e7c
[EC_DRHOOK:atlas7l1:1:1:2214789:2214789] [20230218:153541:1676730941:6427.097] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:340] [15]: [0x49dc29] : 0x49dc29
[EC_DRHOOK:atlas7l1:1:1:2214789:2214789] [20230218:153541:1676730941:6427.097] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:340] [16]: [0x49ce72] : 0x49ce72
[EC_DRHOOK:atlas7l1:1:1:2214789:2214789] [20230218:153541:1676730941:6427.097] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:340] [17]: [0x49cd68] : 0x49cd68
[EC_DRHOOK:atlas7l1:1:1:2214789:2214789] [20230218:153541:1676730941:6427.097] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:340] [18]: [0x4078ad] : 0x4078ad
[EC_DRHOOK:atlas7l1:1:1:2214789:2214789] [20230218:153541:1676730941:6427.097] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:340] [19]: [0x40276f] : 0x40276f
[EC_DRHOOK:atlas7l1:1:1:2214789:2214789] [20230218:153541:1676730941:6427.097] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:340] [20]: [0x40270d] : 0x40270d
[EC_DRHOOK:atlas7l1:1:1:2214789:2214789] [20230218:153541:1676730941:6427.097] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:340] [21]: [0x1d8287a] : 0x1d8287a
[EC_DRHOOK:atlas7l1:1:1:2214789:2214789] [20230218:153541:1676730941:6427.097] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:340] [22]: [0x1d840d7] : 0x1d840d7
[EC_DRHOOK:atlas7l1:1:1:2214789:2214789] [20230218:153541:1676730941:6427.097] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:340] [23]: [0x4025e5] : 0x4025e5
[EC_DRHOOK:atlas7l1:1:1:2214789:2214789] [20230218:153541:1676730941:6427.097] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:370] End of backtrace(s)
 Process            1  thread            1  calling linux_trbk from intel_trbk()
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:240] Backtrace(s) for program '/var/lib/boinc_data/boinc-02/slots/2/oifs_43r3_model.exe' : sigcontextptr=0x7ffe76ad1fa0
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:277] Backtrace (size = 25) with plain hex-dump
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:340] [0]: [0x838803] : 0x838803
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:340] [1]: [0x838e09] : 0x838e09
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:340] [2]: [0x820d70] : 0x820d70
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:340] [3]: [0x1d8bb20] : 0x1d8bb20
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:340] [4]: [0x1cc4a8f] : 0x1cc4a8f
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:340] [5]: [0x12d283b] : 0x12d283b
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:340] [6]: [0x12ba8d9] : 0x12ba8d9
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:340] [7]: [0x12b0762] : 0x12b0762
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:340] [8]: [0x1163fc3] : 0x1163fc3
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:340] [9]: [0x10b682f] : 0x10b682f
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:340] [10]: [0x10a915c] : 0x10a915c
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:340] [11]: [0xe680c3] : 0xe680c3
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:340] [12]: [0x15c1e62] : 0x15c1e62
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:340] [13]: [0x15ac40e] : 0x15ac40e
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:340] [14]: [0x6620b4] : 0x6620b4
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:340] [15]: [0x4a0e7c] : 0x4a0e7c
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:340] [16]: [0x49dc29] : 0x49dc29
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:340] [17]: [0x49ce72] : 0x49ce72
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:340] [18]: [0x49cd68] : 0x49cd68
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:340] [19]: [0x4078ad] : 0x4078ad
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:340] [20]: [0x40276f] : 0x40276f
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:340] [21]: [0x40270d] : 0x40270d
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:340] [22]: [0x1d8287a] : 0x1d8287a
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:340] [23]: [0x1d840d7] : 0x1d840d7
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:340] [24]: [0x4025e5] : 0x4025e5
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:370] End of backtrace(s)
[EC_DRHOOK:atlas7l1:1:1:2214789:2214789] [20230218:153541:1676730941:6427.110] [signal_drhook@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1734] DrHook backtrace done for signal#8, nsigs = 1
[EC_DRHOOK:atlas7l1:1:1:2214789:2214789] [20230218:153541:1676730941:6427.110] [signal_drhook@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1785] Calling previous signal handler at 0x401140 for signal#8, nsigs = 1
forrtl: error (72): floating overflow
Image              PC                Routine            Line        Source             
oifs_43r3_model.e  000000000081FCF1  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001D8BB20  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001CC4A8F  Unknown               Unknown  Unknown
oifs_43r3_model.e  00000000012D283B  vdfmain_                  874  vdfmain.F90
oifs_43r3_model.e  00000000012BA8D9  vdfouter_                 739  vdfouter.F90
oifs_43r3_model.e  00000000012B0762  turbulence_layer_         158  turbulence_layer.F90
oifs_43r3_model.e  0000000001163FC3  callpar_                  869  callpar.F90
oifs_43r3_model.e  00000000010B682F  ec_phys_                  670  ec_phys.F90
oifs_43r3_model.e  00000000010A915C  ec_phys_drv_              599  ec_phys_drv.F90
oifs_43r3_model.e  0000000000E680C3  gp_model_                 613  gp_model.F90
oifs_43r3_model.e  00000000015C1E62  gp_model_heap_             74  gp_model_heap.F90
oifs_43r3_model.e  00000000015AC40E  scan2m_                   535  scan2m.F90
oifs_43r3_model.e  00000000006620B4  stepo_                    327  stepo.F90
oifs_43r3_model.e  00000000004A0E7C  cnt4_                    1133  cnt4.F90
oifs_43r3_model.e  000000000049DC29  cnt3_                     267  cnt3.F90
oifs_43r3_model.e  000000000049CE72  cnt2_                      88  cnt2.F90
oifs_43r3_model.e  000000000049CD68  cnt1_                      92  cnt1.F90
oifs_43r3_model.e  00000000004078AD  cnt0_                     146  cnt0.F90
oifs_43r3_model.e  000000000040276F  MAIN__                     96  master.F90
oifs_43r3_model.e  000000000040270D  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001D8287A  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001D840D7  Unknown               Unknown  Unknown
oifs_43r3_model.e  00000000004025E5  Unknown               Unknown  Unknown
..The child process has been killed with signal: 6
>>> Printing last 70 lines from file: NODE.001_01

 GPNORM PFRC                 AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.174705108590019E+00 0.000000000000000E+00 0.100000000000000E+01
 NSTEP =   384 STEPO   F00000000
 CREATE RESTART FILE ./srf00040000.0001                                                                                                                                                                                                                                              
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ./srf00040000.0001 MODE=w
 IO-STREAM CLOSED - ./srf00040000.0001
 MPI-TASK:   1 - ************ BYTES IN     34 RECORDS TRANSFERRED IN    1.8319 SECONDS******** Mbytes/s, TOTAL TIME=    2.3590(77.7%)
 CREATE RESTART CONTROL FILE ./rcf                                                                                                                   
 WRRESF: DELETED PREVIOUS RESTART FILE OK: ./srf00031200.0001
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
 NSTEP =   384 SCAN2M_HPOS  P
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ICMGGh7zg+000384 MODE=w
 IO-STREAM CLOSED - ICMGGh7zg+000384
 MPI-TASK:   1 -    306737930 BYTES IN      6 RECORDS TRANSFERRED IN    0.0003 SECONDS******** Mbytes/s, TOTAL TIME=    0.0071( 4.2%)
 NSTEP =   384 STEPO   0AA000000
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ./ICMSHh7zg+000384 MODE=a
 IO-STREAM CLOSED - ./ICMSHh7zg+000384
 MPI-TASK:   1 -    707765250 BYTES IN    202 RECORDS TRANSFERRED IN    0.0061 SECONDS116027.1 Mbytes/s, TOTAL TIME=    0.1561( 3.9%)
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ./ICMUAh7zg+000384 MODE=a
 IO-STREAM CLOSED - ./ICMUAh7zg+000384
 MPI-TASK:   1 -   4907806871 BYTES IN    125 RECORDS TRANSFERRED IN    0.0038 SECONDS******** Mbytes/s, TOTAL TIME=    0.1310( 2.9%)
 STEPO_FPOS WAS : B00000F00E
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ./ICMUAh7zg+000384 MODE=a
 IO-STREAM CLOSED - ./ICMUAh7zg+000384
 MPI-TASK:   1 -    298164297 BYTES IN      2 RECORDS TRANSFERRED IN    0.0001 SECONDS******** Mbytes/s, TOTAL TIME=    0.0027( 3.7%)
 STEPO_FPOS WAS : V00000000Y
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ./ICMUAh7zg+000384 MODE=a
 IO-STREAM CLOSED - ./ICMUAh7zg+000384
 MPI-TASK:   1 -    306737930 BYTES IN      4 RECORDS TRANSFERRED IN    0.0002 SECONDS******** Mbytes/s, TOTAL TIME=    0.0042( 4.8%)
 STEPO_FPOS WAS : T00000000M
 NSTEP =   384 STEPO   0AAA00AAA
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
 NSTEP =   384 SCAN2M_HPOS  P
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ICMGGh7zg+000384 MODE=a
 IO-STREAM CLOSED - ICMGGh7zg+000384
 MPI-TASK:   1 -    306737921 BYTES IN      7 RECORDS TRANSFERRED IN    0.0004 SECONDS766844.8 Mbytes/s, TOTAL TIME=    0.0070( 5.7%)
 15:33:12 STEP  384 H=  96:00 +CPU= 22.165
ISTEP=     385 RSOLINC= 1361.4482 RI0= 1365.0000 ZI0= 1361.4482 RIP0= 1361.4482
ISTEP=     385 ZI0= 1361.4482 ZSEASON= 1.000000000 REA=149597870000.00 RDEASO=149597870000.00
 NSTEP =   385 STEPO   0AAA00AAA
 15:33:27 STEP  385 H=  96:15 +CPU= 15.330
 NSTEP =   386 STEPO   0AAA00AAA
 15:33:42 STEP  386 H=  96:30 +CPU= 15.182
 NSTEP =   387 STEPO   0AAA00AAA
 15:33:57 STEP  387 H=  96:45 +CPU= 15.163
 NSTEP =   388 STEPO   0AAA00AAA
 15:34:13 STEP  388 H=  97:00 +CPU= 15.632
 NSTEP =   389 STEPO   0AAA00AAA
 15:34:28 STEP  389 H=  97:15 +CPU= 15.302
ISTEP=     390 RSOLINC= 1361.4482 RI0= 1365.0000 ZI0= 1361.4482 RIP0= 1361.4482
ISTEP=     390 ZI0= 1361.4482 ZSEASON= 1.000000000 REA=149597870000.00 RDEASO=149597870000.00
 NSTEP =   390 STEPO   0AAA00AAA
 15:34:44 STEP  390 H=  97:30 +CPU= 15.264
 NSTEP =   391 STEPO   0AAA00AAA
 15:34:59 STEP  391 H=  97:45 +CPU= 15.329
 NSTEP =   392 STEPO   0AAA00AAA
 15:35:15 STEP  392 H=  98:00 +CPU= 15.752
 NSTEP =   393 STEPO   0AAA00AAA
 15:35:30 STEP  393 H=  98:15 +CPU= 15.261
 NSTEP =   394 STEPO   0AAA00AAA
------------------------------------------------
CNT0 not found; string returned was: 'STEPO'
>>> Printing last 8 lines from file: ifs.stat

 15:33:57 0AAA00AAA STEPO      388    15.163   15.163   15.164    105:16    105:23 0.33214618415141E-04       2GB       0MB
 15:34:13 0AAA00AAA STEPO      389    15.632   15.632   15.633    105:32    105:38 0.33214618415141E-04       2GB       0MB
 15:34:28 0AAA00AAA STEPO      390    15.302   15.302   15.302    105:47    105:54 0.33214618415141E-04       2GB       0MB
 15:34:44 0AAA00AAA STEPO      391    15.264   15.264   15.264    106:02    106:09 0.33214618415141E-04       2GB       0MB
 15:34:59 0AAA00AAA STEPO      392    15.329   15.329   15.389    106:18    106:24 0.33214618415141E-04       2GB       0MB
 15:35:15 0AAA00AAA STEPO      393    15.752   15.752   15.752    106:34    106:40 0.33214618415141E-04       2GB       0MB
 15:35:30 0AAA00AAA STEPO      394    15.261   15.261   15.260    106:49    106:55 0.33214618415141E-04       2GB       0MB
------------------------------------------------
>>> Printing last 11 lines from file: rcf

 CSTEP   = "     384",
 CTIME   = "00040000      ",
 NSTEPLPP        =         201,         202,          49,      228226,      228227, 2*0,         372, 2*0,
 IPRGPNSRES      =           1,
 IPRGPEWRES      =           1,
 IPRTRWRES       =           1,
 IPRTRVRES       =           1,
 GMASS0  =  0.000000000000000E+000,
 GMASSI  =  0.000000000000000E+000
 /
------------------------------------------------
>>> Printing last 8 lines from file: /var/lib/boinc_data/boinc-02/slots/2/progress_file_12208366.xml

<running_values>
  <last_cpu_time>6413.550000</last_cpu_time>
  <upload_file_number>4</upload_file_number>
  <last_iter>394</last_iter>
  <last_upload>345600</last_upload>
  <model_completed>0</model_completed>
</running_values>
------------------------------------------------
..Failed, model did not complete successfully

</stderr_txt>
]]>
No trickles!


©2024 climateprediction.net