climateprediction.net home page
Task 22439259

Task 22439259

Name oifs_43r3_bl_a4gv_2016092300_20_1017_12287618_0
Workunit 12287618
Created 7 Jun 2024, 19:12:53 UTC
Sent 8 Jun 2024, 2:10:00 UTC
Report deadline 7 Aug 2024, 2:10:00 UTC
Received 8 Jun 2024, 23:08:21 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 0 (0x00000000)
Computer ID 1545820
Run time 20 hours 43 min 17 sec
CPU time 20 hours 28 min 20 sec
Validate state Invalid
Credit 1,976.32
Device peak FLOPS 6.25 GFLOPS
Application version OpenIFS 43r3 Baroclinic Lifecycle v1.13
x86_64-pc-linux-gnu
Peak working set size 5,023.30 MB
Peak swap size 5,980.80 MB
Peak disk usage 1,376.19 MB
Stderr
<core_client_version>7.22.1</core_client_version>
<![CDATA[
<stderr_txt>
2 H= 193:00 +CPU= 64.779
  10:51:31 STEP  773 H= 193:15 +CPU= 69.279
  10:52:34 STEP  774 H= 193:30 +CPU= 63.097
  10:53:37 STEP  775 H= 193:45 +CPU= 62.351
  10:54:41 STEP  776 H= 194:00 +CPU= 63.561
  10:55:47 STEP  777 H= 194:15 +CPU= 66.017
  10:56:59 STEP  778 H= 194:30 +CPU= 69.313
  10:58:07 STEP  779 H= 194:45 +CPU= 67.447
  10:59:26 STEP  780 H= 195:00 +CPU= 78.827
Moving to projects directory: /home/test/slots/18/ICMGGh7zg+000780
Moving to projects directory: /home/test/slots/18/ICMSHh7zg+000780
Moving to projects directory: /home/test/slots/18/ICMUAh7zg+000780
  11:00:29 STEP  781 H= 195:15 +CPU= 62.569
  11:01:42 STEP  782 H= 195:30 +CPU= 72.071
  11:02:46 STEP  783 H= 195:45 +CPU= 63.560
  11:03:50 STEP  784 H= 196:00 +CPU= 63.764
  11:05:00 STEP  785 H= 196:15 +CPU= 69.073
  11:06:09 STEP  786 H= 196:30 +CPU= 69.157
  11:07:17 STEP  787 H= 196:45 +CPU= 66.988
  11:08:34 STEP  788 H= 197:00 +CPU= 76.052
  11:09:44 STEP  789 H= 197:15 +CPU= 69.507
  11:10:54 STEP  790 H= 197:30 +CPU= 69.647
  11:12:02 STEP  791 H= 197:45 +CPU= 67.156
  11:13:22 STEP  792 H= 198:00 +CPU= 79.700
Moving to projects directory: /home/test/slots/18/ICMGGh7zg+000792
Moving to projects directory: /home/test/slots/18/ICMSHh7zg+000792
Moving to projects directory: /home/test/slots/18/ICMUAh7zg+000792
  11:14:24 STEP  793 H= 198:15 +CPU= 62.293
  11:15:32 STEP  794 H= 198:30 +CPU= 67.689
  11:16:40 STEP  795 H= 198:45 +CPU= 67.286
  11:17:48 STEP  796 H= 199:00 +CPU= 67.490
  11:18:53 STEP  797 H= 199:15 +CPU= 64.363
  11:19:58 STEP  798 H= 199:30 +CPU= 65.298
  11:21:05 STEP  799 H= 199:45 +CPU= 66.750
  11:22:13 STEP  800 H= 200:00 +CPU= 67.000
  11:23:26 STEP  801 H= 200:15 +CPU= 72.871
  11:24:42 STEP  802 H= 200:30 +CPU= 74.329
  11:25:43 STEP  803 H= 200:45 +CPU= 61.473
  11:26:57 STEP  804 H= 201:00 +CPU= 73.615
Moving to projects directory: /home/test/slots/18/ICMGGh7zg+000804
Moving to projects directory: /home/test/slots/18/ICMSHh7zg+000804
Moving to projects directory: /home/test/slots/18/ICMUAh7zg+000804
  11:28:06 STEP  805 H= 201:15 +CPU= 67.993
  11:29:08 STEP  806 H= 201:30 +CPU= 61.154
  11:30:10 STEP  807 H= 201:45 +CPU= 62.233
  11:31:24 STEP  808 H= 202:00 +CPU= 72.955
  11:32:36 STEP  809 H= 202:15 +CPU= 71.523
  11:33:37 STEP  810 H= 202:30 +CPU= 61.185
  11:34:45 STEP  811 H= 202:45 +CPU= 67.562
  11:35:53 STEP  812 H= 203:00 +CPU= 66.868
  11:37:01 STEP  813 H= 203:15 +CPU= 68.015
  11:38:14 STEP  814 H= 203:30 +CPU= 72.467
  11:39:20 STEP  815 H= 203:45 +CPU= 64.965
  11:40:50 STEP  816 H= 204:00 +CPU= 89.302
Moving to projects directory: /home/test/slots/18/ICMGGh7zg+000816
Moving to projects directory: /home/test/slots/18/ICMSHh7zg+000816
Moving to projects directory: /home/test/slots/18/ICMUAh7zg+000816
  11:42:00 STEP  817 H= 204:15 +CPU= 69.571
  11:43:08 STEP  818 H= 204:30 +CPU= 67.343
  11:44:11 STEP  819 H= 204:45 +CPU= 62.299
  11:45:22 STEP  820 H= 205:00 +CPU= 69.505
  11:46:27 STEP  821 H= 205:15 +CPU= 65.489
  11:47:32 STEP  822 H= 205:30 +CPU= 63.897
  11:48:44 STEP  823 H= 205:45 +CPU= 71.288
  11:49:48 STEP  824 H= 206:00 +CPU= 64.100
  11:50:53 STEP  825 H= 206:15 +CPU= 63.663
  11:52:02 STEP  826 H= 206:30 +CPU= 68.769
  11:53:11 STEP  827 H= 206:45 +CPU= 68.376
  11:54:38 STEP  828 H= 207:00 +CPU= 85.860
Moving to projects directory: /home/test/slots/18/ICMGGh7zg+000828
Moving to projects directory: /home/test/slots/18/ICMSHh7zg+000828
Moving to projects directory: /home/test/slots/18/ICMUAh7zg+000828
  11:55:45 STEP  829 H= 207:15 +CPU= 66.013
  11:56:53 STEP  830 H= 207:30 +CPU= 68.046
  11:57:53 STEP  831 H= 207:45 +CPU= 59.336
  11:59:02 STEP  832 H= 208:00 +CPU= 68.998
  12:00:05 STEP  833 H= 208:15 +CPU= 62.292
  12:01:13 STEP  834 H= 208:30 +CPU= 67.593
  12:02:24 STEP  835 H= 208:45 +CPU= 70.374
  12:03:32 STEP  836 H= 209:00 +CPU= 67.657
  12:04:37 STEP  837 H= 209:15 +CPU= 63.910
  12:05:49 STEP  838 H= 209:30 +CPU= 71.331
  12:06:52 STEP  839 H= 209:45 +CPU= 62.486
  12:08:13 STEP  840 H= 210:00 +CPU= 80.644
Moving to projects directory: /home/test/slots/18/ICMGGh7zg+000840
Moving to projects directory: /home/test/slots/18/ICMSHh7zg+000840
Moving to projects directory: /home/test/slots/18/ICMUAh7zg+000840
  12:09:25 STEP  841 H= 210:15 +CPU= 71.515
  12:10:27 STEP  842 H= 210:30 +CPU= 61.356
  12:11:32 STEP  843 H= 210:45 +CPU= 64.533
  12:12:39 STEP  844 H= 211:00 +CPU= 66.754
  12:13:43 STEP  845 H= 211:15 +CPU= 63.605
  12:14:55 STEP  846 H= 211:30 +CPU= 71.743
  12:16:02 STEP  847 H= 211:45 +CPU= 65.681
  12:16:57 STEP  848 H= 212:00 +CPU= 55.124
  12:18:13 STEP  849 H= 212:15 +CPU= 75.312
  12:19:26 STEP  850 H= 212:30 +CPU= 72.574
  12:20:39 STEP  851 H= 212:45 +CPU= 72.187
  12:22:02 STEP  852 H= 213:00 +CPU= 82.421
Moving to projects directory: /home/test/slots/18/ICMGGh7zg+000852
Moving to projects directory: /home/test/slots/18/ICMSHh7zg+000852
Moving to projects directory: /home/test/slots/18/ICMUAh7zg+000852
  12:23:08 STEP  853 H= 213:15 +CPU= 65.332
  12:24:17 STEP  854 H= 213:30 +CPU= 68.468
  12:25:29 STEP  855 H= 213:45 +CPU= 71.071
  12:26:52 STEP  856 H= 214:00 +CPU= 82.190
  12:28:03 STEP  857 H= 214:15 +CPU= 70.422
  12:29:14 STEP  858 H= 214:30 +CPU= 70.847
  12:30:29 STEP  859 H= 214:45 +CPU= 74.693
  12:31:31 STEP  860 H= 215:00 +CPU= 61.775
  12:32:36 STEP  861 H= 215:15 +CPU= 64.972
  12:33:45 STEP  862 H= 215:30 +CPU= 68.060
  12:34:56 STEP  863 H= 215:45 +CPU= 70.019
  12:36:15 STEP  864 H= 216:00 +CPU= 79.120
Moving to projects directory: /home/test/slots/18/ICMGGh7zg+000864
Moving to projects directory: /home/test/slots/18/ICMSHh7zg+000864
Moving to projects directory: /home/test/slots/18/ICMUAh7zg+000864
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMGGh7zg+000768
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMSHh7zg+000768
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMUAh7zg+000768
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMGGh7zg+000780
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMSHh7zg+000780
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMUAh7zg+000780
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMGGh7zg+000792
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMSHh7zg+000792
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMUAh7zg+000792
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMGGh7zg+000804
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMSHh7zg+000804
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMUAh7zg+000804
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMGGh7zg+000816
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMSHh7zg+000816
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMUAh7zg+000816
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMGGh7zg+000828
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMSHh7zg+000828
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMUAh7zg+000828
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMGGh7zg+000840
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMSHh7zg+000840
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMUAh7zg+000840
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMGGh7zg+000852
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMSHh7zg+000852
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMUAh7zg+000852
Zipping up the intermediate file: /home/test/projects/climateprediction.net/oifs_43r3_bl_a4gv_2016092300_20_1017_12287618_0_r2086098213_8.zip
Uploading the intermediate file: upload_file_8.zip
  12:36:48 STEP  865 H= 216:15 +CPU= 32.382
  12:37:26 STEP  866 H= 216:30 +CPU= 37.172
  12:38:06 STEP  867 H= 216:45 +CPU= 39.994
  12:38:46 STEP  868 H= 217:00 +CPU= 39.695
  12:39:23 STEP  869 H= 217:15 +CPU= 37.750
  12:39:58 STEP  870 H= 217:30 +CPU= 34.541
  12:40:30 STEP  871 H= 217:45 +CPU= 32.095
  12:41:11 STEP  872 H= 218:00 +CPU= 40.485
  12:41:49 STEP  873 H= 218:15 +CPU= 36.701
  12:42:25 STEP  874 H= 218:30 +CPU= 35.869
  12:43:01 STEP  875 H= 218:45 +CPU= 35.982
  12:43:45 STEP  876 H= 219:00 +CPU= 43.855
Moving to projects directory: /home/test/slots/18/ICMGGh7zg+000876
Moving to projects directory: /home/test/slots/18/ICMSHh7zg+000876
Moving to projects directory: /home/test/slots/18/ICMUAh7zg+000876
  12:44:21 STEP  877 H= 219:15 +CPU= 35.124
  12:44:57 STEP  878 H= 219:30 +CPU= 35.820
  12:45:32 STEP  879 H= 219:45 +CPU= 35.401
  12:46:12 STEP  880 H= 220:00 +CPU= 40.067
  12:46:52 STEP  881 H= 220:15 +CPU= 39.056
  12:47:32 STEP  882 H= 220:30 +CPU= 39.947
  12:48:11 STEP  883 H= 220:45 +CPU= 39.305
  12:48:49 STEP  884 H= 221:00 +CPU= 37.144
  12:49:26 STEP  885 H= 221:15 +CPU= 36.565
  12:50:00 STEP  886 H= 221:30 +CPU= 33.646
  12:50:36 STEP  887 H= 221:45 +CPU= 36.270
  12:51:24 STEP  888 H= 222:00 +CPU= 47.077
Moving to projects directory: /home/test/slots/18/ICMGGh7zg+000888
Moving to projects directory: /home/test/slots/18/ICMSHh7zg+000888
Moving to projects directory: /home/test/slots/18/ICMUAh7zg+000888
  12:52:00 STEP  889 H= 222:15 +CPU= 36.108
  12:52:40 STEP  890 H= 222:30 +CPU= 39.809
  12:53:17 STEP  891 H= 222:45 +CPU= 37.021
  12:53:54 STEP  892 H= 223:00 +CPU= 37.084
  12:54:31 STEP  893 H= 223:15 +CPU= 36.696
  12:55:08 STEP  894 H= 223:30 +CPU= 37.075
  12:55:46 STEP  895 H= 223:45 +CPU= 37.634
  12:56:25 STEP  896 H= 224:00 +CPU= 38.839
  12:57:03 STEP  897 H= 224:15 +CPU= 37.728
  12:57:39 STEP  898 H= 224:30 +CPU= 35.896
  12:58:16 STEP  899 H= 224:45 +CPU= 36.722
  12:59:06 STEP  900 H= 225:00 +CPU= 49.798
Moving to projects directory: /home/test/slots/18/ICMGGh7zg+000900
Moving to projects directory: /home/test/slots/18/ICMSHh7zg+000900
Moving to projects directory: /home/test/slots/18/ICMUAh7zg+000900
  12:59:46 STEP  901 H= 225:15 +CPU= 39.451
  13:00:24 STEP  902 H= 225:30 +CPU= 38.103
  13:01:00 STEP  903 H= 225:45 +CPU= 36.001
  13:01:39 STEP  904 H= 226:00 +CPU= 38.503
  13:02:14 STEP  905 H= 226:15 +CPU= 35.291
  13:02:50 STEP  906 H= 226:30 +CPU= 35.658
  13:03:23 STEP  907 H= 226:45 +CPU= 32.707
  13:03:59 STEP  908 H= 227:00 +CPU= 35.030
  13:04:34 STEP  909 H= 227:15 +CPU= 34.826
  13:05:11 STEP  910 H= 227:30 +CPU= 36.455
  13:05:49 STEP  911 H= 227:45 +CPU= 38.371
  13:06:37 STEP  912 H= 228:00 +CPU= 46.795
Moving to projects directory: /home/test/slots/18/ICMGGh7zg+000912
Moving to projects directory: /home/test/slots/18/ICMSHh7zg+000912
Moving to projects directory: /home/test/slots/18/ICMUAh7zg+000912
  13:07:13 STEP  913 H= 228:15 +CPU= 34.956
  13:07:48 STEP  914 H= 228:30 +CPU= 35.468
  13:08:25 STEP  915 H= 228:45 +CPU= 35.628
  13:09:02 STEP  916 H= 229:00 +CPU= 36.953
  13:09:40 STEP  917 H= 229:15 +CPU= 37.677
  13:10:16 STEP  918 H= 229:30 +CPU= 35.368
  13:10:53 STEP  919 H= 229:45 +CPU= 36.780
  13:11:26 STEP  920 H= 230:00 +CPU= 33.284
  13:12:04 STEP  921 H= 230:15 +CPU= 37.474
  13:12:38 STEP  922 H= 230:30 +CPU= 34.372
  13:13:13 STEP  923 H= 230:45 +CPU= 34.751
  13:13:54 STEP  924 H= 231:00 +CPU= 40.431
Moving to projects directory: /home/test/slots/18/ICMGGh7zg+000924
Moving to projects directory: /home/test/slots/18/ICMSHh7zg+000924
Moving to projects directory: /home/test/slots/18/ICMUAh7zg+000924
  13:14:30 STEP  925 H= 231:15 +CPU= 35.959
  13:15:04 STEP  926 H= 231:30 +CPU= 33.557
  13:15:39 STEP  927 H= 231:45 +CPU= 34.905
  13:16:15 STEP  928 H= 232:00 +CPU= 35.708
  13:16:48 STEP  929 H= 232:15 +CPU= 32.975
  13:17:23 STEP  930 H= 232:30 +CPU= 34.765
  13:18:01 STEP  931 H= 232:45 +CPU= 38.330
  13:18:39 STEP  932 H= 233:00 +CPU= 37.720
  13:19:10 STEP  933 H= 233:15 +CPU= 30.205
  13:19:45 STEP  934 H= 233:30 +CPU= 34.866
  13:20:22 STEP  935 H= 233:45 +CPU= 37.147
  13:21:03 STEP  936 H= 234:00 +CPU= 40.922
Moving to projects directory: /home/test/slots/18/ICMGGh7zg+000936
Moving to projects directory: /home/test/slots/18/ICMSHh7zg+000936
Moving to projects directory: /home/test/slots/18/ICMUAh7zg+000936
  13:21:41 STEP  937 H= 234:15 +CPU= 37.123
  13:22:21 STEP  938 H= 234:30 +CPU= 39.320
  13:22:56 STEP  939 H= 234:45 +CPU= 35.585
  13:23:29 STEP  940 H= 235:00 +CPU= 31.718
  13:24:05 STEP  941 H= 235:15 +CPU= 35.200
  13:24:39 STEP  942 H= 235:30 +CPU= 34.072
  13:25:14 STEP  943 H= 235:45 +CPU= 34.614
  13:25:48 STEP  944 H= 236:00 +CPU= 33.684
  13:26:22 STEP  945 H= 236:15 +CPU= 33.294
  13:26:57 STEP  946 H= 236:30 +CPU= 35.833
  13:27:33 STEP  947 H= 236:45 +CPU= 35.379
  13:28:18 STEP  948 H= 237:00 +CPU= 44.672
Moving to projects directory: /home/test/slots/18/ICMGGh7zg+000948
Moving to projects directory: /home/test/slots/18/ICMSHh7zg+000948
Moving to projects directory: /home/test/slots/18/ICMUAh7zg+000948
  13:28:50 STEP  949 H= 237:15 +CPU= 32.416
  13:29:29 STEP  950 H= 237:30 +CPU= 38.232
  13:30:05 STEP  951 H= 237:45 +CPU= 36.422
  13:30:41 STEP  952 H= 238:00 +CPU= 35.293
  13:31:16 STEP  953 H= 238:15 +CPU= 35.030
  13:31:52 STEP  954 H= 238:30 +CPU= 35.450
  13:32:30 STEP  955 H= 238:45 +CPU= 37.711
  13:33:07 STEP  956 H= 239:00 +CPU= 36.730
  13:33:40 STEP  957 H= 239:15 +CPU= 33.739
  13:34:18 STEP  958 H= 239:30 +CPU= 37.183
  13:34:53 STEP  959 H= 239:45 +CPU= 34.453
  13:35:42 STEP  960 H= 240:00 +CPU= 48.740
Moving to projects directory: /home/test/slots/18/ICMGGh7zg+000960
Moving to projects directory: /home/test/slots/18/ICMSHh7zg+000960
Moving to projects directory: /home/test/slots/18/ICMUAh7zg+000960
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMGGh7zg+000864
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMSHh7zg+000864
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMUAh7zg+000864
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMGGh7zg+000876
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMSHh7zg+000876
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMUAh7zg+000876
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMGGh7zg+000888
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMSHh7zg+000888
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMUAh7zg+000888
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMGGh7zg+000900
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMSHh7zg+000900
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMUAh7zg+000900
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMGGh7zg+000912
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMSHh7zg+000912
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMUAh7zg+000912
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMGGh7zg+000924
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMSHh7zg+000924
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMUAh7zg+000924
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMGGh7zg+000936
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMSHh7zg+000936
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMUAh7zg+000936
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMGGh7zg+000948
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMSHh7zg+000948
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMUAh7zg+000948
Zipping up the intermediate file: /home/test/projects/climateprediction.net/oifs_43r3_bl_a4gv_2016092300_20_1017_12287618_0_r2086098213_9.zip
Uploading the intermediate file: upload_file_9.zip
  13:36:20 STEP  961 H= 240:15 +CPU= 38.169
  13:36:57 STEP  962 H= 240:30 +CPU= 35.717
  13:37:34 STEP  963 H= 240:45 +CPU= 36.937
  13:38:12 STEP  964 H= 241:00 +CPU= 37.027
  13:38:47 STEP  965 H= 241:15 +CPU= 35.130
  13:39:24 STEP  966 H= 241:30 +CPU= 36.796
  13:40:00 STEP  967 H= 241:45 +CPU= 35.249
  13:40:34 STEP  968 H= 242:00 +CPU= 34.079
  13:41:10 STEP  969 H= 242:15 +CPU= 36.116
  13:41:47 STEP  970 H= 242:30 +CPU= 36.589
  13:42:25 STEP  971 H= 242:45 +CPU= 37.375
  13:43:11 STEP  972 H= 243:00 +CPU= 46.184
Moving to projects directory: /home/test/slots/18/ICMGGh7zg+000972
Moving to projects directory: /home/test/slots/18/ICMSHh7zg+000972
Moving to projects directory: /home/test/slots/18/ICMUAh7zg+000972
  13:43:49 STEP  973 H= 243:15 +CPU= 37.203
  13:44:25 STEP  974 H= 243:30 +CPU= 36.094
  13:45:01 STEP  975 H= 243:45 +CPU= 36.202
  13:45:40 STEP  976 H= 244:00 +CPU= 38.349
  13:46:19 STEP  977 H= 244:15 +CPU= 38.645
  13:46:55 STEP  978 H= 244:30 +CPU= 35.728
  13:47:29 STEP  979 H= 244:45 +CPU= 33.953
  13:48:04 STEP  980 H= 245:00 +CPU= 34.604
  13:48:38 STEP  981 H= 245:15 +CPU= 33.938
  13:49:18 STEP  982 H= 245:30 +CPU= 39.633
  13:49:58 STEP  983 H= 245:45 +CPU= 39.758
  13:50:44 STEP  984 H= 246:00 +CPU= 45.794
Moving to projects directory: /home/test/slots/18/ICMGGh7zg+000984
Moving to projects directory: /home/test/slots/18/ICMSHh7zg+000984
Moving to projects directory: /home/test/slots/18/ICMUAh7zg+000984
  13:51:23 STEP  985 H= 246:15 +CPU= 37.881
  13:51:55 STEP  986 H= 246:30 +CPU= 32.404
  13:52:33 STEP  987 H= 246:45 +CPU= 37.092
  13:53:09 STEP  988 H= 247:00 +CPU= 36.153
  13:53:48 STEP  989 H= 247:15 +CPU= 37.986
  13:54:28 STEP  990 H= 247:30 +CPU= 40.029
  13:55:09 STEP  991 H= 247:45 +CPU= 40.978
  13:55:51 STEP  992 H= 248:00 +CPU= 41.021
  13:56:29 STEP  993 H= 248:15 +CPU= 37.572
  13:57:04 STEP  994 H= 248:30 +CPU= 34.547
  13:57:42 STEP  995 H= 248:45 +CPU= 37.985
  13:58:28 STEP  996 H= 249:00 +CPU= 45.359
Moving to projects directory: /home/test/slots/18/ICMGGh7zg+000996
Moving to projects directory: /home/test/slots/18/ICMSHh7zg+000996
Moving to projects directory: /home/test/slots/18/ICMUAh7zg+000996
  13:59:03 STEP  997 H= 249:15 +CPU= 34.750
  13:59:35 STEP  998 H= 249:30 +CPU= 31.711
  14:00:12 STEP  999 H= 249:45 +CPU= 36.926
  14:00:49 STEP 1000 H= 250:00 +CPU= 36.441
  14:01:24 STEP 1001 H= 250:15 +CPU= 34.317
  14:02:02 STEP 1002 H= 250:30 +CPU= 37.968
  14:02:39 STEP 1003 H= 250:45 +CPU= 37.528
  14:03:17 STEP 1004 H= 251:00 +CPU= 36.891
  14:03:51 STEP 1005 H= 251:15 +CPU= 33.966
  14:04:24 STEP 1006 H= 251:30 +CPU= 32.655
  14:04:55 STEP 1007 H= 251:45 +CPU= 30.684
  14:05:37 STEP 1008 H= 252:00 +CPU= 42.564
Moving to projects directory: /home/test/slots/18/ICMGGh7zg+001008
Moving to projects directory: /home/test/slots/18/ICMSHh7zg+001008
Moving to projects directory: /home/test/slots/18/ICMUAh7zg+001008
  14:06:09 STEP 1009 H= 252:15 +CPU= 31.544
  14:06:38 STEP 1010 H= 252:30 +CPU= 29.633
  14:07:07 STEP 1011 H= 252:45 +CPU= 28.566
  14:07:34 STEP 1012 H= 253:00 +CPU= 27.164
  14:08:08 STEP 1013 H= 253:15 +CPU= 33.680
  14:08:41 STEP 1014 H= 253:30 +CPU= 32.913
  14:09:15 STEP 1015 H= 253:45 +CPU= 33.317
  14:09:51 STEP 1016 H= 254:00 +CPU= 35.978
  14:10:26 STEP 1017 H= 254:15 +CPU= 34.388
  14:11:03 STEP 1018 H= 254:30 +CPU= 36.438
  14:11:37 STEP 1019 H= 254:45 +CPU= 34.774
  14:12:20 STEP 1020 H= 255:00 +CPU= 41.997
Moving to projects directory: /home/test/slots/18/ICMGGh7zg+001020
Moving to projects directory: /home/test/slots/18/ICMSHh7zg+001020
Moving to projects directory: /home/test/slots/18/ICMUAh7zg+001020
  14:12:54 STEP 1021 H= 255:15 +CPU= 33.931
  14:13:30 STEP 1022 H= 255:30 +CPU= 36.166
  14:14:07 STEP 1023 H= 255:45 +CPU= 36.073
  14:14:42 STEP 1024 H= 256:00 +CPU= 35.485
  14:15:16 STEP 1025 H= 256:15 +CPU= 33.933
  14:15:54 STEP 1026 H= 256:30 +CPU= 37.316
  14:16:33 STEP 1027 H= 256:45 +CPU= 39.132
  14:17:08 STEP 1028 H= 257:00 +CPU= 34.794
  14:17:45 STEP 1029 H= 257:15 +CPU= 36.380
  14:18:19 STEP 1030 H= 257:30 +CPU= 33.968
  14:18:53 STEP 1031 H= 257:45 +CPU= 33.537
  14:19:35 STEP 1032 H= 258:00 +CPU= 41.505
Moving to projects directory: /home/test/slots/18/ICMGGh7zg+001032
Moving to projects directory: /home/test/slots/18/ICMSHh7zg+001032
Moving to projects directory: /home/test/slots/18/ICMUAh7zg+001032
  14:20:09 STEP 1033 H= 258:15 +CPU= 34.337
  14:20:43 STEP 1034 H= 258:30 +CPU= 34.322
  14:21:22 STEP 1035 H= 258:45 +CPU= 38.258
  14:21:55 STEP 1036 H= 259:00 +CPU= 32.752
  14:22:29 STEP 1037 H= 259:15 +CPU= 34.264
  14:23:03 STEP 1038 H= 259:30 +CPU= 33.647
  14:23:37 STEP 1039 H= 259:45 +CPU= 33.284
  14:24:13 STEP 1040 H= 260:00 +CPU= 36.036
  14:24:47 STEP 1041 H= 260:15 +CPU= 33.622
  14:25:24 STEP 1042 H= 260:30 +CPU= 36.309
  14:26:05 STEP 1043 H= 260:45 +CPU= 40.734
  14:26:50 STEP 1044 H= 261:00 +CPU= 45.340
Moving to projects directory: /home/test/slots/18/ICMGGh7zg+001044
Moving to projects directory: /home/test/slots/18/ICMSHh7zg+001044
Moving to projects directory: /home/test/slots/18/ICMUAh7zg+001044
  14:27:25 STEP 1045 H= 261:15 +CPU= 34.364
  14:28:01 STEP 1046 H= 261:30 +CPU= 36.515
  14:28:38 STEP 1047 H= 261:45 +CPU= 36.504
  14:29:16 STEP 1048 H= 262:00 +CPU= 37.653
  14:29:56 STEP 1049 H= 262:15 +CPU= 39.441
  14:30:31 STEP 1050 H= 262:30 +CPU= 35.354
  14:31:04 STEP 1051 H= 262:45 +CPU= 32.443
  14:31:40 STEP 1052 H= 263:00 +CPU= 34.534
  14:32:14 STEP 1053 H= 263:15 +CPU= 33.763
  14:32:48 STEP 1054 H= 263:30 +CPU= 34.029
  14:33:25 STEP 1055 H= 263:45 +CPU= 37.013
  14:34:14 STEP 1056 H= 264:00 +CPU= 48.814
Moving to projects directory: /home/test/slots/18/ICMGGh7zg+001056
Moving to projects directory: /home/test/slots/18/ICMSHh7zg+001056
Moving to projects directory: /home/test/slots/18/ICMUAh7zg+001056
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMGGh7zg+000960
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMSHh7zg+000960
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMUAh7zg+000960
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMGGh7zg+000972
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMSHh7zg+000972
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMUAh7zg+000972
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMGGh7zg+000984
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMSHh7zg+000984
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMUAh7zg+000984
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMGGh7zg+000996
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMSHh7zg+000996
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMUAh7zg+000996
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMGGh7zg+001008
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMSHh7zg+001008
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMUAh7zg+001008
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMGGh7zg+001020
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMSHh7zg+001020
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMUAh7zg+001020
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMGGh7zg+001032
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMSHh7zg+001032
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMUAh7zg+001032
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMGGh7zg+001044
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMSHh7zg+001044
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMUAh7zg+001044
Zipping up the intermediate file: /home/test/projects/climateprediction.net/oifs_43r3_bl_a4gv_2016092300_20_1017_12287618_0_r2086098213_10.zip
Uploading the intermediate file: upload_file_10.zip
  14:34:51 STEP 1057 H= 264:15 +CPU= 36.250
  14:35:27 STEP 1058 H= 264:30 +CPU= 34.978
  14:36:03 STEP 1059 H= 264:45 +CPU= 35.839
  14:36:40 STEP 1060 H= 265:00 +CPU= 36.903
  14:37:15 STEP 1061 H= 265:15 +CPU= 34.817
  14:37:51 STEP 1062 H= 265:30 +CPU= 36.151
  14:38:26 STEP 1063 H= 265:45 +CPU= 34.302
  14:39:01 STEP 1064 H= 266:00 +CPU= 34.740
  14:39:37 STEP 1065 H= 266:15 +CPU= 35.710
  14:40:11 STEP 1066 H= 266:30 +CPU= 33.859
  14:40:45 STEP 1067 H= 266:45 +CPU= 33.572
  14:41:31 STEP 1068 H= 267:00 +CPU= 45.338
Moving to projects directory: /home/test/slots/18/ICMGGh7zg+001068
Moving to projects directory: /home/test/slots/18/ICMSHh7zg+001068
Moving to projects directory: /home/test/slots/18/ICMUAh7zg+001068
  14:42:08 STEP 1069 H= 267:15 +CPU= 37.065
  14:42:44 STEP 1070 H= 267:30 +CPU= 35.491
  14:43:19 STEP 1071 H= 267:45 +CPU= 35.501
  14:43:58 STEP 1072 H= 268:00 +CPU= 38.602
  14:44:36 STEP 1073 H= 268:15 +CPU= 37.987
  14:45:11 STEP 1074 H= 268:30 +CPU= 35.076
  14:45:46 STEP 1075 H= 268:45 +CPU= 34.230
  14:46:26 STEP 1076 H= 269:00 +CPU= 40.109
  14:47:06 STEP 1077 H= 269:15 +CPU= 40.182
  14:47:36 STEP 1078 H= 269:30 +CPU= 30.353
  14:47:59 STEP 1079 H= 269:45 +CPU= 22.715
  14:48:30 STEP 1080 H= 270:00 +CPU= 31.246
Moving to projects directory: /home/test/slots/18/ICMGGh7zg+001080
Moving to projects directory: /home/test/slots/18/ICMSHh7zg+001080
Moving to projects directory: /home/test/slots/18/ICMUAh7zg+001080
  14:48:55 STEP 1081 H= 270:15 +CPU= 24.110
  14:49:19 STEP 1082 H= 270:30 +CPU= 24.238
  14:49:43 STEP 1083 H= 270:45 +CPU= 23.948
  14:50:09 STEP 1084 H= 271:00 +CPU= 26.456
  14:50:36 STEP 1085 H= 271:15 +CPU= 26.613
  14:51:02 STEP 1086 H= 271:30 +CPU= 25.906
  14:51:26 STEP 1087 H= 271:45 +CPU= 24.422
  14:51:52 STEP 1088 H= 272:00 +CPU= 25.126
  14:52:17 STEP 1089 H= 272:15 +CPU= 25.560
  14:52:44 STEP 1090 H= 272:30 +CPU= 26.719
  14:53:11 STEP 1091 H= 272:45 +CPU= 26.460
  14:53:43 STEP 1092 H= 273:00 +CPU= 31.586
Moving to projects directory: /home/test/slots/18/ICMGGh7zg+001092
Moving to projects directory: /home/test/slots/18/ICMSHh7zg+001092
Moving to projects directory: /home/test/slots/18/ICMUAh7zg+001092
  14:54:08 STEP 1093 H= 273:15 +CPU= 25.075
  14:54:34 STEP 1094 H= 273:30 +CPU= 25.972
  14:55:01 STEP 1095 H= 273:45 +CPU= 26.388
  14:55:28 STEP 1096 H= 274:00 +CPU= 27.062
  14:55:55 STEP 1097 H= 274:15 +CPU= 27.000
  14:56:21 STEP 1098 H= 274:30 +CPU= 26.039
  14:56:46 STEP 1099 H= 274:45 +CPU= 25.446
  14:57:12 STEP 1100 H= 275:00 +CPU= 25.682
  14:57:40 STEP 1101 H= 275:15 +CPU= 27.101
  14:58:06 STEP 1102 H= 275:30 +CPU= 26.680
  14:58:33 STEP 1103 H= 275:45 +CPU= 26.114
  14:59:09 STEP 1104 H= 276:00 +CPU= 35.416
Moving to projects directory: /home/test/slots/18/ICMGGh7zg+001104
Moving to projects directory: /home/test/slots/18/ICMSHh7zg+001104
Moving to projects directory: /home/test/slots/18/ICMUAh7zg+001104
  14:59:33 STEP 1105 H= 276:15 +CPU= 24.615
  14:59:57 STEP 1106 H= 276:30 +CPU= 23.828
  15:00:22 STEP 1107 H= 276:45 +CPU= 24.644
  15:00:48 STEP 1108 H= 277:00 +CPU= 26.196
  15:01:14 STEP 1109 H= 277:15 +CPU= 24.928
  15:01:38 STEP 1110 H= 277:30 +CPU= 24.219
  15:02:04 STEP 1111 H= 277:45 +CPU= 25.778
  15:02:30 STEP 1112 H= 278:00 +CPU= 25.728
  15:02:55 STEP 1113 H= 278:15 +CPU= 25.311
  15:03:22 STEP 1114 H= 278:30 +CPU= 25.586
  15:03:48 STEP 1115 H= 278:45 +CPU= 26.037
  15:04:20 STEP 1116 H= 279:00 +CPU= 31.556
Moving to projects directory: /home/test/slots/18/ICMGGh7zg+001116
Moving to projects directory: /home/test/slots/18/ICMSHh7zg+001116
Moving to projects directory: /home/test/slots/18/ICMUAh7zg+001116
  15:04:47 STEP 1117 H= 279:15 +CPU= 26.884
  15:05:14 STEP 1118 H= 279:30 +CPU= 27.057
  15:05:38 STEP 1119 H= 279:45 +CPU= 23.951
  15:06:04 STEP 1120 H= 280:00 +CPU= 25.250
  15:06:28 STEP 1121 H= 280:15 +CPU= 24.757
  15:06:56 STEP 1122 H= 280:30 +CPU= 27.358
  15:07:22 STEP 1123 H= 280:45 +CPU= 25.770
  15:07:49 STEP 1124 H= 281:00 +CPU= 27.080
  15:08:15 STEP 1125 H= 281:15 +CPU= 25.122
  15:08:40 STEP 1126 H= 281:30 +CPU= 25.158
  15:09:07 STEP 1127 H= 281:45 +CPU= 26.494
  15:09:38 STEP 1128 H= 282:00 +CPU= 31.339
Moving to projects directory: /home/test/slots/18/ICMGGh7zg+001128
Moving to projects directory: /home/test/slots/18/ICMSHh7zg+001128
Moving to projects directory: /home/test/slots/18/ICMUAh7zg+001128
  15:10:04 STEP 1129 H= 282:15 +CPU= 25.749
  15:10:29 STEP 1130 H= 282:30 +CPU= 24.777
  15:10:54 STEP 1131 H= 282:45 +CPU= 25.344
  15:11:21 STEP 1132 H= 283:00 +CPU= 26.287
  15:11:47 STEP 1133 H= 283:15 +CPU= 26.057
  15:12:12 STEP 1134 H= 283:30 +CPU= 25.168
  15:12:38 STEP 1135 H= 283:45 +CPU= 24.961
  15:13:04 STEP 1136 H= 284:00 +CPU= 25.979
  15:13:29 STEP 1137 H= 284:15 +CPU= 25.331
  15:13:55 STEP 1138 H= 284:30 +CPU= 25.184
  15:14:21 STEP 1139 H= 284:45 +CPU= 26.256
  15:14:53 STEP 1140 H= 285:00 +CPU= 32.032
Moving to projects directory: /home/test/slots/18/ICMGGh7zg+001140
Moving to projects directory: /home/test/slots/18/ICMSHh7zg+001140
Moving to projects directory: /home/test/slots/18/ICMUAh7zg+001140
  15:15:20 STEP 1141 H= 285:15 +CPU= 26.718
  15:15:46 STEP 1142 H= 285:30 +CPU= 26.013
  15:16:12 STEP 1143 H= 285:45 +CPU= 25.551
  15:16:37 STEP 1144 H= 286:00 +CPU= 25.179
  15:17:02 STEP 1145 H= 286:15 +CPU= 24.767
  15:17:25 STEP 1146 H= 286:30 +CPU= 23.275
  15:17:48 STEP 1147 H= 286:45 +CPU= 22.676
  15:18:12 STEP 1148 H= 287:00 +CPU= 24.108
  15:18:32 STEP 1149 H= 287:15 +CPU= 19.413
  15:18:54 STEP 1150 H= 287:30 +CPU= 22.004
  15:19:21 STEP 1151 H= 287:45 +CPU= 26.448
  15:19:53 STEP 1152 H= 288:00 +CPU= 32.407
Moving to projects directory: /home/test/slots/18/ICMGGh7zg+001152
Moving to projects directory: /home/test/slots/18/ICMSHh7zg+001152
Moving to projects directory: /home/test/slots/18/ICMUAh7zg+001152
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMGGh7zg+001056
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMSHh7zg+001056
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMUAh7zg+001056
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMGGh7zg+001068
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMSHh7zg+001068
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMUAh7zg+001068
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMGGh7zg+001080
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMSHh7zg+001080
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMUAh7zg+001080
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMGGh7zg+001092
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMSHh7zg+001092
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMUAh7zg+001092
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMGGh7zg+001104
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMSHh7zg+001104
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMUAh7zg+001104
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMGGh7zg+001116
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMSHh7zg+001116
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMUAh7zg+001116
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMGGh7zg+001128
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMSHh7zg+001128
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMUAh7zg+001128
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMGGh7zg+001140
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMSHh7zg+001140
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMUAh7zg+001140
Zipping up the intermediate file: /home/test/projects/climateprediction.net/oifs_43r3_bl_a4gv_2016092300_20_1017_12287618_0_r2086098213_11.zip
Uploading the intermediate file: upload_file_11.zip
  15:20:17 STEP 1153 H= 288:15 +CPU= 24.234
  15:20:43 STEP 1154 H= 288:30 +CPU= 25.389
  15:21:06 STEP 1155 H= 288:45 +CPU= 23.302
  15:21:32 STEP 1156 H= 289:00 +CPU= 25.184
  15:21:56 STEP 1157 H= 289:15 +CPU= 24.392
  15:22:21 STEP 1158 H= 289:30 +CPU= 24.155
  15:22:47 STEP 1159 H= 289:45 +CPU= 25.720
  15:23:13 STEP 1160 H= 290:00 +CPU= 26.210
  15:23:40 STEP 1161 H= 290:15 +CPU= 26.272
  15:24:05 STEP 1162 H= 290:30 +CPU= 24.963
  15:24:30 STEP 1163 H= 290:45 +CPU= 24.908
  15:25:02 STEP 1164 H= 291:00 +CPU= 30.729
Moving to projects directory: /home/test/slots/18/ICMGGh7zg+001164
Moving to projects directory: /home/test/slots/18/ICMSHh7zg+001164
Moving to projects directory: /home/test/slots/18/ICMUAh7zg+001164
  15:25:26 STEP 1165 H= 291:15 +CPU= 23.958
  15:25:51 STEP 1166 H= 291:30 +CPU= 24.703
  15:26:14 STEP 1167 H= 291:45 +CPU= 23.516
  15:26:40 STEP 1168 H= 292:00 +CPU= 24.974
  15:27:05 STEP 1169 H= 292:15 +CPU= 24.796
  15:27:30 STEP 1170 H= 292:30 +CPU= 25.589
  15:27:56 STEP 1171 H= 292:45 +CPU= 25.801
  15:28:22 STEP 1172 H= 293:00 +CPU= 25.752
  15:28:47 STEP 1173 H= 293:15 +CPU= 24.036
  15:29:12 STEP 1174 H= 293:30 +CPU= 25.502
  15:29:38 STEP 1175 H= 293:45 +CPU= 25.324
  15:30:08 STEP 1176 H= 294:00 +CPU= 29.770
Moving to projects directory: /home/test/slots/18/ICMGGh7zg+001176
Moving to projects directory: /home/test/slots/18/ICMSHh7zg+001176
Moving to projects directory: /home/test/slots/18/ICMUAh7zg+001176
  15:30:32 STEP 1177 H= 294:15 +CPU= 23.901
  15:30:55 STEP 1178 H= 294:30 +CPU= 23.754
  15:31:19 STEP 1179 H= 294:45 +CPU= 23.730
  15:31:45 STEP 1180 H= 295:00 +CPU= 26.199
  15:32:10 STEP 1181 H= 295:15 +CPU= 24.472
  15:32:36 STEP 1182 H= 295:30 +CPU= 25.335
  15:33:02 STEP 1183 H= 295:45 +CPU= 26.239
  15:33:27 STEP 1184 H= 296:00 +CPU= 24.448
  15:33:51 STEP 1185 H= 296:15 +CPU= 24.580
  15:34:16 STEP 1186 H= 296:30 +CPU= 23.871
  15:34:42 STEP 1187 H= 296:45 +CPU= 25.991
  15:35:15 STEP 1188 H= 297:00 +CPU= 32.626
Moving to projects directory: /home/test/slots/18/ICMGGh7zg+001188
Moving to projects directory: /home/test/slots/18/ICMSHh7zg+001188
Moving to projects directory: /home/test/slots/18/ICMUAh7zg+001188
  15:35:41 STEP 1189 H= 297:15 +CPU= 25.590
  15:36:07 STEP 1190 H= 297:30 +CPU= 25.541
  15:36:33 STEP 1191 H= 297:45 +CPU= 25.754
  15:36:59 STEP 1192 H= 298:00 +CPU= 26.600
  15:37:25 STEP 1193 H= 298:15 +CPU= 25.235
  15:38:01 STEP 1194 H= 298:30 +CPU= 36.630
  15:38:42 STEP 1195 H= 298:45 +CPU= 40.659
  15:39:30 STEP 1196 H= 299:00 +CPU= 48.355
  15:40:15 STEP 1197 H= 299:15 +CPU= 44.665
  15:40:59 STEP 1198 H= 299:30 +CPU= 43.388
  15:41:42 STEP 1199 H= 299:45 +CPU= 43.277
  15:42:46 STEP 1200 H= 300:00 +CPU= 63.726
Moving to projects directory: /home/test/slots/18/ICMGGh7zg+001200
Moving to projects directory: /home/test/slots/18/ICMSHh7zg+001200
Moving to projects directory: /home/test/slots/18/ICMUAh7zg+001200
  15:43:30 STEP 1201 H= 300:15 +CPU= 43.838
  15:44:12 STEP 1202 H= 300:30 +CPU= 42.179
  15:45:00 STEP 1203 H= 300:45 +CPU= 47.149
  15:45:49 STEP 1204 H= 301:00 +CPU= 49.718
  15:46:36 STEP 1205 H= 301:15 +CPU= 46.566
  15:47:18 STEP 1206 H= 301:30 +CPU= 42.255
  15:48:04 STEP 1207 H= 301:45 +CPU= 45.053
  15:48:46 STEP 1208 H= 302:00 +CPU= 42.568
  15:49:27 STEP 1209 H= 302:15 +CPU= 40.543
  15:50:13 STEP 1210 H= 302:30 +CPU= 46.456
  15:50:56 STEP 1211 H= 302:45 +CPU= 42.592
  15:51:46 STEP 1212 H= 303:00 +CPU= 50.113
Moving to projects directory: /home/test/slots/18/ICMGGh7zg+001212
Moving to projects directory: /home/test/slots/18/ICMSHh7zg+001212
Moving to projects directory: /home/test/slots/18/ICMUAh7zg+001212
  15:52:32 STEP 1213 H= 303:15 +CPU= 45.166
  15:53:17 STEP 1214 H= 303:30 +CPU= 45.393
  15:54:03 STEP 1215 H= 303:45 +CPU= 45.260
  15:54:51 STEP 1216 H= 304:00 +CPU= 48.445
  15:55:35 STEP 1217 H= 304:15 +CPU= 43.090
  15:56:22 STEP 1218 H= 304:30 +CPU= 47.349
  15:57:14 STEP 1219 H= 304:45 +CPU= 51.584
  15:57:57 STEP 1220 H= 305:00 +CPU= 43.424
  15:58:36 STEP 1221 H= 305:15 +CPU= 39.113
  15:59:21 STEP 1222 H= 305:30 +CPU= 44.845
  16:00:04 STEP 1223 H= 305:45 +CPU= 42.450
  16:00:59 STEP 1224 H= 306:00 +CPU= 54.642
Moving to projects directory: /home/test/slots/18/ICMGGh7zg+001224
Moving to projects directory: /home/test/slots/18/ICMSHh7zg+001224
Moving to projects directory: /home/test/slots/18/ICMUAh7zg+001224
  16:01:40 STEP 1225 H= 306:15 +CPU= 41.335
  16:02:31 STEP 1226 H= 306:30 +CPU= 50.495
  16:03:19 STEP 1227 H= 306:45 +CPU= 48.439
  16:04:03 STEP 1228 H= 307:00 +CPU= 43.937
  16:04:50 STEP 1229 H= 307:15 +CPU= 46.137
  16:05:34 STEP 1230 H= 307:30 +CPU= 44.687
  16:06:17 STEP 1231 H= 307:45 +CPU= 42.745
  16:07:06 STEP 1232 H= 308:00 +CPU= 47.988
  16:07:49 STEP 1233 H= 308:15 +CPU= 43.269
  16:08:34 STEP 1234 H= 308:30 +CPU= 44.137
  16:09:24 STEP 1235 H= 308:45 +CPU= 49.746
  16:10:22 STEP 1236 H= 309:00 +CPU= 58.071
Moving to projects directory: /home/test/slots/18/ICMGGh7zg+001236
Moving to projects directory: /home/test/slots/18/ICMSHh7zg+001236
Moving to projects directory: /home/test/slots/18/ICMUAh7zg+001236
  16:11:06 STEP 1237 H= 309:15 +CPU= 43.885
  16:11:47 STEP 1238 H= 309:30 +CPU= 41.277
  16:12:38 STEP 1239 H= 309:45 +CPU= 50.315
  16:13:33 STEP 1240 H= 310:00 +CPU= 55.099
  16:14:28 STEP 1241 H= 310:15 +CPU= 54.521
  16:15:17 STEP 1242 H= 310:30 +CPU= 49.027
  16:16:07 STEP 1243 H= 310:45 +CPU= 49.745
  16:16:53 STEP 1244 H= 311:00 +CPU= 46.149
  16:17:38 STEP 1245 H= 311:15 +CPU= 44.820
  16:18:23 STEP 1246 H= 311:30 +CPU= 44.493
  16:19:06 STEP 1247 H= 311:45 +CPU= 42.689
  16:20:05 STEP 1248 H= 312:00 +CPU= 59.078
Moving to projects directory: /home/test/slots/18/ICMGGh7zg+001248
Moving to projects directory: /home/test/slots/18/ICMSHh7zg+001248
Moving to projects directory: /home/test/slots/18/ICMUAh7zg+001248
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMGGh7zg+001152
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMSHh7zg+001152
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMUAh7zg+001152
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMGGh7zg+001164
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMSHh7zg+001164
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMUAh7zg+001164
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMGGh7zg+001176
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMSHh7zg+001176
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMUAh7zg+001176
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMGGh7zg+001188
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMSHh7zg+001188
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMUAh7zg+001188
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMGGh7zg+001200
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMSHh7zg+001200
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMUAh7zg+001200
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMGGh7zg+001212
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMSHh7zg+001212
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMUAh7zg+001212
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMGGh7zg+001224
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMSHh7zg+001224
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMUAh7zg+001224
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMGGh7zg+001236
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMSHh7zg+001236
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMUAh7zg+001236
Zipping up the intermediate file: /home/test/projects/climateprediction.net/oifs_43r3_bl_a4gv_2016092300_20_1017_12287618_0_r2086098213_12.zip
Uploading the intermediate file: upload_file_12.zip
  16:20:48 STEP 1249 H= 312:15 +CPU= 42.693
  16:21:42 STEP 1250 H= 312:30 +CPU= 53.952
  16:22:32 STEP 1251 H= 312:45 +CPU= 49.622
  16:23:22 STEP 1252 H= 313:00 +CPU= 49.301
  16:24:06 STEP 1253 H= 313:15 +CPU= 43.896
  16:24:55 STEP 1254 H= 313:30 +CPU= 49.035
  16:25:34 STEP 1255 H= 313:45 +CPU= 38.296
  16:26:28 STEP 1256 H= 314:00 +CPU= 53.721
  16:27:12 STEP 1257 H= 314:15 +CPU= 44.008
  16:28:00 STEP 1258 H= 314:30 +CPU= 48.096
  16:28:45 STEP 1259 H= 314:45 +CPU= 44.650
  16:29:39 STEP 1260 H= 315:00 +CPU= 54.196
Moving to projects directory: /home/test/slots/18/ICMGGh7zg+001260
Moving to projects directory: /home/test/slots/18/ICMSHh7zg+001260
Moving to projects directory: /home/test/slots/18/ICMUAh7zg+001260
  16:30:26 STEP 1261 H= 315:15 +CPU= 47.102
  16:31:11 STEP 1262 H= 315:30 +CPU= 44.052
  16:31:49 STEP 1263 H= 315:45 +CPU= 38.231
  16:32:27 STEP 1264 H= 316:00 +CPU= 38.406
  16:33:07 STEP 1265 H= 316:15 +CPU= 39.322
  16:33:46 STEP 1266 H= 316:30 +CPU= 39.304
  16:34:24 STEP 1267 H= 316:45 +CPU= 37.469
  16:35:05 STEP 1268 H= 317:00 +CPU= 41.400
  16:35:39 STEP 1269 H= 317:15 +CPU= 34.000
  16:36:13 STEP 1270 H= 317:30 +CPU= 32.937
  16:36:46 STEP 1271 H= 317:45 +CPU= 33.610
  16:37:30 STEP 1272 H= 318:00 +CPU= 43.512
Moving to projects directory: /home/test/slots/18/ICMGGh7zg+001272
Moving to projects directory: /home/test/slots/18/ICMSHh7zg+001272
Moving to projects directory: /home/test/slots/18/ICMUAh7zg+001272
  16:38:06 STEP 1273 H= 318:15 +CPU= 36.463
  16:38:35 STEP 1274 H= 318:30 +CPU= 28.857
  16:39:04 STEP 1275 H= 318:45 +CPU= 28.636
  16:39:33 STEP 1276 H= 319:00 +CPU= 29.359
  16:40:02 STEP 1277 H= 319:15 +CPU= 28.501
  16:40:31 STEP 1278 H= 319:30 +CPU= 29.600
  16:41:00 STEP 1279 H= 319:45 +CPU= 28.810
  16:41:30 STEP 1280 H= 320:00 +CPU= 29.552
  16:41:58 STEP 1281 H= 320:15 +CPU= 28.228
  16:42:24 STEP 1282 H= 320:30 +CPU= 25.613
  16:42:50 STEP 1283 H= 320:45 +CPU= 25.694
  16:43:24 STEP 1284 H= 321:00 +CPU= 34.428
Moving to projects directory: /home/test/slots/18/ICMGGh7zg+001284
Moving to projects directory: /home/test/slots/18/ICMSHh7zg+001284
Moving to projects directory: /home/test/slots/18/ICMUAh7zg+001284
  16:43:50 STEP 1285 H= 321:15 +CPU= 25.405
  16:44:17 STEP 1286 H= 321:30 +CPU= 26.679
  16:44:44 STEP 1287 H= 321:45 +CPU= 26.772
  16:45:12 STEP 1288 H= 322:00 +CPU= 28.344
  16:45:41 STEP 1289 H= 322:15 +CPU= 28.981
  16:46:08 STEP 1290 H= 322:30 +CPU= 26.602
  16:46:33 STEP 1291 H= 322:45 +CPU= 25.149
  16:47:00 STEP 1292 H= 323:00 +CPU= 27.190
  16:47:26 STEP 1293 H= 323:15 +CPU= 25.738
  16:47:51 STEP 1294 H= 323:30 +CPU= 25.260
  16:48:17 STEP 1295 H= 323:45 +CPU= 25.295
  16:48:52 STEP 1296 H= 324:00 +CPU= 34.702
Moving to projects directory: /home/test/slots/18/ICMGGh7zg+001296
Moving to projects directory: /home/test/slots/18/ICMSHh7zg+001296
Moving to projects directory: /home/test/slots/18/ICMUAh7zg+001296
  16:49:18 STEP 1297 H= 324:15 +CPU= 26.348
  16:49:42 STEP 1298 H= 324:30 +CPU= 24.047
  16:50:08 STEP 1299 H= 324:45 +CPU= 25.196
  16:50:34 STEP 1300 H= 325:00 +CPU= 26.716
  16:51:00 STEP 1301 H= 325:15 +CPU= 26.114
  16:51:25 STEP 1302 H= 325:30 +CPU= 24.149
  16:51:49 STEP 1303 H= 325:45 +CPU= 24.524
  16:52:14 STEP 1304 H= 326:00 +CPU= 24.661
  16:52:39 STEP 1305 H= 326:15 +CPU= 24.840
  16:53:04 STEP 1306 H= 326:30 +CPU= 24.620
  16:53:29 STEP 1307 H= 326:45 +CPU= 25.747
  16:54:02 STEP 1308 H= 327:00 +CPU= 32.770
Moving to projects directory: /home/test/slots/18/ICMGGh7zg+001308
Moving to projects directory: /home/test/slots/18/ICMSHh7zg+001308
Moving to projects directory: /home/test/slots/18/ICMUAh7zg+001308
  16:54:28 STEP 1309 H= 327:15 +CPU= 25.452
  16:54:53 STEP 1310 H= 327:30 +CPU= 24.503
  16:55:18 STEP 1311 H= 327:45 +CPU= 24.786
  16:55:42 STEP 1312 H= 328:00 +CPU= 24.663
  16:56:08 STEP 1313 H= 328:15 +CPU= 24.912
  16:56:33 STEP 1314 H= 328:30 +CPU= 25.260
  16:56:58 STEP 1315 H= 328:45 +CPU= 24.507
  16:57:23 STEP 1316 H= 329:00 +CPU= 25.460
  16:57:49 STEP 1317 H= 329:15 +CPU= 25.361
  16:58:13 STEP 1318 H= 329:30 +CPU= 23.945
  16:58:37 STEP 1319 H= 329:45 +CPU= 24.125
  16:59:09 STEP 1320 H= 330:00 +CPU= 31.548
Moving to projects directory: /home/test/slots/18/ICMGGh7zg+001320
Moving to projects directory: /home/test/slots/18/ICMSHh7zg+001320
Moving to projects directory: /home/test/slots/18/ICMUAh7zg+001320
  16:59:33 STEP 1321 H= 330:15 +CPU= 24.302
  16:59:58 STEP 1322 H= 330:30 +CPU= 24.688
  17:00:23 STEP 1323 H= 330:45 +CPU= 25.236
  17:00:51 STEP 1324 H= 331:00 +CPU= 27.341
  17:01:16 STEP 1325 H= 331:15 +CPU= 25.185
  17:01:42 STEP 1326 H= 331:30 +CPU= 25.877
  17:02:06 STEP 1327 H= 331:45 +CPU= 24.525
  17:02:32 STEP 1328 H= 332:00 +CPU= 25.119
  17:02:57 STEP 1329 H= 332:15 +CPU= 25.098
  17:03:23 STEP 1330 H= 332:30 +CPU= 25.978
  17:03:49 STEP 1331 H= 332:45 +CPU= 26.003
  17:04:22 STEP 1332 H= 333:00 +CPU= 32.572
Moving to projects directory: /home/test/slots/18/ICMGGh7zg+001332
Moving to projects directory: /home/test/slots/18/ICMSHh7zg+001332
Moving to projects directory: /home/test/slots/18/ICMUAh7zg+001332
  17:04:47 STEP 1333 H= 333:15 +CPU= 25.655
  17:05:12 STEP 1334 H= 333:30 +CPU= 24.866
  17:05:37 STEP 1335 H= 333:45 +CPU= 24.914
  17:06:03 STEP 1336 H= 334:00 +CPU= 25.643
  17:06:28 STEP 1337 H= 334:15 +CPU= 24.801
  17:06:53 STEP 1338 H= 334:30 +CPU= 25.433
  17:07:19 STEP 1339 H= 334:45 +CPU= 25.688
  17:07:44 STEP 1340 H= 335:00 +CPU= 24.952
  17:08:10 STEP 1341 H= 335:15 +CPU= 25.549
  17:08:36 STEP 1342 H= 335:30 +CPU= 25.704
  17:09:01 STEP 1343 H= 335:45 +CPU= 23.976
  17:09:31 STEP 1344 H= 336:00 +CPU= 30.581
Moving to projects directory: /home/test/slots/18/ICMGGh7zg+001344
Moving to projects directory: /home/test/slots/18/ICMSHh7zg+001344
Moving to projects directory: /home/test/slots/18/ICMUAh7zg+001344
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMGGh7zg+001248
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMSHh7zg+001248
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMUAh7zg+001248
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMGGh7zg+001260
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMSHh7zg+001260
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMUAh7zg+001260
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMGGh7zg+001272
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMSHh7zg+001272
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMUAh7zg+001272
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMGGh7zg+001284
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMSHh7zg+001284
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMUAh7zg+001284
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMGGh7zg+001296
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMSHh7zg+001296
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMUAh7zg+001296
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMGGh7zg+001308
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMSHh7zg+001308
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMUAh7zg+001308
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMGGh7zg+001320
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMSHh7zg+001320
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMUAh7zg+001320
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMGGh7zg+001332
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMSHh7zg+001332
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMUAh7zg+001332
Zipping up the intermediate file: /home/test/projects/climateprediction.net/oifs_43r3_bl_a4gv_2016092300_20_1017_12287618_0_r2086098213_13.zip
Uploading the intermediate file: upload_file_13.zip
  17:09:56 STEP 1345 H= 336:15 +CPU= 23.925
  17:10:19 STEP 1346 H= 336:30 +CPU= 23.174
  17:10:42 STEP 1347 H= 336:45 +CPU= 23.331
  17:11:07 STEP 1348 H= 337:00 +CPU= 24.132
  17:11:31 STEP 1349 H= 337:15 +CPU= 24.780
  17:11:54 STEP 1350 H= 337:30 +CPU= 22.748
  17:12:18 STEP 1351 H= 337:45 +CPU= 23.816
  17:12:43 STEP 1352 H= 338:00 +CPU= 25.248
  17:13:08 STEP 1353 H= 338:15 +CPU= 24.881
  17:13:34 STEP 1354 H= 338:30 +CPU= 25.873
  17:14:00 STEP 1355 H= 338:45 +CPU= 25.529
  17:14:33 STEP 1356 H= 339:00 +CPU= 33.038
Moving to projects directory: /home/test/slots/18/ICMGGh7zg+001356
Moving to projects directory: /home/test/slots/18/ICMSHh7zg+001356
Moving to projects directory: /home/test/slots/18/ICMUAh7zg+001356
  17:14:58 STEP 1357 H= 339:15 +CPU= 25.044
  17:15:24 STEP 1358 H= 339:30 +CPU= 25.740
  17:15:48 STEP 1359 H= 339:45 +CPU= 24.365
  17:16:13 STEP 1360 H= 340:00 +CPU= 25.080
  17:16:38 STEP 1361 H= 340:15 +CPU= 24.260
  17:17:02 STEP 1362 H= 340:30 +CPU= 24.531
  17:17:26 STEP 1363 H= 340:45 +CPU= 23.988
  17:17:51 STEP 1364 H= 341:00 +CPU= 24.862
  17:18:16 STEP 1365 H= 341:15 +CPU= 25.040
  17:18:41 STEP 1366 H= 341:30 +CPU= 24.803
  17:19:07 STEP 1367 H= 341:45 +CPU= 26.179
  17:19:40 STEP 1368 H= 342:00 +CPU= 32.641
Moving to projects directory: /home/test/slots/18/ICMGGh7zg+001368
Moving to projects directory: /home/test/slots/18/ICMSHh7zg+001368
Moving to projects directory: /home/test/slots/18/ICMUAh7zg+001368
  17:20:06 STEP 1369 H= 342:15 +CPU= 24.956
  17:20:31 STEP 1370 H= 342:30 +CPU= 25.039
  17:20:57 STEP 1371 H= 342:45 +CPU= 25.901
  17:21:23 STEP 1372 H= 343:00 +CPU= 25.567
  17:21:47 STEP 1373 H= 343:15 +CPU= 24.080
  17:22:11 STEP 1374 H= 343:30 +CPU= 24.593
  17:22:37 STEP 1375 H= 343:45 +CPU= 25.721
  17:23:04 STEP 1376 H= 344:00 +CPU= 26.766
  17:23:30 STEP 1377 H= 344:15 +CPU= 25.285
  17:23:54 STEP 1378 H= 344:30 +CPU= 24.427
  17:24:19 STEP 1379 H= 344:45 +CPU= 24.272
  17:24:52 STEP 1380 H= 345:00 +CPU= 32.889
Moving to projects directory: /home/test/slots/18/ICMGGh7zg+001380
Moving to projects directory: /home/test/slots/18/ICMSHh7zg+001380
Moving to projects directory: /home/test/slots/18/ICMUAh7zg+001380
  17:25:17 STEP 1381 H= 345:15 +CPU= 25.213
  17:25:41 STEP 1382 H= 345:30 +CPU= 24.220
  17:26:06 STEP 1383 H= 345:45 +CPU= 24.955
  17:26:31 STEP 1384 H= 346:00 +CPU= 25.031
  17:26:56 STEP 1385 H= 346:15 +CPU= 24.478
  17:27:21 STEP 1386 H= 346:30 +CPU= 25.032
  17:27:49 STEP 1387 H= 346:45 +CPU= 27.724
  17:28:14 STEP 1388 H= 347:00 +CPU= 25.167
  17:28:38 STEP 1389 H= 347:15 +CPU= 23.348
  17:29:02 STEP 1390 H= 347:30 +CPU= 24.262
  17:29:27 STEP 1391 H= 347:45 +CPU= 25.142
  17:30:02 STEP 1392 H= 348:00 +CPU= 34.717
Moving to projects directory: /home/test/slots/18/ICMGGh7zg+001392
Moving to projects directory: /home/test/slots/18/ICMSHh7zg+001392
Moving to projects directory: /home/test/slots/18/ICMUAh7zg+001392
  17:30:27 STEP 1393 H= 348:15 +CPU= 24.910
  17:30:52 STEP 1394 H= 348:30 +CPU= 24.559
  17:31:17 STEP 1395 H= 348:45 +CPU= 25.165
  17:31:42 STEP 1396 H= 349:00 +CPU= 25.591
  17:32:08 STEP 1397 H= 349:15 +CPU= 25.282
  17:32:32 STEP 1398 H= 349:30 +CPU= 23.982
  17:32:57 STEP 1399 H= 349:45 +CPU= 24.644
  17:33:23 STEP 1400 H= 350:00 +CPU= 26.829
  17:33:48 STEP 1401 H= 350:15 +CPU= 24.216
  17:34:12 STEP 1402 H= 350:30 +CPU= 24.479
  17:34:37 STEP 1403 H= 350:45 +CPU= 24.396
  17:35:08 STEP 1404 H= 351:00 +CPU= 31.413
Moving to projects directory: /home/test/slots/18/ICMGGh7zg+001404
Moving to projects directory: /home/test/slots/18/ICMSHh7zg+001404
Moving to projects directory: /home/test/slots/18/ICMUAh7zg+001404
  17:35:32 STEP 1405 H= 351:15 +CPU= 24.140
  17:35:58 STEP 1406 H= 351:30 +CPU= 25.174
  17:36:24 STEP 1407 H= 351:45 +CPU= 26.296
  17:36:48 STEP 1408 H= 352:00 +CPU= 24.330
  17:37:13 STEP 1409 H= 352:15 +CPU= 24.366
  17:37:38 STEP 1410 H= 352:30 +CPU= 24.778
  17:38:03 STEP 1411 H= 352:45 +CPU= 25.713
  17:38:28 STEP 1412 H= 353:00 +CPU= 24.875
  17:38:53 STEP 1413 H= 353:15 +CPU= 24.951
  17:39:18 STEP 1414 H= 353:30 +CPU= 24.770
  17:39:44 STEP 1415 H= 353:45 +CPU= 25.256
  17:40:15 STEP 1416 H= 354:00 +CPU= 31.254
Moving to projects directory: /home/test/slots/18/ICMGGh7zg+001416
Moving to projects directory: /home/test/slots/18/ICMSHh7zg+001416
Moving to projects directory: /home/test/slots/18/ICMUAh7zg+001416
  17:40:39 STEP 1417 H= 354:15 +CPU= 23.841
  17:41:05 STEP 1418 H= 354:30 +CPU= 25.727
  17:41:29 STEP 1419 H= 354:45 +CPU= 23.720
  17:41:54 STEP 1420 H= 355:00 +CPU= 25.264
  17:42:18 STEP 1421 H= 355:15 +CPU= 24.070
  17:42:43 STEP 1422 H= 355:30 +CPU= 24.359
  17:43:08 STEP 1423 H= 355:45 +CPU= 25.077
  17:43:34 STEP 1424 H= 356:00 +CPU= 25.731
  17:43:58 STEP 1425 H= 356:15 +CPU= 24.267
  17:44:25 STEP 1426 H= 356:30 +CPU= 26.191
  17:44:52 STEP 1427 H= 356:45 +CPU= 27.022
  17:45:22 STEP 1428 H= 357:00 +CPU= 30.172
Moving to projects directory: /home/test/slots/18/ICMGGh7zg+001428
Moving to projects directory: /home/test/slots/18/ICMSHh7zg+001428
Moving to projects directory: /home/test/slots/18/ICMUAh7zg+001428
  17:45:47 STEP 1429 H= 357:15 +CPU= 24.844
  17:46:12 STEP 1430 H= 357:30 +CPU= 24.737
  17:46:37 STEP 1431 H= 357:45 +CPU= 25.140
  17:47:05 STEP 1432 H= 358:00 +CPU= 27.593
  17:47:31 STEP 1433 H= 358:15 +CPU= 25.834
  17:47:57 STEP 1434 H= 358:30 +CPU= 25.918
  17:48:24 STEP 1435 H= 358:45 +CPU= 26.731
  17:48:57 STEP 1436 H= 359:00 +CPU= 33.022
  17:49:32 STEP 1437 H= 359:15 +CPU= 35.316
  17:50:08 STEP 1438 H= 359:30 +CPU= 35.382
  17:50:43 STEP 1439 H= 359:45 +CPU= 35.035
  17:51:27 STEP 1440 H= 360:00 +CPU= 43.347
..The child process terminated with status: 0
>>> Printing last 70 lines from file: NODE.001_01

 
DDH------------------------------------------------------------------------
I                                                                         I
I TIME =  360.000H DDH * Diagnostic Var.,     AREA                GLOBAL  I
I                                                                         I
I-------------------------------------------------------------------------I
I   VARIABLE   I      Initial value         I        Current value        I
I-------------------------------------------------------------------------I
I  OMEGA (P/S) I     -0.5145576445E-21      I       0.6840273290E-06      I
ICloud fractionI      0.0000000000E+00      I       0.3357017290E-01      I
I  Relat. Hum. I      0.2253349673E+00      I       0.3092974760E+00      I
I  PASS 01     I      0.0000000000E+00      I       0.2530062705E+03      I
DDH------------------------------------------------------------------------
  
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
 NSTEP =  1440 SCAN2M_HPOS  P
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ICMGGh7zg+001440 MODE=a
 IO-STREAM CLOSED - ICMGGh7zg+001440
 MPI-TASK:   1 -    316064660 BYTES IN      7 RECORDS TRANSFERRED IN    0.0007 SECONDS451520.9 Mbytes/s, TOTAL TIME=    0.0154( 4.5%)
 17:51:27 STEP 1440 H= 360:00 +CPU= 43.347
 END CNT3
 NSTEP =  1440 CNT0   000000000  31.813  31.813  31.998   0.994               0               0               0               0               0
IO-STREAM STATISTICS, TOTAL NO OF REC/BYTES READ           46/  1069125708 WRITTEN        27170/************
 -TASK-OPENED-OPEN-RECS IN -KBYTE IN    -RECS OUT-KBYTE OUT-WALL   -WALL IN-WALL
  OU-TOT IN -TOT OUT
   1    478    0       46      1044068    27170 ************    87.2     2.8    84.4     5.3   167.2
===-=== START OF TIMING STATISTICS ===-===
 
STATS FOR ALL TASKS
 NUM ROUTINE                                     CALLS  MEAN(ms)   MAX(ms)   FRAC(%)  UNBAL(%)
   0 CNT0     - COMPLETE EXECUTION                   1 ********* *********    100.00      0.00
   1 CNT4     - FORWARD INTEGRATION                  1 ********* *********     99.98      0.00
   8 SCAN2M - GRID-POINT DYNAMICS                  989    9047.2    9047.2     21.42      0.00
   9 SPCM     - SPECTRAL COMP.                     912    2202.6    2202.6      4.81      0.00
  10 SCAN2M - PHYSICS                              913   19150.7   19150.7     41.86      0.00
  11 IOPACK   - OUTPUT P.P. RESULTS                 76    1774.7    1774.7      0.32      0.00
  12 SPNORM   - SPECTRAL NORM COMP.                 39     127.0     127.0      0.01      0.00
  17 GRIDFPOS IN CNT4                               76      48.9      48.9      0.01      0.00
  27 RESTART FILES - WRITING                        19    7098.9    7098.9      0.32      0.00
  28 RESTART FILES - READING                         1    7384.2    7384.2      0.02      0.00
  29 SU4FPOS IN CNT4                                76       0.1       0.1      0.00      0.00
  30 DYNFPOS IN CNT4                                76    8006.2    8006.2      1.46      0.00
  31 POSDDH IN STEPO                                 4       4.6       4.6      0.00      0.00
  37 CPGLAG   - SL COMPUTATIONS                    913    8564.5    8564.5     18.72      0.00
  39 SU0YOMB                                         1     706.5     706.5      0.00      0.00
  51 SCAN2M   - SL COMM. PART 1                    913      53.5      53.5      0.12      0.00
  54 SPCM     - M TO S/S TO M TRANSP.              912    1165.0    1165.0      2.54      0.00
  55 SPCIMPF  - S TO M/M TO S TRANSP.              912     122.0     122.0      0.27      0.00
  56 SPNORM   - SPECTRAL NORM COMM.                 39       0.8       0.8      0.00      0.00
 102 LTINV_CTL   - INVERSE LEGENDRE TRANSFORM     1980     575.5     575.5      2.73      0.00
 103 LTDIR_CTL   - DIRECT LEGENDRE TRANSFORM      1900     317.1     317.1      1.44      0.00
 106 FTDIR_CTL   - DIRECT FOURIER TRANSFORM       1900     142.1     142.1      0.65      0.00
 107 FTINV_CTL   - INVERSE FOURIER TRANSFORM      1980     207.7     207.7      0.98      0.00
 140 SULEG       - COMP. OF LEGENDRE POL.            1      62.0      62.0      0.00      0.00
 152 LTINV_CTL   - M TO L TRANSPOSITION           1980     133.6     133.6      0.63      0.00
 153 LTDIR_CTL   - L TO M TRANSPOSITION           1900      84.1      84.1      0.38      0.00
 157 FTINV_CTL   - L TO G TRANSPOSITION           1980     222.3     222.3      1.05      0.00
 158 FTDIR_CTL   - G TO L TRANSPOSITION           1900     110.0     110.0      0.50      0.00
 400 GSTATS                                     106546       0.0       0.0      0.00      0.00
 401 GSTATS HOOK                                 99080       0.0       0.0      0.00      0.00
TOTAL MEASURED IMBALANCE =       0.0 SECONDS,  0.0 PERCENT
TOTAL WALLCLOCK TIME    41765.5 CPU TIME   41465.7 VECTOR TIME    41465.7


===-=== END   OF TIMING STATISTICS ===-===
FORECAST DAYS PER DAY       31.0
  *** END CNT0 *** 
------------------------------------------------
Moving to projects directory: /home/test/slots/18/ICMGGh7zg+001440
Moving to projects directory: /home/test/slots/18/ICMSHh7zg+001440
Moving to projects directory: /home/test/slots/18/ICMUAh7zg+001440
Adding to the zip: /home/test/slots/18/NODE.001_01
Adding to the zip: /home/test/slots/18/ifs.stat
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMGGh7zg+001344
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMSHh7zg+001380
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMGGh7zg+001440
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMSHh7zg+001368
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMUAh7zg+001428
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMSHh7zg+001416
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMUAh7zg+001380
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMGGh7zg+001416
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMUAh7zg+001344
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMSHh7zg+001404
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMGGh7zg+001356
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMUAh7zg+001356
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMGGh7zg+001428
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMSHh7zg+001356
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMUAh7zg+001404
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMGGh7zg+001392
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMGGh7zg+001380
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMUAh7zg+001392
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMUAh7zg+001416
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMSHh7zg+001392
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMGGh7zg+001368
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMGGh7zg+001404
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMSHh7zg+001428
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMUAh7zg+001368
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMSHh7zg+001440
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMUAh7zg+001440
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12287618/ICMSHh7zg+001344
Zipping up the final file: /home/test/projects/climateprediction.net/oifs_43r3_bl_a4gv_2016092300_20_1017_12287618_0_r2086098213_14.zip
Uploading the final file: upload_file_14.zip
Uploading trickle at timestep: 1295100
17:54:57 (8865): called boinc_finish(0)

</stderr_txt>
<message>
upload failure: <file_xfer_error>
  <file_name>oifs_43r3_bl_a4gv_2016092300_20_1017_12287618_0_r2086098213_15.zip</file_name>
  <error_code>-161 (not found)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>oifs_43r3_bl_a4gv_2016092300_20_1017_12287618_0_r2086098213_16.zip</file_name>
  <error_code>-161 (not found)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>oifs_43r3_bl_a4gv_2016092300_20_1017_12287618_0_r2086098213_17.zip</file_name>
  <error_code>-161 (not found)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>oifs_43r3_bl_a4gv_2016092300_20_1017_12287618_0_r2086098213_18.zip</file_name>
  <error_code>-161 (not found)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>oifs_43r3_bl_a4gv_2016092300_20_1017_12287618_0_r2086098213_19.zip</file_name>
  <error_code>-161 (not found)</error_code>
</file_xfer_error>
</message>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
08 Jun 2024 23:09:50 1545820 22439259 oifs_43r3_bl_a4gv_2016092300_20_1017_12287618_0 1,295,100 73,700 0.0569


©2024 cpdn.org