climateprediction.net home page
Task 22434072

Task 22434072

Name oifs_43r3_bl_a0iu_2016092300_20_1017_12282505_0
Workunit 12282505
Created 7 Jun 2024, 11:49:39 UTC
Sent 8 Jun 2024, 0:23:41 UTC
Report deadline 7 Aug 2024, 0:23:41 UTC
Received 8 Jun 2024, 23:08:21 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 0 (0x00000000)
Computer ID 1545820
Run time 21 hours 29 min 53 sec
CPU time 21 hours 29 min 42 sec
Validate state Invalid
Credit 1,976.32
Device peak FLOPS 6.25 GFLOPS
Application version OpenIFS 43r3 Baroclinic Lifecycle v1.13
x86_64-pc-linux-gnu
Peak working set size 5,021.23 MB
Peak swap size 5,980.80 MB
Peak disk usage 1,363.68 MB
Stderr
<core_client_version>7.22.1</core_client_version>
<![CDATA[
<stderr_txt>
2 H= 193:00 +CPU= 70.811
  08:57:13 STEP  773 H= 193:15 +CPU= 67.482
  08:58:25 STEP  774 H= 193:30 +CPU= 71.962
  08:59:40 STEP  775 H= 193:45 +CPU= 73.628
  09:00:55 STEP  776 H= 194:00 +CPU= 74.424
  09:02:09 STEP  777 H= 194:15 +CPU= 73.435
  09:03:17 STEP  778 H= 194:30 +CPU= 67.539
  09:04:23 STEP  779 H= 194:45 +CPU= 64.845
  09:05:50 STEP  780 H= 195:00 +CPU= 85.972
Moving to projects directory: /home/test/slots/36/ICMGGh7zg+000780
Moving to projects directory: /home/test/slots/36/ICMSHh7zg+000780
Moving to projects directory: /home/test/slots/36/ICMUAh7zg+000780
  09:06:58 STEP  781 H= 195:15 +CPU= 67.471
  09:08:10 STEP  782 H= 195:30 +CPU= 71.311
  09:09:19 STEP  783 H= 195:45 +CPU= 67.562
  09:10:33 STEP  784 H= 196:00 +CPU= 73.262
  09:11:46 STEP  785 H= 196:15 +CPU= 72.812
  09:13:00 STEP  786 H= 196:30 +CPU= 73.773
  09:14:08 STEP  787 H= 196:45 +CPU= 66.960
  09:15:21 STEP  788 H= 197:00 +CPU= 73.161
  09:16:38 STEP  789 H= 197:15 +CPU= 76.291
  09:17:41 STEP  790 H= 197:30 +CPU= 62.155
  09:19:01 STEP  791 H= 197:45 +CPU= 79.412
  09:20:20 STEP  792 H= 198:00 +CPU= 78.919
Moving to projects directory: /home/test/slots/36/ICMGGh7zg+000792
Moving to projects directory: /home/test/slots/36/ICMSHh7zg+000792
Moving to projects directory: /home/test/slots/36/ICMUAh7zg+000792
  09:21:28 STEP  793 H= 198:15 +CPU= 66.638
  09:22:33 STEP  794 H= 198:30 +CPU= 64.175
  09:23:51 STEP  795 H= 198:45 +CPU= 77.204
  09:25:03 STEP  796 H= 199:00 +CPU= 70.972
  09:26:17 STEP  797 H= 199:15 +CPU= 73.664
  09:27:25 STEP  798 H= 199:30 +CPU= 67.573
  09:28:38 STEP  799 H= 199:45 +CPU= 72.175
  09:29:45 STEP  800 H= 200:00 +CPU= 65.917
  09:31:01 STEP  801 H= 200:15 +CPU= 75.139
  09:32:11 STEP  802 H= 200:30 +CPU= 69.306
  09:33:21 STEP  803 H= 200:45 +CPU= 69.734
  09:34:46 STEP  804 H= 201:00 +CPU= 83.829
Moving to projects directory: /home/test/slots/36/ICMGGh7zg+000804
Moving to projects directory: /home/test/slots/36/ICMSHh7zg+000804
Moving to projects directory: /home/test/slots/36/ICMUAh7zg+000804
  09:35:53 STEP  805 H= 201:15 +CPU= 65.831
  09:37:09 STEP  806 H= 201:30 +CPU= 74.614
  09:38:17 STEP  807 H= 201:45 +CPU= 68.274
  09:39:25 STEP  808 H= 202:00 +CPU= 66.295
  09:40:39 STEP  809 H= 202:15 +CPU= 73.732
  09:41:52 STEP  810 H= 202:30 +CPU= 72.342
  09:42:59 STEP  811 H= 202:45 +CPU= 66.665
  09:44:13 STEP  812 H= 203:00 +CPU= 73.725
  09:45:20 STEP  813 H= 203:15 +CPU= 66.815
  09:46:28 STEP  814 H= 203:30 +CPU= 67.113
  09:47:39 STEP  815 H= 203:45 +CPU= 69.676
  09:49:20 STEP  816 H= 204:00 +CPU=100.161
Moving to projects directory: /home/test/slots/36/ICMGGh7zg+000816
Moving to projects directory: /home/test/slots/36/ICMSHh7zg+000816
Moving to projects directory: /home/test/slots/36/ICMUAh7zg+000816
  09:50:25 STEP  817 H= 204:15 +CPU= 64.657
  09:51:34 STEP  818 H= 204:30 +CPU= 67.608
  09:52:43 STEP  819 H= 204:45 +CPU= 67.617
  09:53:56 STEP  820 H= 205:00 +CPU= 72.013
  09:55:01 STEP  821 H= 205:15 +CPU= 63.956
  09:56:06 STEP  822 H= 205:30 +CPU= 64.140
  09:57:21 STEP  823 H= 205:45 +CPU= 74.930
  09:58:36 STEP  824 H= 206:00 +CPU= 74.369
  09:59:50 STEP  825 H= 206:15 +CPU= 72.558
  10:00:55 STEP  826 H= 206:30 +CPU= 64.828
  10:02:11 STEP  827 H= 206:45 +CPU= 75.780
  10:03:31 STEP  828 H= 207:00 +CPU= 79.668
Moving to projects directory: /home/test/slots/36/ICMGGh7zg+000828
Moving to projects directory: /home/test/slots/36/ICMSHh7zg+000828
Moving to projects directory: /home/test/slots/36/ICMUAh7zg+000828
  10:04:54 STEP  829 H= 207:15 +CPU= 82.182
  10:06:01 STEP  830 H= 207:30 +CPU= 67.468
  10:07:08 STEP  831 H= 207:45 +CPU= 65.810
  10:08:18 STEP  832 H= 208:00 +CPU= 69.742
  10:09:32 STEP  833 H= 208:15 +CPU= 73.606
  10:10:43 STEP  834 H= 208:30 +CPU= 71.025
  10:11:52 STEP  835 H= 208:45 +CPU= 67.678
  10:13:07 STEP  836 H= 209:00 +CPU= 74.577
  10:14:12 STEP  837 H= 209:15 +CPU= 63.931
  10:15:24 STEP  838 H= 209:30 +CPU= 72.494
  10:16:34 STEP  839 H= 209:45 +CPU= 69.150
  10:17:51 STEP  840 H= 210:00 +CPU= 76.556
Moving to projects directory: /home/test/slots/36/ICMGGh7zg+000840
Moving to projects directory: /home/test/slots/36/ICMSHh7zg+000840
Moving to projects directory: /home/test/slots/36/ICMUAh7zg+000840
  10:19:01 STEP  841 H= 210:15 +CPU= 68.807
  10:20:06 STEP  842 H= 210:30 +CPU= 64.692
  10:21:10 STEP  843 H= 210:45 +CPU= 63.162
  10:22:20 STEP  844 H= 211:00 +CPU= 69.055
  10:23:36 STEP  845 H= 211:15 +CPU= 74.939
  10:24:40 STEP  846 H= 211:30 +CPU= 63.351
  10:25:50 STEP  847 H= 211:45 +CPU= 69.259
  10:27:05 STEP  848 H= 212:00 +CPU= 75.127
  10:28:20 STEP  849 H= 212:15 +CPU= 74.002
  10:29:34 STEP  850 H= 212:30 +CPU= 73.952
  10:30:43 STEP  851 H= 212:45 +CPU= 68.154
  10:32:09 STEP  852 H= 213:00 +CPU= 85.418
Moving to projects directory: /home/test/slots/36/ICMGGh7zg+000852
Moving to projects directory: /home/test/slots/36/ICMSHh7zg+000852
Moving to projects directory: /home/test/slots/36/ICMUAh7zg+000852
  10:33:23 STEP  853 H= 213:15 +CPU= 73.417
  10:34:28 STEP  854 H= 213:30 +CPU= 64.184
  10:35:36 STEP  855 H= 213:45 +CPU= 67.078
  10:36:46 STEP  856 H= 214:00 +CPU= 69.978
  10:37:50 STEP  857 H= 214:15 +CPU= 63.240
  10:39:05 STEP  858 H= 214:30 +CPU= 74.434
  10:40:13 STEP  859 H= 214:45 +CPU= 67.295
  10:41:34 STEP  860 H= 215:00 +CPU= 80.877
  10:42:42 STEP  861 H= 215:15 +CPU= 67.183
  10:43:56 STEP  862 H= 215:30 +CPU= 73.005
  10:45:12 STEP  863 H= 215:45 +CPU= 74.912
  10:46:44 STEP  864 H= 216:00 +CPU= 91.304
Moving to projects directory: /home/test/slots/36/ICMGGh7zg+000864
Moving to projects directory: /home/test/slots/36/ICMSHh7zg+000864
Moving to projects directory: /home/test/slots/36/ICMUAh7zg+000864
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMGGh7zg+000768
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMSHh7zg+000768
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMUAh7zg+000768
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMGGh7zg+000780
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMSHh7zg+000780
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMUAh7zg+000780
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMGGh7zg+000792
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMSHh7zg+000792
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMUAh7zg+000792
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMGGh7zg+000804
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMSHh7zg+000804
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMUAh7zg+000804
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMGGh7zg+000816
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMSHh7zg+000816
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMUAh7zg+000816
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMGGh7zg+000828
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMSHh7zg+000828
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMUAh7zg+000828
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMGGh7zg+000840
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMSHh7zg+000840
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMUAh7zg+000840
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMGGh7zg+000852
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMSHh7zg+000852
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMUAh7zg+000852
Zipping up the intermediate file: /home/test/projects/climateprediction.net/oifs_43r3_bl_a0iu_2016092300_20_1017_12282505_0_r1111415241_8.zip
Uploading the intermediate file: upload_file_8.zip
  10:47:52 STEP  865 H= 216:15 +CPU= 66.572
  10:49:02 STEP  866 H= 216:30 +CPU= 69.791
  10:50:09 STEP  867 H= 216:45 +CPU= 67.233
  10:51:10 STEP  868 H= 217:00 +CPU= 60.213
  10:52:04 STEP  869 H= 217:15 +CPU= 54.744
  10:53:09 STEP  870 H= 217:30 +CPU= 64.101
  10:54:19 STEP  871 H= 217:45 +CPU= 70.409
  10:55:30 STEP  872 H= 218:00 +CPU= 70.540
  10:56:42 STEP  873 H= 218:15 +CPU= 70.165
  10:57:51 STEP  874 H= 218:30 +CPU= 68.195
  10:58:57 STEP  875 H= 218:45 +CPU= 65.408
  11:00:17 STEP  876 H= 219:00 +CPU= 79.753
Moving to projects directory: /home/test/slots/36/ICMGGh7zg+000876
Moving to projects directory: /home/test/slots/36/ICMSHh7zg+000876
Moving to projects directory: /home/test/slots/36/ICMUAh7zg+000876
  11:01:35 STEP  877 H= 219:15 +CPU= 77.160
  11:02:39 STEP  878 H= 219:30 +CPU= 64.200
  11:03:45 STEP  879 H= 219:45 +CPU= 64.665
  11:04:58 STEP  880 H= 220:00 +CPU= 72.571
  11:06:12 STEP  881 H= 220:15 +CPU= 74.266
  11:07:20 STEP  882 H= 220:30 +CPU= 67.156
  11:08:26 STEP  883 H= 220:45 +CPU= 65.941
  11:09:34 STEP  884 H= 221:00 +CPU= 67.750
  11:10:37 STEP  885 H= 221:15 +CPU= 62.675
  11:11:41 STEP  886 H= 221:30 +CPU= 63.453
  11:12:42 STEP  887 H= 221:45 +CPU= 60.766
  11:13:48 STEP  888 H= 222:00 +CPU= 66.306
Moving to projects directory: /home/test/slots/36/ICMGGh7zg+000888
Moving to projects directory: /home/test/slots/36/ICMSHh7zg+000888
Moving to projects directory: /home/test/slots/36/ICMUAh7zg+000888
  11:15:00 STEP  889 H= 222:15 +CPU= 71.453
  11:16:02 STEP  890 H= 222:30 +CPU= 62.060
  11:17:18 STEP  891 H= 222:45 +CPU= 75.758
  11:18:28 STEP  892 H= 223:00 +CPU= 69.584
  11:19:36 STEP  893 H= 223:15 +CPU= 67.695
  11:20:42 STEP  894 H= 223:30 +CPU= 65.819
  11:21:52 STEP  895 H= 223:45 +CPU= 69.265
  11:22:58 STEP  896 H= 224:00 +CPU= 66.292
  11:24:09 STEP  897 H= 224:15 +CPU= 69.926
  11:25:16 STEP  898 H= 224:30 +CPU= 66.204
  11:26:26 STEP  899 H= 224:45 +CPU= 70.173
  11:27:56 STEP  900 H= 225:00 +CPU= 89.151
Moving to projects directory: /home/test/slots/36/ICMGGh7zg+000900
Moving to projects directory: /home/test/slots/36/ICMSHh7zg+000900
Moving to projects directory: /home/test/slots/36/ICMUAh7zg+000900
  11:28:57 STEP  901 H= 225:15 +CPU= 60.932
  11:30:08 STEP  902 H= 225:30 +CPU= 70.569
  11:31:12 STEP  903 H= 225:45 +CPU= 63.355
  11:32:21 STEP  904 H= 226:00 +CPU= 67.875
  11:33:20 STEP  905 H= 226:15 +CPU= 58.762
  11:34:34 STEP  906 H= 226:30 +CPU= 73.564
  11:35:49 STEP  907 H= 226:45 +CPU= 74.557
  11:37:00 STEP  908 H= 227:00 +CPU= 70.245
  11:38:12 STEP  909 H= 227:15 +CPU= 71.377
  11:39:23 STEP  910 H= 227:30 +CPU= 71.036
  11:40:32 STEP  911 H= 227:45 +CPU= 67.786
  11:42:06 STEP  912 H= 228:00 +CPU= 93.263
Moving to projects directory: /home/test/slots/36/ICMGGh7zg+000912
Moving to projects directory: /home/test/slots/36/ICMSHh7zg+000912
Moving to projects directory: /home/test/slots/36/ICMUAh7zg+000912
  11:43:12 STEP  913 H= 228:15 +CPU= 65.653
  11:44:16 STEP  914 H= 228:30 +CPU= 63.935
  11:45:24 STEP  915 H= 228:45 +CPU= 66.415
  11:46:28 STEP  916 H= 229:00 +CPU= 64.378
  11:47:36 STEP  917 H= 229:15 +CPU= 67.060
  11:48:45 STEP  918 H= 229:30 +CPU= 68.302
  11:49:53 STEP  919 H= 229:45 +CPU= 68.055
  11:50:59 STEP  920 H= 230:00 +CPU= 65.578
  11:52:14 STEP  921 H= 230:15 +CPU= 73.829
  11:53:16 STEP  922 H= 230:30 +CPU= 62.444
  11:54:24 STEP  923 H= 230:45 +CPU= 68.164
  11:55:47 STEP  924 H= 231:00 +CPU= 81.429
Moving to projects directory: /home/test/slots/36/ICMGGh7zg+000924
Moving to projects directory: /home/test/slots/36/ICMSHh7zg+000924
Moving to projects directory: /home/test/slots/36/ICMUAh7zg+000924
  11:56:55 STEP  925 H= 231:15 +CPU= 67.684
  11:58:00 STEP  926 H= 231:30 +CPU= 64.788
  11:59:07 STEP  927 H= 231:45 +CPU= 66.794
  12:00:14 STEP  928 H= 232:00 +CPU= 66.376
  12:01:21 STEP  929 H= 232:15 +CPU= 66.284
  12:02:21 STEP  930 H= 232:30 +CPU= 60.153
  12:03:26 STEP  931 H= 232:45 +CPU= 64.807
  12:04:32 STEP  932 H= 233:00 +CPU= 64.802
  12:05:45 STEP  933 H= 233:15 +CPU= 72.419
  12:06:45 STEP  934 H= 233:30 +CPU= 59.264
  12:07:56 STEP  935 H= 233:45 +CPU= 71.148
  12:09:27 STEP  936 H= 234:00 +CPU= 90.140
Moving to projects directory: /home/test/slots/36/ICMGGh7zg+000936
Moving to projects directory: /home/test/slots/36/ICMSHh7zg+000936
Moving to projects directory: /home/test/slots/36/ICMUAh7zg+000936
  12:10:35 STEP  937 H= 234:15 +CPU= 68.152
  12:11:39 STEP  938 H= 234:30 +CPU= 63.676
  12:12:49 STEP  939 H= 234:45 +CPU= 69.463
  12:14:00 STEP  940 H= 235:00 +CPU= 70.072
  12:15:02 STEP  941 H= 235:15 +CPU= 62.337
  12:16:05 STEP  942 H= 235:30 +CPU= 62.633
  12:17:17 STEP  943 H= 235:45 +CPU= 70.987
  12:18:35 STEP  944 H= 236:00 +CPU= 77.931
  12:19:47 STEP  945 H= 236:15 +CPU= 71.530
  12:20:51 STEP  946 H= 236:30 +CPU= 62.740
  12:21:56 STEP  947 H= 236:45 +CPU= 64.690
  12:23:10 STEP  948 H= 237:00 +CPU= 74.281
Moving to projects directory: /home/test/slots/36/ICMGGh7zg+000948
Moving to projects directory: /home/test/slots/36/ICMSHh7zg+000948
Moving to projects directory: /home/test/slots/36/ICMUAh7zg+000948
  12:24:17 STEP  949 H= 237:15 +CPU= 66.311
  12:25:21 STEP  950 H= 237:30 +CPU= 63.784
  12:26:29 STEP  951 H= 237:45 +CPU= 66.765
  12:27:39 STEP  952 H= 238:00 +CPU= 69.611
  12:28:41 STEP  953 H= 238:15 +CPU= 61.652
  12:29:44 STEP  954 H= 238:30 +CPU= 62.547
  12:30:55 STEP  955 H= 238:45 +CPU= 70.807
  12:32:06 STEP  956 H= 239:00 +CPU= 70.945
  12:33:14 STEP  957 H= 239:15 +CPU= 67.637
  12:34:19 STEP  958 H= 239:30 +CPU= 64.195
  12:35:29 STEP  959 H= 239:45 +CPU= 69.205
  12:36:36 STEP  960 H= 240:00 +CPU= 67.208
Moving to projects directory: /home/test/slots/36/ICMGGh7zg+000960
Moving to projects directory: /home/test/slots/36/ICMSHh7zg+000960
Moving to projects directory: /home/test/slots/36/ICMUAh7zg+000960
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMGGh7zg+000864
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMSHh7zg+000864
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMUAh7zg+000864
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMGGh7zg+000876
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMSHh7zg+000876
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMUAh7zg+000876
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMGGh7zg+000888
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMSHh7zg+000888
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMUAh7zg+000888
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMGGh7zg+000900
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMSHh7zg+000900
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMUAh7zg+000900
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMGGh7zg+000912
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMSHh7zg+000912
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMUAh7zg+000912
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMGGh7zg+000924
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMSHh7zg+000924
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMUAh7zg+000924
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMGGh7zg+000936
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMSHh7zg+000936
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMUAh7zg+000936
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMGGh7zg+000948
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMSHh7zg+000948
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMUAh7zg+000948
Zipping up the intermediate file: /home/test/projects/climateprediction.net/oifs_43r3_bl_a0iu_2016092300_20_1017_12282505_0_r1111415241_9.zip
Uploading the intermediate file: upload_file_9.zip
  12:37:10 STEP  961 H= 240:15 +CPU= 33.194
  12:37:51 STEP  962 H= 240:30 +CPU= 40.783
  12:38:29 STEP  963 H= 240:45 +CPU= 37.400
  12:39:07 STEP  964 H= 241:00 +CPU= 37.799
  12:39:45 STEP  965 H= 241:15 +CPU= 37.590
  12:40:23 STEP  966 H= 241:30 +CPU= 38.189
  12:40:58 STEP  967 H= 241:45 +CPU= 34.543
  12:41:36 STEP  968 H= 242:00 +CPU= 37.352
  12:42:12 STEP  969 H= 242:15 +CPU= 35.867
  12:42:50 STEP  970 H= 242:30 +CPU= 37.712
  12:43:26 STEP  971 H= 242:45 +CPU= 35.926
  12:44:11 STEP  972 H= 243:00 +CPU= 45.447
Moving to projects directory: /home/test/slots/36/ICMGGh7zg+000972
Moving to projects directory: /home/test/slots/36/ICMSHh7zg+000972
Moving to projects directory: /home/test/slots/36/ICMUAh7zg+000972
  12:44:47 STEP  973 H= 243:15 +CPU= 35.576
  12:45:24 STEP  974 H= 243:30 +CPU= 36.905
  12:46:00 STEP  975 H= 243:45 +CPU= 35.539
  12:46:38 STEP  976 H= 244:00 +CPU= 37.818
  12:47:18 STEP  977 H= 244:15 +CPU= 40.001
  12:47:55 STEP  978 H= 244:30 +CPU= 36.749
  12:48:31 STEP  979 H= 244:45 +CPU= 36.324
  12:49:08 STEP  980 H= 245:00 +CPU= 36.122
  12:49:43 STEP  981 H= 245:15 +CPU= 35.282
  12:50:20 STEP  982 H= 245:30 +CPU= 36.710
  12:50:59 STEP  983 H= 245:45 +CPU= 38.783
  12:51:43 STEP  984 H= 246:00 +CPU= 44.164
Moving to projects directory: /home/test/slots/36/ICMGGh7zg+000984
Moving to projects directory: /home/test/slots/36/ICMSHh7zg+000984
Moving to projects directory: /home/test/slots/36/ICMUAh7zg+000984
  12:52:21 STEP  985 H= 246:15 +CPU= 37.843
  12:53:00 STEP  986 H= 246:30 +CPU= 38.611
  12:53:38 STEP  987 H= 246:45 +CPU= 37.521
  12:54:16 STEP  988 H= 247:00 +CPU= 37.647
  12:54:53 STEP  989 H= 247:15 +CPU= 36.671
  12:55:30 STEP  990 H= 247:30 +CPU= 37.074
  12:56:08 STEP  991 H= 247:45 +CPU= 37.928
  12:56:49 STEP  992 H= 248:00 +CPU= 40.455
  12:57:30 STEP  993 H= 248:15 +CPU= 41.486
  12:58:08 STEP  994 H= 248:30 +CPU= 37.777
  12:58:43 STEP  995 H= 248:45 +CPU= 34.857
  12:59:26 STEP  996 H= 249:00 +CPU= 42.626
Moving to projects directory: /home/test/slots/36/ICMGGh7zg+000996
Moving to projects directory: /home/test/slots/36/ICMSHh7zg+000996
Moving to projects directory: /home/test/slots/36/ICMUAh7zg+000996
  13:00:02 STEP  997 H= 249:15 +CPU= 35.080
  13:00:39 STEP  998 H= 249:30 +CPU= 37.102
  13:01:17 STEP  999 H= 249:45 +CPU= 38.147
  13:01:57 STEP 1000 H= 250:00 +CPU= 39.270
  13:02:33 STEP 1001 H= 250:15 +CPU= 35.601
  13:03:06 STEP 1002 H= 250:30 +CPU= 32.945
  13:03:41 STEP 1003 H= 250:45 +CPU= 35.457
  13:04:19 STEP 1004 H= 251:00 +CPU= 37.495
  13:04:55 STEP 1005 H= 251:15 +CPU= 35.936
  13:05:32 STEP 1006 H= 251:30 +CPU= 36.750
  13:06:10 STEP 1007 H= 251:45 +CPU= 36.533
  13:06:59 STEP 1008 H= 252:00 +CPU= 48.587
Moving to projects directory: /home/test/slots/36/ICMGGh7zg+001008
Moving to projects directory: /home/test/slots/36/ICMSHh7zg+001008
Moving to projects directory: /home/test/slots/36/ICMUAh7zg+001008
  13:07:35 STEP 1009 H= 252:15 +CPU= 35.466
  13:08:10 STEP 1010 H= 252:30 +CPU= 35.146
  13:08:48 STEP 1011 H= 252:45 +CPU= 37.016
  13:09:24 STEP 1012 H= 253:00 +CPU= 35.803
  13:10:05 STEP 1013 H= 253:15 +CPU= 40.474
  13:10:38 STEP 1014 H= 253:30 +CPU= 32.893
  13:11:14 STEP 1015 H= 253:45 +CPU= 35.704
  13:11:50 STEP 1016 H= 254:00 +CPU= 35.956
  13:12:27 STEP 1017 H= 254:15 +CPU= 36.471
  13:13:05 STEP 1018 H= 254:30 +CPU= 37.753
  13:13:43 STEP 1019 H= 254:45 +CPU= 38.181
  13:14:25 STEP 1020 H= 255:00 +CPU= 41.984
Moving to projects directory: /home/test/slots/36/ICMGGh7zg+001020
Moving to projects directory: /home/test/slots/36/ICMSHh7zg+001020
Moving to projects directory: /home/test/slots/36/ICMUAh7zg+001020
  13:15:01 STEP 1021 H= 255:15 +CPU= 36.293
  13:15:35 STEP 1022 H= 255:30 +CPU= 33.352
  13:16:09 STEP 1023 H= 255:45 +CPU= 34.263
  13:16:45 STEP 1024 H= 256:00 +CPU= 35.610
  13:17:22 STEP 1025 H= 256:15 +CPU= 36.538
  13:18:00 STEP 1026 H= 256:30 +CPU= 38.847
  13:18:38 STEP 1027 H= 256:45 +CPU= 37.527
  13:19:14 STEP 1028 H= 257:00 +CPU= 35.957
  13:19:46 STEP 1029 H= 257:15 +CPU= 32.420
  13:20:25 STEP 1030 H= 257:30 +CPU= 38.767
  13:21:01 STEP 1031 H= 257:45 +CPU= 35.394
  13:21:46 STEP 1032 H= 258:00 +CPU= 44.654
Moving to projects directory: /home/test/slots/36/ICMGGh7zg+001032
Moving to projects directory: /home/test/slots/36/ICMSHh7zg+001032
Moving to projects directory: /home/test/slots/36/ICMUAh7zg+001032
  13:22:25 STEP 1033 H= 258:15 +CPU= 38.988
  13:23:02 STEP 1034 H= 258:30 +CPU= 36.676
  13:23:35 STEP 1035 H= 258:45 +CPU= 32.963
  13:24:11 STEP 1036 H= 259:00 +CPU= 35.623
  13:24:45 STEP 1037 H= 259:15 +CPU= 33.046
  13:25:21 STEP 1038 H= 259:30 +CPU= 36.131
  13:25:57 STEP 1039 H= 259:45 +CPU= 36.280
  13:26:34 STEP 1040 H= 260:00 +CPU= 36.963
  13:27:14 STEP 1041 H= 260:15 +CPU= 38.934
  13:27:50 STEP 1042 H= 260:30 +CPU= 35.986
  13:28:23 STEP 1043 H= 260:45 +CPU= 33.466
  13:29:08 STEP 1044 H= 261:00 +CPU= 44.526
Moving to projects directory: /home/test/slots/36/ICMGGh7zg+001044
Moving to projects directory: /home/test/slots/36/ICMSHh7zg+001044
Moving to projects directory: /home/test/slots/36/ICMUAh7zg+001044
  13:29:42 STEP 1045 H= 261:15 +CPU= 34.103
  13:30:17 STEP 1046 H= 261:30 +CPU= 34.724
  13:30:49 STEP 1047 H= 261:45 +CPU= 31.136
  13:31:23 STEP 1048 H= 262:00 +CPU= 34.413
  13:31:56 STEP 1049 H= 262:15 +CPU= 32.317
  13:32:29 STEP 1050 H= 262:30 +CPU= 33.286
  13:33:05 STEP 1051 H= 262:45 +CPU= 36.275
  13:33:43 STEP 1052 H= 263:00 +CPU= 37.323
  13:34:19 STEP 1053 H= 263:15 +CPU= 35.355
  13:34:56 STEP 1054 H= 263:30 +CPU= 36.871
  13:35:34 STEP 1055 H= 263:45 +CPU= 37.922
  13:36:23 STEP 1056 H= 264:00 +CPU= 48.148
Moving to projects directory: /home/test/slots/36/ICMGGh7zg+001056
Moving to projects directory: /home/test/slots/36/ICMSHh7zg+001056
Moving to projects directory: /home/test/slots/36/ICMUAh7zg+001056
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMGGh7zg+000960
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMSHh7zg+000960
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMUAh7zg+000960
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMGGh7zg+000972
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMSHh7zg+000972
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMUAh7zg+000972
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMGGh7zg+000984
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMSHh7zg+000984
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMUAh7zg+000984
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMGGh7zg+000996
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMSHh7zg+000996
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMUAh7zg+000996
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMGGh7zg+001008
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMSHh7zg+001008
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMUAh7zg+001008
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMGGh7zg+001020
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMSHh7zg+001020
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMUAh7zg+001020
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMGGh7zg+001032
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMSHh7zg+001032
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMUAh7zg+001032
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMGGh7zg+001044
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMSHh7zg+001044
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMUAh7zg+001044
Zipping up the intermediate file: /home/test/projects/climateprediction.net/oifs_43r3_bl_a0iu_2016092300_20_1017_12282505_0_r1111415241_10.zip
Uploading the intermediate file: upload_file_10.zip
  13:36:58 STEP 1057 H= 264:15 +CPU= 34.817
  13:37:35 STEP 1058 H= 264:30 +CPU= 36.469
  13:38:11 STEP 1059 H= 264:45 +CPU= 35.537
  13:38:48 STEP 1060 H= 265:00 +CPU= 36.782
  13:39:26 STEP 1061 H= 265:15 +CPU= 36.988
  13:40:01 STEP 1062 H= 265:30 +CPU= 34.967
  13:40:36 STEP 1063 H= 265:45 +CPU= 34.831
  13:41:11 STEP 1064 H= 266:00 +CPU= 35.009
  13:41:49 STEP 1065 H= 266:15 +CPU= 37.278
  13:42:25 STEP 1066 H= 266:30 +CPU= 36.058
  13:43:03 STEP 1067 H= 266:45 +CPU= 38.007
  13:43:47 STEP 1068 H= 267:00 +CPU= 43.615
Moving to projects directory: /home/test/slots/36/ICMGGh7zg+001068
Moving to projects directory: /home/test/slots/36/ICMSHh7zg+001068
Moving to projects directory: /home/test/slots/36/ICMUAh7zg+001068
  13:44:25 STEP 1069 H= 267:15 +CPU= 37.471
  13:45:02 STEP 1070 H= 267:30 +CPU= 36.756
  13:45:36 STEP 1071 H= 267:45 +CPU= 33.864
  13:46:13 STEP 1072 H= 268:00 +CPU= 35.863
  13:46:51 STEP 1073 H= 268:15 +CPU= 38.462
  13:47:25 STEP 1074 H= 268:30 +CPU= 34.043
  13:48:04 STEP 1075 H= 268:45 +CPU= 38.300
  13:48:41 STEP 1076 H= 269:00 +CPU= 36.878
  13:49:22 STEP 1077 H= 269:15 +CPU= 40.836
  13:50:01 STEP 1078 H= 269:30 +CPU= 39.379
  13:50:36 STEP 1079 H= 269:45 +CPU= 35.047
  13:51:20 STEP 1080 H= 270:00 +CPU= 43.558
Moving to projects directory: /home/test/slots/36/ICMGGh7zg+001080
Moving to projects directory: /home/test/slots/36/ICMSHh7zg+001080
Moving to projects directory: /home/test/slots/36/ICMUAh7zg+001080
  13:51:55 STEP 1081 H= 270:15 +CPU= 33.993
  13:52:31 STEP 1082 H= 270:30 +CPU= 35.848
  13:53:06 STEP 1083 H= 270:45 +CPU= 34.979
  13:53:42 STEP 1084 H= 271:00 +CPU= 36.019
  13:54:21 STEP 1085 H= 271:15 +CPU= 38.194
  13:55:03 STEP 1086 H= 271:30 +CPU= 41.601
  13:55:42 STEP 1087 H= 271:45 +CPU= 38.529
  13:56:16 STEP 1088 H= 272:00 +CPU= 34.179
  13:56:53 STEP 1089 H= 272:15 +CPU= 37.410
  13:57:33 STEP 1090 H= 272:30 +CPU= 39.544
  13:58:12 STEP 1091 H= 272:45 +CPU= 39.027
  13:58:51 STEP 1092 H= 273:00 +CPU= 38.576
Moving to projects directory: /home/test/slots/36/ICMGGh7zg+001092
Moving to projects directory: /home/test/slots/36/ICMSHh7zg+001092
Moving to projects directory: /home/test/slots/36/ICMUAh7zg+001092
  13:59:25 STEP 1093 H= 273:15 +CPU= 33.837
  14:00:00 STEP 1094 H= 273:30 +CPU= 34.340
  14:00:37 STEP 1095 H= 273:45 +CPU= 36.728
  14:01:12 STEP 1096 H= 274:00 +CPU= 35.127
  14:01:50 STEP 1097 H= 274:15 +CPU= 38.031
  14:02:30 STEP 1098 H= 274:30 +CPU= 39.180
  14:03:07 STEP 1099 H= 274:45 +CPU= 36.964
  14:03:42 STEP 1100 H= 275:00 +CPU= 35.530
  14:04:16 STEP 1101 H= 275:15 +CPU= 33.169
  14:04:47 STEP 1102 H= 275:30 +CPU= 30.961
  14:05:18 STEP 1103 H= 275:45 +CPU= 30.503
  14:05:59 STEP 1104 H= 276:00 +CPU= 40.812
Moving to projects directory: /home/test/slots/36/ICMGGh7zg+001104
Moving to projects directory: /home/test/slots/36/ICMSHh7zg+001104
Moving to projects directory: /home/test/slots/36/ICMUAh7zg+001104
  14:06:32 STEP 1105 H= 276:15 +CPU= 33.751
  14:07:01 STEP 1106 H= 276:30 +CPU= 28.569
  14:07:32 STEP 1107 H= 276:45 +CPU= 30.824
  14:08:02 STEP 1108 H= 277:00 +CPU= 30.343
  14:08:34 STEP 1109 H= 277:15 +CPU= 31.347
  14:09:07 STEP 1110 H= 277:30 +CPU= 33.423
  14:09:42 STEP 1111 H= 277:45 +CPU= 34.636
  14:10:19 STEP 1112 H= 278:00 +CPU= 36.068
  14:10:58 STEP 1113 H= 278:15 +CPU= 39.070
  14:11:40 STEP 1114 H= 278:30 +CPU= 42.333
  14:12:17 STEP 1115 H= 278:45 +CPU= 36.867
  14:12:59 STEP 1116 H= 279:00 +CPU= 41.952
Moving to projects directory: /home/test/slots/36/ICMGGh7zg+001116
Moving to projects directory: /home/test/slots/36/ICMSHh7zg+001116
Moving to projects directory: /home/test/slots/36/ICMUAh7zg+001116
  14:13:35 STEP 1117 H= 279:15 +CPU= 35.283
  14:14:10 STEP 1118 H= 279:30 +CPU= 34.852
  14:14:46 STEP 1119 H= 279:45 +CPU= 35.789
  14:15:23 STEP 1120 H= 280:00 +CPU= 36.628
  14:15:57 STEP 1121 H= 280:15 +CPU= 34.078
  14:16:35 STEP 1122 H= 280:30 +CPU= 37.491
  14:17:09 STEP 1123 H= 280:45 +CPU= 34.713
  14:17:47 STEP 1124 H= 281:00 +CPU= 37.796
  14:18:21 STEP 1125 H= 281:15 +CPU= 33.228
  14:18:55 STEP 1126 H= 281:30 +CPU= 34.785
  14:19:31 STEP 1127 H= 281:45 +CPU= 34.828
  14:20:13 STEP 1128 H= 282:00 +CPU= 42.546
Moving to projects directory: /home/test/slots/36/ICMGGh7zg+001128
Moving to projects directory: /home/test/slots/36/ICMSHh7zg+001128
Moving to projects directory: /home/test/slots/36/ICMUAh7zg+001128
  14:20:50 STEP 1129 H= 282:15 +CPU= 36.849
  14:21:25 STEP 1130 H= 282:30 +CPU= 34.008
  14:21:58 STEP 1131 H= 282:45 +CPU= 33.806
  14:22:33 STEP 1132 H= 283:00 +CPU= 34.129
  14:23:08 STEP 1133 H= 283:15 +CPU= 34.488
  14:23:41 STEP 1134 H= 283:30 +CPU= 33.267
  14:24:20 STEP 1135 H= 283:45 +CPU= 38.498
  14:24:57 STEP 1136 H= 284:00 +CPU= 37.106
  14:25:37 STEP 1137 H= 284:15 +CPU= 39.474
  14:26:11 STEP 1138 H= 284:30 +CPU= 34.315
  14:26:43 STEP 1139 H= 284:45 +CPU= 31.761
  14:27:26 STEP 1140 H= 285:00 +CPU= 42.294
Moving to projects directory: /home/test/slots/36/ICMGGh7zg+001140
Moving to projects directory: /home/test/slots/36/ICMSHh7zg+001140
Moving to projects directory: /home/test/slots/36/ICMUAh7zg+001140
  14:27:59 STEP 1141 H= 285:15 +CPU= 32.801
  14:28:35 STEP 1142 H= 285:30 +CPU= 35.621
  14:29:12 STEP 1143 H= 285:45 +CPU= 36.582
  14:29:49 STEP 1144 H= 286:00 +CPU= 37.038
  14:30:26 STEP 1145 H= 286:15 +CPU= 36.484
  14:31:01 STEP 1146 H= 286:30 +CPU= 34.864
  14:31:37 STEP 1147 H= 286:45 +CPU= 35.166
  14:32:12 STEP 1148 H= 287:00 +CPU= 34.723
  14:32:48 STEP 1149 H= 287:15 +CPU= 36.332
  14:33:23 STEP 1150 H= 287:30 +CPU= 34.693
  14:33:59 STEP 1151 H= 287:45 +CPU= 35.484
  14:34:51 STEP 1152 H= 288:00 +CPU= 51.028
Moving to projects directory: /home/test/slots/36/ICMGGh7zg+001152
Moving to projects directory: /home/test/slots/36/ICMSHh7zg+001152
Moving to projects directory: /home/test/slots/36/ICMUAh7zg+001152
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMGGh7zg+001056
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMSHh7zg+001056
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMUAh7zg+001056
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMGGh7zg+001068
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMSHh7zg+001068
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMUAh7zg+001068
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMGGh7zg+001080
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMSHh7zg+001080
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMUAh7zg+001080
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMGGh7zg+001092
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMSHh7zg+001092
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMUAh7zg+001092
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMGGh7zg+001104
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMSHh7zg+001104
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMUAh7zg+001104
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMGGh7zg+001116
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMSHh7zg+001116
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMUAh7zg+001116
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMGGh7zg+001128
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMSHh7zg+001128
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMUAh7zg+001128
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMGGh7zg+001140
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMSHh7zg+001140
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMUAh7zg+001140
Zipping up the intermediate file: /home/test/projects/climateprediction.net/oifs_43r3_bl_a0iu_2016092300_20_1017_12282505_0_r1111415241_11.zip
Uploading the intermediate file: upload_file_11.zip
  14:35:24 STEP 1153 H= 288:15 +CPU= 32.946
  14:35:59 STEP 1154 H= 288:30 +CPU= 34.143
  14:36:32 STEP 1155 H= 288:45 +CPU= 32.398
  14:37:09 STEP 1156 H= 289:00 +CPU= 37.091
  14:37:47 STEP 1157 H= 289:15 +CPU= 37.781
  14:38:23 STEP 1158 H= 289:30 +CPU= 35.393
  14:39:01 STEP 1159 H= 289:45 +CPU= 37.333
  14:39:37 STEP 1160 H= 290:00 +CPU= 35.579
  14:40:12 STEP 1161 H= 290:15 +CPU= 34.874
  14:40:45 STEP 1162 H= 290:30 +CPU= 33.158
  14:41:18 STEP 1163 H= 290:45 +CPU= 32.939
  14:42:02 STEP 1164 H= 291:00 +CPU= 42.918
Moving to projects directory: /home/test/slots/36/ICMGGh7zg+001164
Moving to projects directory: /home/test/slots/36/ICMSHh7zg+001164
Moving to projects directory: /home/test/slots/36/ICMUAh7zg+001164
  14:42:36 STEP 1165 H= 291:15 +CPU= 34.526
  14:43:12 STEP 1166 H= 291:30 +CPU= 36.030
  14:43:44 STEP 1167 H= 291:45 +CPU= 31.408
  14:44:20 STEP 1168 H= 292:00 +CPU= 35.342
  14:44:54 STEP 1169 H= 292:15 +CPU= 33.915
  14:45:27 STEP 1170 H= 292:30 +CPU= 33.675
  14:46:03 STEP 1171 H= 292:45 +CPU= 35.173
  14:46:43 STEP 1172 H= 293:00 +CPU= 40.275
  14:47:23 STEP 1173 H= 293:15 +CPU= 40.006
  14:47:48 STEP 1174 H= 293:30 +CPU= 24.251
  14:48:12 STEP 1175 H= 293:45 +CPU= 24.172
  14:48:43 STEP 1176 H= 294:00 +CPU= 30.641
Moving to projects directory: /home/test/slots/36/ICMGGh7zg+001176
Moving to projects directory: /home/test/slots/36/ICMSHh7zg+001176
Moving to projects directory: /home/test/slots/36/ICMUAh7zg+001176
  14:49:08 STEP 1177 H= 294:15 +CPU= 24.665
  14:49:33 STEP 1178 H= 294:30 +CPU= 25.495
  14:49:58 STEP 1179 H= 294:45 +CPU= 24.411
  14:50:25 STEP 1180 H= 295:00 +CPU= 27.517
  14:50:51 STEP 1181 H= 295:15 +CPU= 25.584
  14:51:15 STEP 1182 H= 295:30 +CPU= 24.253
  14:51:39 STEP 1183 H= 295:45 +CPU= 23.973
  14:52:05 STEP 1184 H= 296:00 +CPU= 25.459
  14:52:31 STEP 1185 H= 296:15 +CPU= 25.949
  14:52:55 STEP 1186 H= 296:30 +CPU= 24.113
  14:53:20 STEP 1187 H= 296:45 +CPU= 25.656
  14:53:52 STEP 1188 H= 297:00 +CPU= 31.641
Moving to projects directory: /home/test/slots/36/ICMGGh7zg+001188
Moving to projects directory: /home/test/slots/36/ICMSHh7zg+001188
Moving to projects directory: /home/test/slots/36/ICMUAh7zg+001188
  14:54:17 STEP 1189 H= 297:15 +CPU= 24.778
  14:54:43 STEP 1190 H= 297:30 +CPU= 26.097
  14:55:09 STEP 1191 H= 297:45 +CPU= 25.647
  14:55:34 STEP 1192 H= 298:00 +CPU= 24.289
  14:55:58 STEP 1193 H= 298:15 +CPU= 24.177
  14:56:21 STEP 1194 H= 298:30 +CPU= 23.320
  14:56:45 STEP 1195 H= 298:45 +CPU= 23.297
  14:57:10 STEP 1196 H= 299:00 +CPU= 24.778
  14:57:36 STEP 1197 H= 299:15 +CPU= 25.973
  14:58:02 STEP 1198 H= 299:30 +CPU= 26.097
  14:58:27 STEP 1199 H= 299:45 +CPU= 25.332
  14:59:00 STEP 1200 H= 300:00 +CPU= 32.897
Moving to projects directory: /home/test/slots/36/ICMGGh7zg+001200
Moving to projects directory: /home/test/slots/36/ICMSHh7zg+001200
Moving to projects directory: /home/test/slots/36/ICMUAh7zg+001200
  14:59:25 STEP 1201 H= 300:15 +CPU= 24.648
  14:59:50 STEP 1202 H= 300:30 +CPU= 24.663
  15:00:14 STEP 1203 H= 300:45 +CPU= 24.214
  15:00:40 STEP 1204 H= 301:00 +CPU= 26.008
  15:01:07 STEP 1205 H= 301:15 +CPU= 25.567
  15:01:34 STEP 1206 H= 301:30 +CPU= 27.214
  15:02:00 STEP 1207 H= 301:45 +CPU= 25.421
  15:02:26 STEP 1208 H= 302:00 +CPU= 26.086
  15:02:51 STEP 1209 H= 302:15 +CPU= 25.039
  15:03:16 STEP 1210 H= 302:30 +CPU= 24.696
  15:03:43 STEP 1211 H= 302:45 +CPU= 26.401
  15:04:15 STEP 1212 H= 303:00 +CPU= 31.959
Moving to projects directory: /home/test/slots/36/ICMGGh7zg+001212
Moving to projects directory: /home/test/slots/36/ICMSHh7zg+001212
Moving to projects directory: /home/test/slots/36/ICMUAh7zg+001212
  15:04:41 STEP 1213 H= 303:15 +CPU= 25.772
  15:05:08 STEP 1214 H= 303:30 +CPU= 27.097
  15:05:34 STEP 1215 H= 303:45 +CPU= 25.774
  15:06:01 STEP 1216 H= 304:00 +CPU= 27.135
  15:06:28 STEP 1217 H= 304:15 +CPU= 25.883
  15:06:54 STEP 1218 H= 304:30 +CPU= 26.187
  15:07:20 STEP 1219 H= 304:45 +CPU= 25.365
  15:07:46 STEP 1220 H= 305:00 +CPU= 26.111
  15:08:12 STEP 1221 H= 305:15 +CPU= 25.876
  15:08:37 STEP 1222 H= 305:30 +CPU= 24.181
  15:09:02 STEP 1223 H= 305:45 +CPU= 25.223
  15:09:35 STEP 1224 H= 306:00 +CPU= 33.134
Moving to projects directory: /home/test/slots/36/ICMGGh7zg+001224
Moving to projects directory: /home/test/slots/36/ICMSHh7zg+001224
Moving to projects directory: /home/test/slots/36/ICMUAh7zg+001224
  15:10:00 STEP 1225 H= 306:15 +CPU= 24.666
  15:10:26 STEP 1226 H= 306:30 +CPU= 26.121
  15:10:52 STEP 1227 H= 306:45 +CPU= 25.012
  15:11:17 STEP 1228 H= 307:00 +CPU= 24.585
  15:11:41 STEP 1229 H= 307:15 +CPU= 23.782
  15:12:05 STEP 1230 H= 307:30 +CPU= 24.380
  15:12:30 STEP 1231 H= 307:45 +CPU= 24.495
  15:12:57 STEP 1232 H= 308:00 +CPU= 26.564
  15:13:21 STEP 1233 H= 308:15 +CPU= 24.434
  15:13:46 STEP 1234 H= 308:30 +CPU= 24.040
  15:14:11 STEP 1235 H= 308:45 +CPU= 25.654
  15:14:43 STEP 1236 H= 309:00 +CPU= 31.712
Moving to projects directory: /home/test/slots/36/ICMGGh7zg+001236
Moving to projects directory: /home/test/slots/36/ICMSHh7zg+001236
Moving to projects directory: /home/test/slots/36/ICMUAh7zg+001236
  15:15:10 STEP 1237 H= 309:15 +CPU= 26.330
  15:15:36 STEP 1238 H= 309:30 +CPU= 26.521
  15:16:02 STEP 1239 H= 309:45 +CPU= 25.257
  15:16:28 STEP 1240 H= 310:00 +CPU= 25.648
  15:16:52 STEP 1241 H= 310:15 +CPU= 24.483
  15:17:16 STEP 1242 H= 310:30 +CPU= 24.100
  15:17:39 STEP 1243 H= 310:45 +CPU= 22.618
  15:18:02 STEP 1244 H= 311:00 +CPU= 22.542
  15:18:25 STEP 1245 H= 311:15 +CPU= 23.544
  15:18:51 STEP 1246 H= 311:30 +CPU= 25.294
  15:19:17 STEP 1247 H= 311:45 +CPU= 26.261
  15:19:51 STEP 1248 H= 312:00 +CPU= 34.328
Moving to projects directory: /home/test/slots/36/ICMGGh7zg+001248
Moving to projects directory: /home/test/slots/36/ICMSHh7zg+001248
Moving to projects directory: /home/test/slots/36/ICMUAh7zg+001248
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMGGh7zg+001152
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMSHh7zg+001152
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMUAh7zg+001152
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMGGh7zg+001164
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMSHh7zg+001164
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMUAh7zg+001164
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMGGh7zg+001176
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMSHh7zg+001176
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMUAh7zg+001176
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMGGh7zg+001188
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMSHh7zg+001188
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMUAh7zg+001188
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMGGh7zg+001200
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMSHh7zg+001200
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMUAh7zg+001200
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMGGh7zg+001212
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMSHh7zg+001212
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMUAh7zg+001212
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMGGh7zg+001224
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMSHh7zg+001224
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMUAh7zg+001224
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMGGh7zg+001236
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMSHh7zg+001236
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMUAh7zg+001236
Zipping up the intermediate file: /home/test/projects/climateprediction.net/oifs_43r3_bl_a0iu_2016092300_20_1017_12282505_0_r1111415241_12.zip
Uploading the intermediate file: upload_file_12.zip
  15:20:17 STEP 1249 H= 312:15 +CPU= 25.758
  15:20:42 STEP 1250 H= 312:30 +CPU= 25.124
  15:21:05 STEP 1251 H= 312:45 +CPU= 22.828
  15:21:30 STEP 1252 H= 313:00 +CPU= 24.710
  15:21:55 STEP 1253 H= 313:15 +CPU= 24.967
  15:22:21 STEP 1254 H= 313:30 +CPU= 25.670
  15:22:46 STEP 1255 H= 313:45 +CPU= 24.676
  15:23:13 STEP 1256 H= 314:00 +CPU= 26.823
  15:23:39 STEP 1257 H= 314:15 +CPU= 25.812
  15:24:04 STEP 1258 H= 314:30 +CPU= 24.878
  15:24:29 STEP 1259 H= 314:45 +CPU= 24.479
  15:24:59 STEP 1260 H= 315:00 +CPU= 29.986
Moving to projects directory: /home/test/slots/36/ICMGGh7zg+001260
Moving to projects directory: /home/test/slots/36/ICMSHh7zg+001260
Moving to projects directory: /home/test/slots/36/ICMUAh7zg+001260
  15:25:22 STEP 1261 H= 315:15 +CPU= 23.111
  15:25:46 STEP 1262 H= 315:30 +CPU= 23.935
  15:26:11 STEP 1263 H= 315:45 +CPU= 24.665
  15:26:36 STEP 1264 H= 316:00 +CPU= 24.842
  15:27:01 STEP 1265 H= 316:15 +CPU= 24.813
  15:27:26 STEP 1266 H= 316:30 +CPU= 24.815
  15:27:51 STEP 1267 H= 316:45 +CPU= 24.902
  15:28:17 STEP 1268 H= 317:00 +CPU= 26.044
  15:28:43 STEP 1269 H= 317:15 +CPU= 25.697
  15:29:08 STEP 1270 H= 317:30 +CPU= 24.581
  15:29:34 STEP 1271 H= 317:45 +CPU= 25.991
  15:30:04 STEP 1272 H= 318:00 +CPU= 29.782
Moving to projects directory: /home/test/slots/36/ICMGGh7zg+001272
Moving to projects directory: /home/test/slots/36/ICMSHh7zg+001272
Moving to projects directory: /home/test/slots/36/ICMUAh7zg+001272
  15:30:29 STEP 1273 H= 318:15 +CPU= 25.395
  15:30:55 STEP 1274 H= 318:30 +CPU= 26.116
  15:31:20 STEP 1275 H= 318:45 +CPU= 24.504
  15:31:44 STEP 1276 H= 319:00 +CPU= 23.829
  15:32:09 STEP 1277 H= 319:15 +CPU= 24.975
  15:32:34 STEP 1278 H= 319:30 +CPU= 25.419
  15:33:00 STEP 1279 H= 319:45 +CPU= 25.887
  15:33:27 STEP 1280 H= 320:00 +CPU= 26.693
  15:33:52 STEP 1281 H= 320:15 +CPU= 24.379
  15:34:17 STEP 1282 H= 320:30 +CPU= 25.157
  15:34:43 STEP 1283 H= 320:45 +CPU= 26.285
  15:35:16 STEP 1284 H= 321:00 +CPU= 32.288
Moving to projects directory: /home/test/slots/36/ICMGGh7zg+001284
Moving to projects directory: /home/test/slots/36/ICMSHh7zg+001284
Moving to projects directory: /home/test/slots/36/ICMUAh7zg+001284
  15:35:41 STEP 1285 H= 321:15 +CPU= 24.973
  15:36:05 STEP 1286 H= 321:30 +CPU= 24.021
  15:36:29 STEP 1287 H= 321:45 +CPU= 24.376
  15:36:55 STEP 1288 H= 322:00 +CPU= 25.540
  15:37:22 STEP 1289 H= 322:15 +CPU= 26.295
  15:37:55 STEP 1290 H= 322:30 +CPU= 33.148
  15:38:41 STEP 1291 H= 322:45 +CPU= 45.746
  15:39:27 STEP 1292 H= 323:00 +CPU= 46.681
  15:40:16 STEP 1293 H= 323:15 +CPU= 48.986
  15:41:07 STEP 1294 H= 323:30 +CPU= 50.536
  15:41:52 STEP 1295 H= 323:45 +CPU= 44.734
  15:42:55 STEP 1296 H= 324:00 +CPU= 62.556
Moving to projects directory: /home/test/slots/36/ICMGGh7zg+001296
Moving to projects directory: /home/test/slots/36/ICMSHh7zg+001296
Moving to projects directory: /home/test/slots/36/ICMUAh7zg+001296
  15:43:44 STEP 1297 H= 324:15 +CPU= 48.794
  15:44:26 STEP 1298 H= 324:30 +CPU= 41.993
  15:45:08 STEP 1299 H= 324:45 +CPU= 42.263
  15:45:51 STEP 1300 H= 325:00 +CPU= 43.228
  15:46:37 STEP 1301 H= 325:15 +CPU= 45.541
  15:47:23 STEP 1302 H= 325:30 +CPU= 45.961
  15:48:16 STEP 1303 H= 325:45 +CPU= 52.186
  15:49:03 STEP 1304 H= 326:00 +CPU= 47.863
  15:49:45 STEP 1305 H= 326:15 +CPU= 41.123
  15:50:29 STEP 1306 H= 326:30 +CPU= 44.057
  15:51:23 STEP 1307 H= 326:45 +CPU= 54.401
  15:52:24 STEP 1308 H= 327:00 +CPU= 60.312
Moving to projects directory: /home/test/slots/36/ICMGGh7zg+001308
Moving to projects directory: /home/test/slots/36/ICMSHh7zg+001308
Moving to projects directory: /home/test/slots/36/ICMUAh7zg+001308
  15:53:11 STEP 1309 H= 327:15 +CPU= 47.346
  15:53:55 STEP 1310 H= 327:30 +CPU= 43.408
  15:54:40 STEP 1311 H= 327:45 +CPU= 45.756
  15:55:25 STEP 1312 H= 328:00 +CPU= 44.693
  15:56:11 STEP 1313 H= 328:15 +CPU= 45.674
  15:56:53 STEP 1314 H= 328:30 +CPU= 41.197
  15:57:35 STEP 1315 H= 328:45 +CPU= 41.902
  15:58:18 STEP 1316 H= 329:00 +CPU= 43.380
  15:58:56 STEP 1317 H= 329:15 +CPU= 37.381
  15:59:39 STEP 1318 H= 329:30 +CPU= 43.000
  16:00:21 STEP 1319 H= 329:45 +CPU= 41.505
  16:01:10 STEP 1320 H= 330:00 +CPU= 49.483
Moving to projects directory: /home/test/slots/36/ICMGGh7zg+001320
Moving to projects directory: /home/test/slots/36/ICMSHh7zg+001320
Moving to projects directory: /home/test/slots/36/ICMUAh7zg+001320
  16:01:51 STEP 1321 H= 330:15 +CPU= 40.910
  16:02:40 STEP 1322 H= 330:30 +CPU= 48.228
  16:03:25 STEP 1323 H= 330:45 +CPU= 45.514
  16:04:18 STEP 1324 H= 331:00 +CPU= 52.472
  16:05:01 STEP 1325 H= 331:15 +CPU= 43.382
  16:05:42 STEP 1326 H= 331:30 +CPU= 40.316
  16:06:26 STEP 1327 H= 331:45 +CPU= 43.710
  16:07:12 STEP 1328 H= 332:00 +CPU= 46.254
  16:07:57 STEP 1329 H= 332:15 +CPU= 44.333
  16:08:44 STEP 1330 H= 332:30 +CPU= 47.369
  16:09:26 STEP 1331 H= 332:45 +CPU= 41.996
  16:10:24 STEP 1332 H= 333:00 +CPU= 57.342
Moving to projects directory: /home/test/slots/36/ICMGGh7zg+001332
Moving to projects directory: /home/test/slots/36/ICMSHh7zg+001332
Moving to projects directory: /home/test/slots/36/ICMUAh7zg+001332
  16:11:10 STEP 1333 H= 333:15 +CPU= 46.090
  16:11:53 STEP 1334 H= 333:30 +CPU= 42.367
  16:12:31 STEP 1335 H= 333:45 +CPU= 38.364
  16:13:13 STEP 1336 H= 334:00 +CPU= 41.834
  16:13:56 STEP 1337 H= 334:15 +CPU= 43.010
  16:14:44 STEP 1338 H= 334:30 +CPU= 47.557
  16:15:33 STEP 1339 H= 334:45 +CPU= 48.705
  16:16:27 STEP 1340 H= 335:00 +CPU= 53.801
  16:17:10 STEP 1341 H= 335:15 +CPU= 42.980
  16:17:55 STEP 1342 H= 335:30 +CPU= 44.759
  16:18:44 STEP 1343 H= 335:45 +CPU= 48.565
  16:19:46 STEP 1344 H= 336:00 +CPU= 61.961
Moving to projects directory: /home/test/slots/36/ICMGGh7zg+001344
Moving to projects directory: /home/test/slots/36/ICMSHh7zg+001344
Moving to projects directory: /home/test/slots/36/ICMUAh7zg+001344
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMGGh7zg+001248
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMSHh7zg+001248
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMUAh7zg+001248
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMGGh7zg+001260
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMSHh7zg+001260
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMUAh7zg+001260
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMGGh7zg+001272
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMSHh7zg+001272
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMUAh7zg+001272
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMGGh7zg+001284
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMSHh7zg+001284
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMUAh7zg+001284
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMGGh7zg+001296
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMSHh7zg+001296
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMUAh7zg+001296
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMGGh7zg+001308
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMSHh7zg+001308
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMUAh7zg+001308
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMGGh7zg+001320
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMSHh7zg+001320
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMUAh7zg+001320
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMGGh7zg+001332
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMSHh7zg+001332
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMUAh7zg+001332
Zipping up the intermediate file: /home/test/projects/climateprediction.net/oifs_43r3_bl_a0iu_2016092300_20_1017_12282505_0_r1111415241_13.zip
Uploading the intermediate file: upload_file_13.zip
  16:20:30 STEP 1345 H= 336:15 +CPU= 43.315
  16:21:14 STEP 1346 H= 336:30 +CPU= 44.195
  16:21:55 STEP 1347 H= 336:45 +CPU= 40.258
  16:22:41 STEP 1348 H= 337:00 +CPU= 45.502
  16:23:22 STEP 1349 H= 337:15 +CPU= 41.169
  16:24:12 STEP 1350 H= 337:30 +CPU= 50.095
  16:24:53 STEP 1351 H= 337:45 +CPU= 40.482
  16:25:40 STEP 1352 H= 338:00 +CPU= 47.067
  16:26:33 STEP 1353 H= 338:15 +CPU= 53.167
  16:27:22 STEP 1354 H= 338:30 +CPU= 48.740
  16:28:03 STEP 1355 H= 338:45 +CPU= 40.411
  16:29:04 STEP 1356 H= 339:00 +CPU= 60.116
Moving to projects directory: /home/test/slots/36/ICMGGh7zg+001356
Moving to projects directory: /home/test/slots/36/ICMSHh7zg+001356
Moving to projects directory: /home/test/slots/36/ICMUAh7zg+001356
  16:29:54 STEP 1357 H= 339:15 +CPU= 50.725
  16:30:40 STEP 1358 H= 339:30 +CPU= 45.108
  16:31:22 STEP 1359 H= 339:45 +CPU= 42.314
  16:32:07 STEP 1360 H= 340:00 +CPU= 44.847
  16:32:52 STEP 1361 H= 340:15 +CPU= 45.000
  16:33:30 STEP 1362 H= 340:30 +CPU= 37.837
  16:34:19 STEP 1363 H= 340:45 +CPU= 48.962
  16:34:59 STEP 1364 H= 341:00 +CPU= 40.047
  16:35:33 STEP 1365 H= 341:15 +CPU= 34.102
  16:36:06 STEP 1366 H= 341:30 +CPU= 32.422
  16:36:37 STEP 1367 H= 341:45 +CPU= 31.503
  16:37:17 STEP 1368 H= 342:00 +CPU= 39.631
Moving to projects directory: /home/test/slots/36/ICMGGh7zg+001368
Moving to projects directory: /home/test/slots/36/ICMSHh7zg+001368
Moving to projects directory: /home/test/slots/36/ICMUAh7zg+001368
  16:37:49 STEP 1369 H= 342:15 +CPU= 31.815
  16:38:17 STEP 1370 H= 342:30 +CPU= 27.867
  16:38:42 STEP 1371 H= 342:45 +CPU= 24.912
  16:39:08 STEP 1372 H= 343:00 +CPU= 26.209
  16:39:33 STEP 1373 H= 343:15 +CPU= 25.491
  16:40:00 STEP 1374 H= 343:30 +CPU= 26.222
  16:40:27 STEP 1375 H= 343:45 +CPU= 27.490
  16:40:57 STEP 1376 H= 344:00 +CPU= 29.660
  16:41:24 STEP 1377 H= 344:15 +CPU= 27.241
  16:41:52 STEP 1378 H= 344:30 +CPU= 27.386
  16:42:18 STEP 1379 H= 344:45 +CPU= 26.570
  16:42:50 STEP 1380 H= 345:00 +CPU= 31.473
Moving to projects directory: /home/test/slots/36/ICMGGh7zg+001380
Moving to projects directory: /home/test/slots/36/ICMSHh7zg+001380
Moving to projects directory: /home/test/slots/36/ICMUAh7zg+001380
  16:43:17 STEP 1381 H= 345:15 +CPU= 27.094
  16:43:43 STEP 1382 H= 345:30 +CPU= 25.770
  16:44:07 STEP 1383 H= 345:45 +CPU= 24.199
  16:44:33 STEP 1384 H= 346:00 +CPU= 25.496
  16:44:58 STEP 1385 H= 346:15 +CPU= 24.693
  16:45:24 STEP 1386 H= 346:30 +CPU= 25.641
  16:45:51 STEP 1387 H= 346:45 +CPU= 26.919
  16:46:17 STEP 1388 H= 347:00 +CPU= 26.235
  16:46:42 STEP 1389 H= 347:15 +CPU= 25.199
  16:47:07 STEP 1390 H= 347:30 +CPU= 24.271
  16:47:31 STEP 1391 H= 347:45 +CPU= 24.686
  16:48:06 STEP 1392 H= 348:00 +CPU= 34.465
Moving to projects directory: /home/test/slots/36/ICMGGh7zg+001392
Moving to projects directory: /home/test/slots/36/ICMSHh7zg+001392
Moving to projects directory: /home/test/slots/36/ICMUAh7zg+001392
  16:48:32 STEP 1393 H= 348:15 +CPU= 25.472
  16:48:55 STEP 1394 H= 348:30 +CPU= 23.183
  16:49:18 STEP 1395 H= 348:45 +CPU= 23.196
  16:49:43 STEP 1396 H= 349:00 +CPU= 24.093
  16:50:08 STEP 1397 H= 349:15 +CPU= 24.959
  16:50:34 STEP 1398 H= 349:30 +CPU= 26.460
  16:51:00 STEP 1399 H= 349:45 +CPU= 25.775
  16:51:25 STEP 1400 H= 350:00 +CPU= 25.229
  16:51:49 STEP 1401 H= 350:15 +CPU= 23.953
  16:52:13 STEP 1402 H= 350:30 +CPU= 24.184
  16:52:39 STEP 1403 H= 350:45 +CPU= 25.420
  16:53:10 STEP 1404 H= 351:00 +CPU= 31.207
Moving to projects directory: /home/test/slots/36/ICMGGh7zg+001404
Moving to projects directory: /home/test/slots/36/ICMSHh7zg+001404
Moving to projects directory: /home/test/slots/36/ICMUAh7zg+001404
  16:53:37 STEP 1405 H= 351:15 +CPU= 26.322
  16:54:03 STEP 1406 H= 351:30 +CPU= 25.962
  16:54:28 STEP 1407 H= 351:45 +CPU= 24.835
  16:54:54 STEP 1408 H= 352:00 +CPU= 26.023
  16:55:20 STEP 1409 H= 352:15 +CPU= 26.241
  16:55:44 STEP 1410 H= 352:30 +CPU= 23.449
  16:56:08 STEP 1411 H= 352:45 +CPU= 24.064
  16:56:33 STEP 1412 H= 353:00 +CPU= 25.306
  16:56:57 STEP 1413 H= 353:15 +CPU= 23.942
  16:57:22 STEP 1414 H= 353:30 +CPU= 23.949
  16:57:47 STEP 1415 H= 353:45 +CPU= 24.926
  16:58:18 STEP 1416 H= 354:00 +CPU= 30.733
Moving to projects directory: /home/test/slots/36/ICMGGh7zg+001416
Moving to projects directory: /home/test/slots/36/ICMSHh7zg+001416
Moving to projects directory: /home/test/slots/36/ICMUAh7zg+001416
  16:58:42 STEP 1417 H= 354:15 +CPU= 23.905
  16:59:07 STEP 1418 H= 354:30 +CPU= 24.609
  16:59:32 STEP 1419 H= 354:45 +CPU= 25.269
  16:59:56 STEP 1420 H= 355:00 +CPU= 24.577
  17:00:22 STEP 1421 H= 355:15 +CPU= 24.978
  17:00:47 STEP 1422 H= 355:30 +CPU= 25.161
  17:01:12 STEP 1423 H= 355:45 +CPU= 25.222
  17:01:37 STEP 1424 H= 356:00 +CPU= 24.713
  17:02:02 STEP 1425 H= 356:15 +CPU= 25.422
  17:02:28 STEP 1426 H= 356:30 +CPU= 25.463
  17:02:54 STEP 1427 H= 356:45 +CPU= 26.146
  17:03:25 STEP 1428 H= 357:00 +CPU= 31.400
Moving to projects directory: /home/test/slots/36/ICMGGh7zg+001428
Moving to projects directory: /home/test/slots/36/ICMSHh7zg+001428
Moving to projects directory: /home/test/slots/36/ICMUAh7zg+001428
  17:03:52 STEP 1429 H= 357:15 +CPU= 26.483
  17:04:17 STEP 1430 H= 357:30 +CPU= 25.403
  17:04:42 STEP 1431 H= 357:45 +CPU= 24.983
  17:05:07 STEP 1432 H= 358:00 +CPU= 24.887
  17:05:32 STEP 1433 H= 358:15 +CPU= 24.797
  17:05:58 STEP 1434 H= 358:30 +CPU= 25.700
  17:06:23 STEP 1435 H= 358:45 +CPU= 24.894
  17:06:48 STEP 1436 H= 359:00 +CPU= 25.081
  17:07:13 STEP 1437 H= 359:15 +CPU= 24.490
  17:07:38 STEP 1438 H= 359:30 +CPU= 25.001
  17:08:03 STEP 1439 H= 359:45 +CPU= 25.544
  17:08:36 STEP 1440 H= 360:00 +CPU= 32.707
..The child process terminated with status: 0
>>> Printing last 70 lines from file: NODE.001_01

 
DDH------------------------------------------------------------------------
I                                                                         I
I TIME =  360.000H DDH * Diagnostic Var.,     AREA                GLOBAL  I
I                                                                         I
I-------------------------------------------------------------------------I
I   VARIABLE   I      Initial value         I        Current value        I
I-------------------------------------------------------------------------I
I  OMEGA (P/S) I     -0.8517519536E-21      I       0.6078043463E-05      I
ICloud fractionI      0.0000000000E+00      I       0.2553433671E-01      I
I  Relat. Hum. I      0.3106023497E+00      I       0.3512741524E+00      I
I  PASS 01     I      0.0000000000E+00      I       0.4510665748E+03      I
DDH------------------------------------------------------------------------
  
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
 NSTEP =  1440 SCAN2M_HPOS  P
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ICMGGh7zg+001440 MODE=a
 IO-STREAM CLOSED - ICMGGh7zg+001440
 MPI-TASK:   1 -    309257108 BYTES IN      7 RECORDS TRANSFERRED IN    0.0007 SECONDS441795.9 Mbytes/s, TOTAL TIME=    0.0119( 5.9%)
 17:08:36 STEP 1440 H= 360:00 +CPU= 32.707
 END CNT3
 NSTEP =  1440 CNT0   000000000  23.238  23.238  23.384   0.994               0               0               0               0               0
IO-STREAM STATISTICS, TOTAL NO OF REC/BYTES READ           46/  1069125708 WRITTEN        24310/************
 -TASK-OPENED-OPEN-RECS IN -KBYTE IN    -RECS OUT-KBYTE OUT-WALL   -WALL IN-WALL
  OU-TOT IN -TOT OUT
   1    428    0       46      1044068    24310 ************    82.3     2.5    79.8     4.8   156.8
===-=== START OF TIMING STATISTICS ===-===
 
STATS FOR ALL TASKS
 NUM ROUTINE                                     CALLS  MEAN(ms)   MAX(ms)   FRAC(%)  UNBAL(%)
   0 CNT0     - COMPLETE EXECUTION                   1 ********* *********    100.00      0.00
   1 CNT4     - FORWARD INTEGRATION                  1 ********* *********     99.98      0.00
   8 SCAN2M - GRID-POINT DYNAMICS                  885    9678.6    9678.6     21.90      0.00
   9 SPCM     - SPECTRAL COMP.                     816    2351.4    2351.4      4.91      0.00
  10 SCAN2M - PHYSICS                              817   19640.4   19640.4     41.02      0.00
  11 IOPACK   - OUTPUT P.P. RESULTS                 68    1878.0    1878.0      0.33      0.00
  12 SPNORM   - SPECTRAL NORM COMP.                 35     128.2     128.2      0.01      0.00
  17 GRIDFPOS IN CNT4                               68      54.9      54.9      0.01      0.00
  27 RESTART FILES - WRITING                        17    7511.8    7511.8      0.33      0.00
  28 RESTART FILES - READING                         1    6927.3    6927.3      0.02      0.00
  29 SU4FPOS IN CNT4                                68       0.1       0.1      0.00      0.00
  30 DYNFPOS IN CNT4                                68    8484.5    8484.5      1.48      0.00
  31 POSDDH IN STEPO                                 4       2.8       2.8      0.00      0.00
  37 CPGLAG   - SL COMPUTATIONS                    817    8994.9    8994.9     18.79      0.00
  39 SU0YOMB                                         1     667.1     667.1      0.00      0.00
  51 SCAN2M   - SL COMM. PART 1                    817      56.8      56.8      0.12      0.00
  54 SPCM     - M TO S/S TO M TRANSP.              816    1246.6    1246.6      2.60      0.00
  55 SPCIMPF  - S TO M/M TO S TRANSP.              816     128.2     128.2      0.27      0.00
  56 SPNORM   - SPECTRAL NORM COMM.                 35       0.9       0.9      0.00      0.00
 102 LTINV_CTL   - INVERSE LEGENDRE TRANSFORM     1772     618.8     618.8      2.80      0.00
 103 LTDIR_CTL   - DIRECT LEGENDRE TRANSFORM      1700     332.2     332.2      1.44      0.00
 106 FTDIR_CTL   - DIRECT FOURIER TRANSFORM       1700     148.9     148.9      0.65      0.00
 107 FTINV_CTL   - INVERSE FOURIER TRANSFORM      1772     220.7     220.7      1.00      0.00
 140 SULEG       - COMP. OF LEGENDRE POL.            1      63.7      63.7      0.00      0.00
 152 LTINV_CTL   - M TO L TRANSPOSITION           1772     144.5     144.5      0.65      0.00
 153 LTDIR_CTL   - L TO M TRANSPOSITION           1700      89.3      89.3      0.39      0.00
 157 FTINV_CTL   - L TO G TRANSPOSITION           1772     235.7     235.7      1.07      0.00
 158 FTDIR_CTL   - G TO L TRANSPOSITION           1700     114.4     114.4      0.50      0.00
 400 GSTATS                                      95450       0.0       0.0      0.00      0.00
 401 GSTATS HOOK                                 88756       0.0       0.0      0.00      0.00
TOTAL MEASURED IMBALANCE =       0.0 SECONDS,  0.0 PERCENT
TOTAL WALLCLOCK TIME    39114.3 CPU TIME   38846.8 VECTOR TIME    38846.8


===-=== END   OF TIMING STATISTICS ===-===
FORECAST DAYS PER DAY       33.1
  *** END CNT0 *** 
------------------------------------------------
Moving to projects directory: /home/test/slots/36/ICMGGh7zg+001440
Moving to projects directory: /home/test/slots/36/ICMSHh7zg+001440
Moving to projects directory: /home/test/slots/36/ICMUAh7zg+001440
Adding to the zip: /home/test/slots/36/NODE.001_01
Adding to the zip: /home/test/slots/36/ifs.stat
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMGGh7zg+001344
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMSHh7zg+001380
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMGGh7zg+001440
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMSHh7zg+001368
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMUAh7zg+001428
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMSHh7zg+001416
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMUAh7zg+001380
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMGGh7zg+001416
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMUAh7zg+001344
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMSHh7zg+001404
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMGGh7zg+001356
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMUAh7zg+001356
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMGGh7zg+001428
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMSHh7zg+001356
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMUAh7zg+001404
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMGGh7zg+001392
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMGGh7zg+001380
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMUAh7zg+001392
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMUAh7zg+001416
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMSHh7zg+001392
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMGGh7zg+001368
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMGGh7zg+001404
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMSHh7zg+001428
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMUAh7zg+001368
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMSHh7zg+001440
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMUAh7zg+001440
Adding to the zip: /home/test/projects/climateprediction.net/oifs_43r3_bl_12282505/ICMSHh7zg+001344
Zipping up the final file: /home/test/projects/climateprediction.net/oifs_43r3_bl_a0iu_2016092300_20_1017_12282505_0_r1111415241_14.zip
Uploading the final file: upload_file_14.zip
Uploading trickle at timestep: 1295100
17:12:06 (8877): called boinc_finish(0)

</stderr_txt>
<message>
upload failure: <file_xfer_error>
  <file_name>oifs_43r3_bl_a0iu_2016092300_20_1017_12282505_0_r1111415241_15.zip</file_name>
  <error_code>-161 (not found)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>oifs_43r3_bl_a0iu_2016092300_20_1017_12282505_0_r1111415241_16.zip</file_name>
  <error_code>-161 (not found)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>oifs_43r3_bl_a0iu_2016092300_20_1017_12282505_0_r1111415241_17.zip</file_name>
  <error_code>-161 (not found)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>oifs_43r3_bl_a0iu_2016092300_20_1017_12282505_0_r1111415241_18.zip</file_name>
  <error_code>-161 (not found)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>oifs_43r3_bl_a0iu_2016092300_20_1017_12282505_0_r1111415241_19.zip</file_name>
  <error_code>-161 (not found)</error_code>
</file_xfer_error>
</message>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
08 Jun 2024 23:09:50 1545820 22434072 oifs_43r3_bl_a0iu_2016092300_20_1017_12282505_0 1,295,100 77,382 0.0597


©2024 cpdn.org