climateprediction.net home page
Task 22440819

Task 22440819

Name oifs_43r3_bl_a10n_2016092300_20_1017_12283146_1
Workunit 12283146
Created 8 Jun 2024, 10:01:04 UTC
Sent 8 Jun 2024, 10:05:06 UTC
Report deadline 7 Aug 2024, 10:05:06 UTC
Received 8 Jun 2024, 20:04:23 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 0 (0x00000000)
Computer ID 1548614
Run time 8 hours 44 min 15 sec
CPU time 8 hours 38 min 29 sec
Validate state Invalid
Credit 1,976.32
Device peak FLOPS 6.70 GFLOPS
Application version OpenIFS 43r3 Baroclinic Lifecycle v1.13
x86_64-pc-linux-gnu
Peak working set size 5,023.44 MB
Peak swap size 5,980.80 MB
Peak disk usage 1,359.15 MB
Stderr
<core_client_version>7.22.1</core_client_version>
<![CDATA[
<stderr_txt>
CPU= 20.929
  16:27:09 STEP  848 H= 212:00 +CPU= 21.279
  16:27:30 STEP  849 H= 212:15 +CPU= 20.452
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:28:11 STEP  850 H= 212:30 +CPU= 19.772
  16:28:31 STEP  851 H= 212:45 +CPU= 20.715
  16:28:57 STEP  852 H= 213:00 +CPU= 25.947
Moving to projects directory: /home/robbie/slots/18/ICMGGh7zg+000852
Moving to projects directory: /home/robbie/slots/18/ICMSHh7zg+000852
Moving to projects directory: /home/robbie/slots/18/ICMUAh7zg+000852
  16:29:18 STEP  853 H= 213:15 +CPU= 21.004
  16:29:40 STEP  854 H= 213:30 +CPU= 21.160
  16:30:01 STEP  855 H= 213:45 +CPU= 21.168
  16:30:22 STEP  856 H= 214:00 +CPU= 21.456
  16:30:43 STEP  857 H= 214:15 +CPU= 20.859
  16:31:04 STEP  858 H= 214:30 +CPU= 21.201
  16:31:26 STEP  859 H= 214:45 +CPU= 21.065
  16:31:47 STEP  860 H= 215:00 +CPU= 21.395
  16:32:08 STEP  861 H= 215:15 +CPU= 21.016
  16:32:29 STEP  862 H= 215:30 +CPU= 20.917
  16:32:50 STEP  863 H= 215:45 +CPU= 20.944
  16:33:20 STEP  864 H= 216:00 +CPU= 30.151
Moving to projects directory: /home/robbie/slots/18/ICMGGh7zg+000864
Moving to projects directory: /home/robbie/slots/18/ICMSHh7zg+000864
Moving to projects directory: /home/robbie/slots/18/ICMUAh7zg+000864
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMGGh7zg+000768
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMSHh7zg+000768
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMUAh7zg+000768
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMGGh7zg+000780
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMSHh7zg+000780
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMUAh7zg+000780
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMGGh7zg+000792
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMSHh7zg+000792
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMUAh7zg+000792
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMGGh7zg+000804
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMSHh7zg+000804
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMUAh7zg+000804
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMGGh7zg+000816
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMSHh7zg+000816
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMUAh7zg+000816
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMGGh7zg+000828
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMSHh7zg+000828
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMUAh7zg+000828
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMGGh7zg+000840
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMSHh7zg+000840
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMUAh7zg+000840
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMGGh7zg+000852
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMSHh7zg+000852
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMUAh7zg+000852
Zipping up the intermediate file: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_a10n_2016092300_20_1017_12283146_1_r906428035_8.zip
Uploading the intermediate file: upload_file_8.zip
  16:33:42 STEP  865 H= 216:15 +CPU= 21.094
  16:34:03 STEP  866 H= 216:30 +CPU= 21.034
  16:34:24 STEP  867 H= 216:45 +CPU= 21.037
  16:34:45 STEP  868 H= 217:00 +CPU= 21.392
  16:35:06 STEP  869 H= 217:15 +CPU= 21.191
  16:35:27 STEP  870 H= 217:30 +CPU= 21.026
  16:35:48 STEP  871 H= 217:45 +CPU= 20.983
  16:36:10 STEP  872 H= 218:00 +CPU= 21.294
  16:36:31 STEP  873 H= 218:15 +CPU= 20.979
  16:36:52 STEP  874 H= 218:30 +CPU= 20.950
  16:37:13 STEP  875 H= 218:45 +CPU= 20.902
  16:37:39 STEP  876 H= 219:00 +CPU= 26.069
Moving to projects directory: /home/robbie/slots/18/ICMGGh7zg+000876
Moving to projects directory: /home/robbie/slots/18/ICMSHh7zg+000876
Moving to projects directory: /home/robbie/slots/18/ICMUAh7zg+000876
  16:38:00 STEP  877 H= 219:15 +CPU= 20.893
  16:38:21 STEP  878 H= 219:30 +CPU= 20.876
  16:38:42 STEP  879 H= 219:45 +CPU= 20.903
  16:39:03 STEP  880 H= 220:00 +CPU= 21.352
  16:39:24 STEP  881 H= 220:15 +CPU= 21.013
  16:39:45 STEP  882 H= 220:30 +CPU= 20.894
  16:40:06 STEP  883 H= 220:45 +CPU= 20.935
  16:40:27 STEP  884 H= 221:00 +CPU= 21.462
  16:40:48 STEP  885 H= 221:15 +CPU= 21.083
  16:41:09 STEP  886 H= 221:30 +CPU= 20.948
  16:41:30 STEP  887 H= 221:45 +CPU= 21.020
  16:41:57 STEP  888 H= 222:00 +CPU= 26.389
Moving to projects directory: /home/robbie/slots/18/ICMGGh7zg+000888
Moving to projects directory: /home/robbie/slots/18/ICMSHh7zg+000888
Moving to projects directory: /home/robbie/slots/18/ICMUAh7zg+000888
  16:42:18 STEP  889 H= 222:15 +CPU= 20.941
  16:42:39 STEP  890 H= 222:30 +CPU= 21.036
  16:43:00 STEP  891 H= 222:45 +CPU= 21.113
  16:43:22 STEP  892 H= 223:00 +CPU= 21.276
  16:43:43 STEP  893 H= 223:15 +CPU= 20.997
  16:44:03 STEP  894 H= 223:30 +CPU= 20.883
  16:44:25 STEP  895 H= 223:45 +CPU= 21.045
  16:44:46 STEP  896 H= 224:00 +CPU= 21.368
  16:45:07 STEP  897 H= 224:15 +CPU= 21.034
  16:45:28 STEP  898 H= 224:30 +CPU= 21.060
  16:45:49 STEP  899 H= 224:45 +CPU= 21.013
  16:46:15 STEP  900 H= 225:00 +CPU= 26.123
Moving to projects directory: /home/robbie/slots/18/ICMGGh7zg+000900
Moving to projects directory: /home/robbie/slots/18/ICMSHh7zg+000900
Moving to projects directory: /home/robbie/slots/18/ICMUAh7zg+000900
  16:46:36 STEP  901 H= 225:15 +CPU= 20.929
  16:46:57 STEP  902 H= 225:30 +CPU= 20.922
  16:47:18 STEP  903 H= 225:45 +CPU= 20.952
  16:47:40 STEP  904 H= 226:00 +CPU= 21.459
  16:48:01 STEP  905 H= 226:15 +CPU= 20.902
  16:48:22 STEP  906 H= 226:30 +CPU= 20.906
  16:48:42 STEP  907 H= 226:45 +CPU= 20.811
  16:49:04 STEP  908 H= 227:00 +CPU= 21.304
  16:49:25 STEP  909 H= 227:15 +CPU= 20.900
  16:49:46 STEP  910 H= 227:30 +CPU= 20.941
  16:50:06 STEP  911 H= 227:45 +CPU= 20.872
  16:50:37 STEP  912 H= 228:00 +CPU= 30.221
Moving to projects directory: /home/robbie/slots/18/ICMGGh7zg+000912
Moving to projects directory: /home/robbie/slots/18/ICMSHh7zg+000912
Moving to projects directory: /home/robbie/slots/18/ICMUAh7zg+000912
  16:50:58 STEP  913 H= 228:15 +CPU= 21.005
  16:51:19 STEP  914 H= 228:30 +CPU= 21.176
  16:51:40 STEP  915 H= 228:45 +CPU= 20.992
  16:52:02 STEP  916 H= 229:00 +CPU= 21.534
  16:52:23 STEP  917 H= 229:15 +CPU= 20.927
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:53:05 STEP  918 H= 229:30 +CPU= 20.166
  16:53:26 STEP  919 H= 229:45 +CPU= 20.800
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:54:08 STEP  920 H= 230:00 +CPU= 20.813
  16:54:29 STEP  921 H= 230:15 +CPU= 20.992
  16:54:49 STEP  922 H= 230:30 +CPU= 20.844
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:55:31 STEP  923 H= 230:45 +CPU= 20.279
  16:55:57 STEP  924 H= 231:00 +CPU= 25.953
Moving to projects directory: /home/robbie/slots/18/ICMGGh7zg+000924
Moving to projects directory: /home/robbie/slots/18/ICMSHh7zg+000924
Moving to projects directory: /home/robbie/slots/18/ICMUAh7zg+000924
  16:56:18 STEP  925 H= 231:15 +CPU= 20.915
  16:56:39 STEP  926 H= 231:30 +CPU= 20.862
  16:57:00 STEP  927 H= 231:45 +CPU= 20.942
  16:57:21 STEP  928 H= 232:00 +CPU= 21.330
  16:57:42 STEP  929 H= 232:15 +CPU= 21.025
  16:58:03 STEP  930 H= 232:30 +CPU= 20.959
  16:58:24 STEP  931 H= 232:45 +CPU= 20.998
  16:58:46 STEP  932 H= 233:00 +CPU= 21.497
  16:59:07 STEP  933 H= 233:15 +CPU= 20.927
  16:59:28 STEP  934 H= 233:30 +CPU= 21.124
  16:59:49 STEP  935 H= 233:45 +CPU= 21.060
  17:00:15 STEP  936 H= 234:00 +CPU= 26.200
Moving to projects directory: /home/robbie/slots/18/ICMGGh7zg+000936
Moving to projects directory: /home/robbie/slots/18/ICMSHh7zg+000936
Moving to projects directory: /home/robbie/slots/18/ICMUAh7zg+000936
  17:00:36 STEP  937 H= 234:15 +CPU= 21.233
  17:00:57 STEP  938 H= 234:30 +CPU= 21.064
  17:01:18 STEP  939 H= 234:45 +CPU= 20.991
  17:01:40 STEP  940 H= 235:00 +CPU= 21.377
  17:02:01 STEP  941 H= 235:15 +CPU= 20.991
  17:02:22 STEP  942 H= 235:30 +CPU= 21.085
  17:02:43 STEP  943 H= 235:45 +CPU= 21.007
  17:03:04 STEP  944 H= 236:00 +CPU= 21.357
  17:03:26 STEP  945 H= 236:15 +CPU= 21.116
  17:03:46 STEP  946 H= 236:30 +CPU= 20.905
  17:04:07 STEP  947 H= 236:45 +CPU= 20.935
  17:04:34 STEP  948 H= 237:00 +CPU= 26.133
Moving to projects directory: /home/robbie/slots/18/ICMGGh7zg+000948
Moving to projects directory: /home/robbie/slots/18/ICMSHh7zg+000948
Moving to projects directory: /home/robbie/slots/18/ICMUAh7zg+000948
  17:04:55 STEP  949 H= 237:15 +CPU= 21.026
  17:05:16 STEP  950 H= 237:30 +CPU= 21.024
  17:05:37 STEP  951 H= 237:45 +CPU= 20.945
  17:05:58 STEP  952 H= 238:00 +CPU= 21.441
  17:06:19 STEP  953 H= 238:15 +CPU= 21.025
  17:06:40 STEP  954 H= 238:30 +CPU= 21.284
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:07:22 STEP  955 H= 238:45 +CPU= 20.633
  17:07:44 STEP  956 H= 239:00 +CPU= 21.455
  17:08:05 STEP  957 H= 239:15 +CPU= 20.971
  17:08:26 STEP  958 H= 239:30 +CPU= 21.201
  17:08:47 STEP  959 H= 239:45 +CPU= 21.298
  17:09:18 STEP  960 H= 240:00 +CPU= 30.443
Moving to projects directory: /home/robbie/slots/18/ICMGGh7zg+000960
Moving to projects directory: /home/robbie/slots/18/ICMSHh7zg+000960
Moving to projects directory: /home/robbie/slots/18/ICMUAh7zg+000960
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMGGh7zg+000864
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMSHh7zg+000864
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMUAh7zg+000864
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMGGh7zg+000876
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMSHh7zg+000876
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMUAh7zg+000876
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMGGh7zg+000888
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMSHh7zg+000888
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMUAh7zg+000888
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMGGh7zg+000900
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMSHh7zg+000900
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMUAh7zg+000900
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMGGh7zg+000912
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMSHh7zg+000912
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMUAh7zg+000912
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMGGh7zg+000924
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMSHh7zg+000924
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMUAh7zg+000924
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMGGh7zg+000936
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMSHh7zg+000936
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMUAh7zg+000936
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMGGh7zg+000948
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMSHh7zg+000948
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMUAh7zg+000948
Zipping up the intermediate file: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_a10n_2016092300_20_1017_12283146_1_r906428035_9.zip
Uploading the intermediate file: upload_file_9.zip
  17:09:39 STEP  961 H= 240:15 +CPU= 21.178
Uploading trickle at timestep: 864000
  17:10:01 STEP  962 H= 240:30 +CPU= 21.042
  17:10:22 STEP  963 H= 240:45 +CPU= 21.424
  17:10:44 STEP  964 H= 241:00 +CPU= 21.719
  17:11:05 STEP  965 H= 241:15 +CPU= 21.277
  17:11:26 STEP  966 H= 241:30 +CPU= 21.028
  17:11:47 STEP  967 H= 241:45 +CPU= 21.184
  17:12:09 STEP  968 H= 242:00 +CPU= 21.400
  17:12:30 STEP  969 H= 242:15 +CPU= 21.076
  17:12:51 STEP  970 H= 242:30 +CPU= 21.044
  17:13:12 STEP  971 H= 242:45 +CPU= 21.091
  17:13:38 STEP  972 H= 243:00 +CPU= 26.194
Moving to projects directory: /home/robbie/slots/18/ICMGGh7zg+000972
Moving to projects directory: /home/robbie/slots/18/ICMSHh7zg+000972
Moving to projects directory: /home/robbie/slots/18/ICMUAh7zg+000972
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:14:20 STEP  973 H= 243:15 +CPU= 20.311
  17:14:41 STEP  974 H= 243:30 +CPU= 20.867
  17:15:02 STEP  975 H= 243:45 +CPU= 21.014
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:15:43 STEP  976 H= 244:00 +CPU= 20.728
  17:16:05 STEP  977 H= 244:15 +CPU= 21.008
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:16:45 STEP  978 H= 244:30 +CPU= 19.808
  17:17:06 STEP  979 H= 244:45 +CPU= 21.056
  17:17:28 STEP  980 H= 245:00 +CPU= 21.355
  17:17:49 STEP  981 H= 245:15 +CPU= 21.146
  17:18:10 STEP  982 H= 245:30 +CPU= 20.962
  17:18:31 STEP  983 H= 245:45 +CPU= 21.081
  17:18:57 STEP  984 H= 246:00 +CPU= 26.239
Moving to projects directory: /home/robbie/slots/18/ICMGGh7zg+000984
Moving to projects directory: /home/robbie/slots/18/ICMSHh7zg+000984
Moving to projects directory: /home/robbie/slots/18/ICMUAh7zg+000984
  17:19:19 STEP  985 H= 246:15 +CPU= 21.176
  17:19:40 STEP  986 H= 246:30 +CPU= 21.099
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:20:23 STEP  987 H= 246:45 +CPU= 20.934
  17:20:44 STEP  988 H= 247:00 +CPU= 21.601
  17:21:05 STEP  989 H= 247:15 +CPU= 21.064
  17:21:26 STEP  990 H= 247:30 +CPU= 21.051
  17:21:47 STEP  991 H= 247:45 +CPU= 21.010
  17:22:09 STEP  992 H= 248:00 +CPU= 21.442
  17:22:30 STEP  993 H= 248:15 +CPU= 21.102
  17:22:51 STEP  994 H= 248:30 +CPU= 21.154
  17:23:12 STEP  995 H= 248:45 +CPU= 21.070
  17:23:38 STEP  996 H= 249:00 +CPU= 26.058
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /home/robbie/slots/18/ICMGGh7zg+000996
Moving to projects directory: /home/robbie/slots/18/ICMSHh7zg+000996
Moving to projects directory: /home/robbie/slots/18/ICMUAh7zg+000996
  17:24:19 STEP  997 H= 249:15 +CPU= 20.116
  17:24:40 STEP  998 H= 249:30 +CPU= 20.846
  17:25:01 STEP  999 H= 249:45 +CPU= 20.838
  17:25:23 STEP 1000 H= 250:00 +CPU= 21.380
  17:25:44 STEP 1001 H= 250:15 +CPU= 20.993
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:26:25 STEP 1002 H= 250:30 +CPU= 20.107
  17:26:46 STEP 1003 H= 250:45 +CPU= 20.903
  17:27:07 STEP 1004 H= 251:00 +CPU= 21.315
  17:27:28 STEP 1005 H= 251:15 +CPU= 20.923
  17:27:49 STEP 1006 H= 251:30 +CPU= 21.032
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:28:30 STEP 1007 H= 251:45 +CPU= 20.371
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:29:20 STEP 1008 H= 252:00 +CPU= 28.931
Moving to projects directory: /home/robbie/slots/18/ICMGGh7zg+001008
Moving to projects directory: /home/robbie/slots/18/ICMSHh7zg+001008
Moving to projects directory: /home/robbie/slots/18/ICMUAh7zg+001008
  17:29:41 STEP 1009 H= 252:15 +CPU= 20.812
  17:30:02 STEP 1010 H= 252:30 +CPU= 20.807
  17:30:23 STEP 1011 H= 252:45 +CPU= 20.862
  17:30:44 STEP 1012 H= 253:00 +CPU= 21.396
  17:31:05 STEP 1013 H= 253:15 +CPU= 20.931
  17:31:27 STEP 1014 H= 253:30 +CPU= 21.043
  17:31:48 STEP 1015 H= 253:45 +CPU= 21.146
  17:32:09 STEP 1016 H= 254:00 +CPU= 21.504
  17:32:30 STEP 1017 H= 254:15 +CPU= 21.116
  17:32:52 STEP 1018 H= 254:30 +CPU= 21.104
  17:33:13 STEP 1019 H= 254:45 +CPU= 20.978
  17:33:39 STEP 1020 H= 255:00 +CPU= 26.305
Moving to projects directory: /home/robbie/slots/18/ICMGGh7zg+001020
Moving to projects directory: /home/robbie/slots/18/ICMSHh7zg+001020
Moving to projects directory: /home/robbie/slots/18/ICMUAh7zg+001020
  17:34:00 STEP 1021 H= 255:15 +CPU= 21.199
  17:34:21 STEP 1022 H= 255:30 +CPU= 20.874
  17:34:42 STEP 1023 H= 255:45 +CPU= 21.050
  17:35:04 STEP 1024 H= 256:00 +CPU= 21.354
  17:35:25 STEP 1025 H= 256:15 +CPU= 21.075
  17:35:46 STEP 1026 H= 256:30 +CPU= 21.052
  17:36:07 STEP 1027 H= 256:45 +CPU= 20.984
  17:36:28 STEP 1028 H= 257:00 +CPU= 21.330
  17:36:49 STEP 1029 H= 257:15 +CPU= 20.925
  17:37:10 STEP 1030 H= 257:30 +CPU= 20.921
  17:37:31 STEP 1031 H= 257:45 +CPU= 21.016
  17:37:58 STEP 1032 H= 258:00 +CPU= 26.273
Moving to projects directory: /home/robbie/slots/18/ICMGGh7zg+001032
Moving to projects directory: /home/robbie/slots/18/ICMSHh7zg+001032
Moving to projects directory: /home/robbie/slots/18/ICMUAh7zg+001032
  17:38:19 STEP 1033 H= 258:15 +CPU= 21.053
  17:38:40 STEP 1034 H= 258:30 +CPU= 21.018
  17:39:02 STEP 1035 H= 258:45 +CPU= 21.122
  17:39:23 STEP 1036 H= 259:00 +CPU= 21.426
  17:39:44 STEP 1037 H= 259:15 +CPU= 21.080
  17:40:05 STEP 1038 H= 259:30 +CPU= 20.983
  17:40:26 STEP 1039 H= 259:45 +CPU= 21.082
  17:40:48 STEP 1040 H= 260:00 +CPU= 21.411
  17:41:09 STEP 1041 H= 260:15 +CPU= 21.006
  17:41:30 STEP 1042 H= 260:30 +CPU= 20.907
  17:41:51 STEP 1043 H= 260:45 +CPU= 21.046
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:42:37 STEP 1044 H= 261:00 +CPU= 25.361
Moving to projects directory: /home/robbie/slots/18/ICMGGh7zg+001044
Moving to projects directory: /home/robbie/slots/18/ICMSHh7zg+001044
Moving to projects directory: /home/robbie/slots/18/ICMUAh7zg+001044
  17:42:58 STEP 1045 H= 261:15 +CPU= 21.007
  17:43:19 STEP 1046 H= 261:30 +CPU= 20.956
  17:43:40 STEP 1047 H= 261:45 +CPU= 20.878
  17:44:01 STEP 1048 H= 262:00 +CPU= 21.373
  17:44:22 STEP 1049 H= 262:15 +CPU= 21.036
  17:44:44 STEP 1050 H= 262:30 +CPU= 21.140
  17:45:05 STEP 1051 H= 262:45 +CPU= 21.025
  17:45:26 STEP 1052 H= 263:00 +CPU= 21.644
  17:45:47 STEP 1053 H= 263:15 +CPU= 21.168
  17:46:08 STEP 1054 H= 263:30 +CPU= 21.006
  17:46:29 STEP 1055 H= 263:45 +CPU= 21.000
  17:47:00 STEP 1056 H= 264:00 +CPU= 30.212
Moving to projects directory: /home/robbie/slots/18/ICMGGh7zg+001056
Moving to projects directory: /home/robbie/slots/18/ICMSHh7zg+001056
Moving to projects directory: /home/robbie/slots/18/ICMUAh7zg+001056
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMGGh7zg+000960
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMSHh7zg+000960
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMUAh7zg+000960
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMGGh7zg+000972
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMSHh7zg+000972
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMUAh7zg+000972
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMGGh7zg+000984
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMSHh7zg+000984
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMUAh7zg+000984
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMGGh7zg+000996
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMSHh7zg+000996
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMUAh7zg+000996
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMGGh7zg+001008
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMSHh7zg+001008
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMUAh7zg+001008
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMGGh7zg+001020
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMSHh7zg+001020
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMUAh7zg+001020
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMGGh7zg+001032
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMSHh7zg+001032
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMUAh7zg+001032
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMGGh7zg+001044
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMSHh7zg+001044
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMUAh7zg+001044
Zipping up the intermediate file: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_a10n_2016092300_20_1017_12283146_1_r906428035_10.zip
Uploading the intermediate file: upload_file_10.zip
  17:47:21 STEP 1057 H= 264:15 +CPU= 21.096
  17:47:42 STEP 1058 H= 264:30 +CPU= 20.922
  17:48:03 STEP 1059 H= 264:45 +CPU= 21.136
  17:48:25 STEP 1060 H= 265:00 +CPU= 21.577
  17:48:46 STEP 1061 H= 265:15 +CPU= 21.063
  17:49:07 STEP 1062 H= 265:30 +CPU= 21.049
  17:49:28 STEP 1063 H= 265:45 +CPU= 21.012
  17:49:50 STEP 1064 H= 266:00 +CPU= 21.517
  17:50:11 STEP 1065 H= 266:15 +CPU= 21.189
  17:50:32 STEP 1066 H= 266:30 +CPU= 21.077
  17:50:53 STEP 1067 H= 266:45 +CPU= 21.094
  17:51:20 STEP 1068 H= 267:00 +CPU= 26.236
Moving to projects directory: /home/robbie/slots/18/ICMGGh7zg+001068
Moving to projects directory: /home/robbie/slots/18/ICMSHh7zg+001068
Moving to projects directory: /home/robbie/slots/18/ICMUAh7zg+001068
  17:51:41 STEP 1069 H= 267:15 +CPU= 21.099
  17:52:02 STEP 1070 H= 267:30 +CPU= 21.104
  17:52:23 STEP 1071 H= 267:45 +CPU= 21.089
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:53:04 STEP 1072 H= 268:00 +CPU= 19.998
  17:53:25 STEP 1073 H= 268:15 +CPU= 20.992
  17:53:46 STEP 1074 H= 268:30 +CPU= 21.115
  17:54:08 STEP 1075 H= 268:45 +CPU= 21.161
  17:54:29 STEP 1076 H= 269:00 +CPU= 21.503
  17:54:50 STEP 1077 H= 269:15 +CPU= 21.403
  17:55:12 STEP 1078 H= 269:30 +CPU= 21.356
  17:55:33 STEP 1079 H= 269:45 +CPU= 21.302
  17:56:00 STEP 1080 H= 270:00 +CPU= 26.529
Moving to projects directory: /home/robbie/slots/18/ICMGGh7zg+001080
Moving to projects directory: /home/robbie/slots/18/ICMSHh7zg+001080
Moving to projects directory: /home/robbie/slots/18/ICMUAh7zg+001080
  17:56:21 STEP 1081 H= 270:15 +CPU= 21.094
  17:56:42 STEP 1082 H= 270:30 +CPU= 21.110
  17:57:03 STEP 1083 H= 270:45 +CPU= 21.141
  17:57:25 STEP 1084 H= 271:00 +CPU= 21.770
  17:57:46 STEP 1085 H= 271:15 +CPU= 21.377
  17:58:08 STEP 1086 H= 271:30 +CPU= 21.334
  17:58:28 STEP 1087 H= 271:45 +CPU= 20.574
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:59:10 STEP 1088 H= 272:00 +CPU= 20.904
  17:59:31 STEP 1089 H= 272:15 +CPU= 21.162
  17:59:53 STEP 1090 H= 272:30 +CPU= 21.166
  18:00:14 STEP 1091 H= 272:45 +CPU= 21.095
  18:00:40 STEP 1092 H= 273:00 +CPU= 26.242
Moving to projects directory: /home/robbie/slots/18/ICMGGh7zg+001092
Moving to projects directory: /home/robbie/slots/18/ICMSHh7zg+001092
Moving to projects directory: /home/robbie/slots/18/ICMUAh7zg+001092
  18:01:01 STEP 1093 H= 273:15 +CPU= 21.175
  18:01:22 STEP 1094 H= 273:30 +CPU= 21.125
  18:01:44 STEP 1095 H= 273:45 +CPU= 21.213
  18:02:05 STEP 1096 H= 274:00 +CPU= 21.519
  18:02:26 STEP 1097 H= 274:15 +CPU= 21.081
  18:02:47 STEP 1098 H= 274:30 +CPU= 21.092
  18:03:09 STEP 1099 H= 274:45 +CPU= 21.180
  18:03:30 STEP 1100 H= 275:00 +CPU= 21.711
  18:03:52 STEP 1101 H= 275:15 +CPU= 21.379
  18:04:13 STEP 1102 H= 275:30 +CPU= 21.268
  18:04:34 STEP 1103 H= 275:45 +CPU= 21.272
  18:05:05 STEP 1104 H= 276:00 +CPU= 30.374
Moving to projects directory: /home/robbie/slots/18/ICMGGh7zg+001104
Moving to projects directory: /home/robbie/slots/18/ICMSHh7zg+001104
Moving to projects directory: /home/robbie/slots/18/ICMUAh7zg+001104
  18:05:26 STEP 1105 H= 276:15 +CPU= 21.186
  18:05:47 STEP 1106 H= 276:30 +CPU= 21.183
  18:06:09 STEP 1107 H= 276:45 +CPU= 21.267
  18:06:30 STEP 1108 H= 277:00 +CPU= 21.526
  18:06:51 STEP 1109 H= 277:15 +CPU= 21.102
  18:07:13 STEP 1110 H= 277:30 +CPU= 21.267
  18:07:34 STEP 1111 H= 277:45 +CPU= 21.138
  18:07:55 STEP 1112 H= 278:00 +CPU= 21.573
  18:08:17 STEP 1113 H= 278:15 +CPU= 21.201
  18:08:38 STEP 1114 H= 278:30 +CPU= 21.320
  18:08:59 STEP 1115 H= 278:45 +CPU= 21.298
  18:09:26 STEP 1116 H= 279:00 +CPU= 26.258
Moving to projects directory: /home/robbie/slots/18/ICMGGh7zg+001116
Moving to projects directory: /home/robbie/slots/18/ICMSHh7zg+001116
Moving to projects directory: /home/robbie/slots/18/ICMUAh7zg+001116
  18:09:47 STEP 1117 H= 279:15 +CPU= 21.229
  18:10:08 STEP 1118 H= 279:30 +CPU= 21.113
  18:10:30 STEP 1119 H= 279:45 +CPU= 21.424
  18:10:51 STEP 1120 H= 280:00 +CPU= 21.798
  18:11:13 STEP 1121 H= 280:15 +CPU= 21.454
  18:11:34 STEP 1122 H= 280:30 +CPU= 21.442
  18:11:56 STEP 1123 H= 280:45 +CPU= 21.373
  18:12:17 STEP 1124 H= 281:00 +CPU= 21.787
  18:12:39 STEP 1125 H= 281:15 +CPU= 21.435
  18:13:00 STEP 1126 H= 281:30 +CPU= 21.108
  18:13:21 STEP 1127 H= 281:45 +CPU= 21.269
  18:13:48 STEP 1128 H= 282:00 +CPU= 26.503
Moving to projects directory: /home/robbie/slots/18/ICMGGh7zg+001128
Moving to projects directory: /home/robbie/slots/18/ICMSHh7zg+001128
Moving to projects directory: /home/robbie/slots/18/ICMUAh7zg+001128
  18:14:09 STEP 1129 H= 282:15 +CPU= 21.276
  18:14:30 STEP 1130 H= 282:30 +CPU= 21.253
  18:14:52 STEP 1131 H= 282:45 +CPU= 21.131
  18:15:13 STEP 1132 H= 283:00 +CPU= 21.683
  18:15:35 STEP 1133 H= 283:15 +CPU= 21.312
  18:15:56 STEP 1134 H= 283:30 +CPU= 21.198
  18:16:17 STEP 1135 H= 283:45 +CPU= 21.200
  18:16:39 STEP 1136 H= 284:00 +CPU= 21.698
  18:17:00 STEP 1137 H= 284:15 +CPU= 21.256
  18:17:21 STEP 1138 H= 284:30 +CPU= 21.308
  18:17:42 STEP 1139 H= 284:45 +CPU= 21.157
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  18:18:49 STEP 1140 H= 285:00 +CPU= 24.663
Moving to projects directory: /home/robbie/slots/18/ICMGGh7zg+001140
Moving to projects directory: /home/robbie/slots/18/ICMSHh7zg+001140
Moving to projects directory: /home/robbie/slots/18/ICMUAh7zg+001140
  18:19:10 STEP 1141 H= 285:15 +CPU= 20.890
  18:19:31 STEP 1142 H= 285:30 +CPU= 21.182
  18:19:52 STEP 1143 H= 285:45 +CPU= 21.079
  18:20:14 STEP 1144 H= 286:00 +CPU= 21.604
  18:20:35 STEP 1145 H= 286:15 +CPU= 21.162
  18:20:57 STEP 1146 H= 286:30 +CPU= 21.317
  18:21:18 STEP 1147 H= 286:45 +CPU= 21.448
  18:21:40 STEP 1148 H= 287:00 +CPU= 21.800
  18:22:01 STEP 1149 H= 287:15 +CPU= 21.453
  18:22:23 STEP 1150 H= 287:30 +CPU= 21.481
  18:22:44 STEP 1151 H= 287:45 +CPU= 21.266
  18:23:15 STEP 1152 H= 288:00 +CPU= 30.599
Moving to projects directory: /home/robbie/slots/18/ICMGGh7zg+001152
Moving to projects directory: /home/robbie/slots/18/ICMSHh7zg+001152
Moving to projects directory: /home/robbie/slots/18/ICMUAh7zg+001152
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMGGh7zg+001056
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMSHh7zg+001056
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMUAh7zg+001056
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMGGh7zg+001068
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMSHh7zg+001068
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMUAh7zg+001068
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMGGh7zg+001080
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMSHh7zg+001080
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMUAh7zg+001080
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMGGh7zg+001092
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMSHh7zg+001092
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMUAh7zg+001092
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMGGh7zg+001104
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMSHh7zg+001104
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMUAh7zg+001104
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMGGh7zg+001116
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMSHh7zg+001116
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMUAh7zg+001116
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMGGh7zg+001128
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMSHh7zg+001128
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMUAh7zg+001128
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMGGh7zg+001140
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMSHh7zg+001140
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMUAh7zg+001140
Zipping up the intermediate file: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_a10n_2016092300_20_1017_12283146_1_r906428035_11.zip
Uploading the intermediate file: upload_file_11.zip
  18:23:37 STEP 1153 H= 288:15 +CPU= 21.413
  18:23:58 STEP 1154 H= 288:30 +CPU= 21.275
  18:24:19 STEP 1155 H= 288:45 +CPU= 21.308
  18:24:41 STEP 1156 H= 289:00 +CPU= 21.782
  18:25:03 STEP 1157 H= 289:15 +CPU= 21.404
  18:25:24 STEP 1158 H= 289:30 +CPU= 21.405
  18:25:45 STEP 1159 H= 289:45 +CPU= 21.360
  18:26:07 STEP 1160 H= 290:00 +CPU= 21.881
  18:26:29 STEP 1161 H= 290:15 +CPU= 21.475
  18:26:50 STEP 1162 H= 290:30 +CPU= 21.507
  18:27:12 STEP 1163 H= 290:45 +CPU= 21.365
  18:27:38 STEP 1164 H= 291:00 +CPU= 26.456
Moving to projects directory: /home/robbie/slots/18/ICMGGh7zg+001164
Moving to projects directory: /home/robbie/slots/18/ICMSHh7zg+001164
Moving to projects directory: /home/robbie/slots/18/ICMUAh7zg+001164
  18:28:00 STEP 1165 H= 291:15 +CPU= 21.548
  18:28:21 STEP 1166 H= 291:30 +CPU= 21.407
  18:28:42 STEP 1167 H= 291:45 +CPU= 21.145
  18:29:04 STEP 1168 H= 292:00 +CPU= 21.642
  18:29:26 STEP 1169 H= 292:15 +CPU= 21.365
  18:29:47 STEP 1170 H= 292:30 +CPU= 21.250
  18:30:08 STEP 1171 H= 292:45 +CPU= 21.399
  18:30:30 STEP 1172 H= 293:00 +CPU= 21.710
  18:30:51 STEP 1173 H= 293:15 +CPU= 21.412
  18:31:13 STEP 1174 H= 293:30 +CPU= 21.289
  18:31:34 STEP 1175 H= 293:45 +CPU= 21.268
  18:32:01 STEP 1176 H= 294:00 +CPU= 26.455
Moving to projects directory: /home/robbie/slots/18/ICMGGh7zg+001176
Moving to projects directory: /home/robbie/slots/18/ICMSHh7zg+001176
Moving to projects directory: /home/robbie/slots/18/ICMUAh7zg+001176
  18:32:22 STEP 1177 H= 294:15 +CPU= 21.278
  18:32:43 STEP 1178 H= 294:30 +CPU= 21.470
  18:33:05 STEP 1179 H= 294:45 +CPU= 21.363
  18:33:27 STEP 1180 H= 295:00 +CPU= 21.697
  18:33:48 STEP 1181 H= 295:15 +CPU= 21.234
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  18:34:29 STEP 1182 H= 295:30 +CPU= 20.382
  18:34:51 STEP 1183 H= 295:45 +CPU= 21.307
  18:35:12 STEP 1184 H= 296:00 +CPU= 21.642
  18:35:33 STEP 1185 H= 296:15 +CPU= 21.210
  18:35:55 STEP 1186 H= 296:30 +CPU= 21.179
  18:36:16 STEP 1187 H= 296:45 +CPU= 21.194
  18:36:42 STEP 1188 H= 297:00 +CPU= 26.402
Moving to projects directory: /home/robbie/slots/18/ICMGGh7zg+001188
Moving to projects directory: /home/robbie/slots/18/ICMSHh7zg+001188
Moving to projects directory: /home/robbie/slots/18/ICMUAh7zg+001188
  18:37:04 STEP 1189 H= 297:15 +CPU= 21.182
  18:37:25 STEP 1190 H= 297:30 +CPU= 21.217
  18:37:46 STEP 1191 H= 297:45 +CPU= 21.385
  18:38:08 STEP 1192 H= 298:00 +CPU= 21.695
  18:38:29 STEP 1193 H= 298:15 +CPU= 21.315
  18:38:51 STEP 1194 H= 298:30 +CPU= 21.229
  18:39:12 STEP 1195 H= 298:45 +CPU= 21.175
  18:39:34 STEP 1196 H= 299:00 +CPU= 21.740
  18:39:55 STEP 1197 H= 299:15 +CPU= 21.232
  18:40:16 STEP 1198 H= 299:30 +CPU= 21.218
  18:40:38 STEP 1199 H= 299:45 +CPU= 21.351
  18:41:08 STEP 1200 H= 300:00 +CPU= 30.556
Moving to projects directory: /home/robbie/slots/18/ICMGGh7zg+001200
Moving to projects directory: /home/robbie/slots/18/ICMSHh7zg+001200
Moving to projects directory: /home/robbie/slots/18/ICMUAh7zg+001200
  18:41:30 STEP 1201 H= 300:15 +CPU= 21.401
  18:41:51 STEP 1202 H= 300:30 +CPU= 21.512
  18:42:13 STEP 1203 H= 300:45 +CPU= 21.468
  18:42:35 STEP 1204 H= 301:00 +CPU= 22.135
  18:42:56 STEP 1205 H= 301:15 +CPU= 21.273
  18:43:18 STEP 1206 H= 301:30 +CPU= 21.403
  18:43:39 STEP 1207 H= 301:45 +CPU= 21.313
  18:44:01 STEP 1208 H= 302:00 +CPU= 21.644
  18:44:22 STEP 1209 H= 302:15 +CPU= 21.348
  18:44:43 STEP 1210 H= 302:30 +CPU= 21.359
  18:45:05 STEP 1211 H= 302:45 +CPU= 21.431
  18:45:31 STEP 1212 H= 303:00 +CPU= 26.501
Moving to projects directory: /home/robbie/slots/18/ICMGGh7zg+001212
Moving to projects directory: /home/robbie/slots/18/ICMSHh7zg+001212
Moving to projects directory: /home/robbie/slots/18/ICMUAh7zg+001212
  18:45:53 STEP 1213 H= 303:15 +CPU= 21.257
  18:46:14 STEP 1214 H= 303:30 +CPU= 21.373
  18:46:35 STEP 1215 H= 303:45 +CPU= 21.278
  18:46:57 STEP 1216 H= 304:00 +CPU= 22.015
  18:47:19 STEP 1217 H= 304:15 +CPU= 21.570
  18:47:40 STEP 1218 H= 304:30 +CPU= 21.449
  18:48:02 STEP 1219 H= 304:45 +CPU= 21.515
  18:48:24 STEP 1220 H= 305:00 +CPU= 21.860
  18:48:45 STEP 1221 H= 305:15 +CPU= 21.299
  18:49:06 STEP 1222 H= 305:30 +CPU= 21.228
  18:49:28 STEP 1223 H= 305:45 +CPU= 21.234
  18:49:54 STEP 1224 H= 306:00 +CPU= 26.371
Moving to projects directory: /home/robbie/slots/18/ICMGGh7zg+001224
Moving to projects directory: /home/robbie/slots/18/ICMSHh7zg+001224
Moving to projects directory: /home/robbie/slots/18/ICMUAh7zg+001224
  18:50:16 STEP 1225 H= 306:15 +CPU= 21.387
  18:50:37 STEP 1226 H= 306:30 +CPU= 21.381
  18:50:58 STEP 1227 H= 306:45 +CPU= 21.239
  18:51:20 STEP 1228 H= 307:00 +CPU= 21.755
  18:51:41 STEP 1229 H= 307:15 +CPU= 21.221
  18:52:03 STEP 1230 H= 307:30 +CPU= 21.332
  18:52:24 STEP 1231 H= 307:45 +CPU= 21.413
  18:52:46 STEP 1232 H= 308:00 +CPU= 21.769
  18:53:07 STEP 1233 H= 308:15 +CPU= 21.250
  18:53:29 STEP 1234 H= 308:30 +CPU= 21.326
  18:53:50 STEP 1235 H= 308:45 +CPU= 21.482
  18:54:17 STEP 1236 H= 309:00 +CPU= 26.615
Moving to projects directory: /home/robbie/slots/18/ICMGGh7zg+001236
Moving to projects directory: /home/robbie/slots/18/ICMSHh7zg+001236
Moving to projects directory: /home/robbie/slots/18/ICMUAh7zg+001236
  18:54:38 STEP 1237 H= 309:15 +CPU= 21.382
  18:55:00 STEP 1238 H= 309:30 +CPU= 21.396
  18:55:21 STEP 1239 H= 309:45 +CPU= 21.292
  18:55:43 STEP 1240 H= 310:00 +CPU= 21.786
  18:56:04 STEP 1241 H= 310:15 +CPU= 21.210
  18:56:26 STEP 1242 H= 310:30 +CPU= 21.459
  18:56:47 STEP 1243 H= 310:45 +CPU= 21.348
  18:57:09 STEP 1244 H= 311:00 +CPU= 21.844
  18:57:30 STEP 1245 H= 311:15 +CPU= 21.401
  18:57:52 STEP 1246 H= 311:30 +CPU= 21.280
  18:58:13 STEP 1247 H= 311:45 +CPU= 21.281
  18:58:44 STEP 1248 H= 312:00 +CPU= 30.604
Moving to projects directory: /home/robbie/slots/18/ICMGGh7zg+001248
Moving to projects directory: /home/robbie/slots/18/ICMSHh7zg+001248
Moving to projects directory: /home/robbie/slots/18/ICMUAh7zg+001248
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMGGh7zg+001152
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMSHh7zg+001152
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMUAh7zg+001152
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMGGh7zg+001164
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMSHh7zg+001164
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMUAh7zg+001164
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMGGh7zg+001176
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMSHh7zg+001176
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMUAh7zg+001176
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMGGh7zg+001188
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMSHh7zg+001188
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMUAh7zg+001188
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMGGh7zg+001200
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMSHh7zg+001200
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMUAh7zg+001200
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMGGh7zg+001212
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMSHh7zg+001212
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMUAh7zg+001212
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMGGh7zg+001224
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMSHh7zg+001224
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMUAh7zg+001224
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMGGh7zg+001236
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMSHh7zg+001236
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMUAh7zg+001236
Zipping up the intermediate file: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_a10n_2016092300_20_1017_12283146_1_r906428035_12.zip
Uploading the intermediate file: upload_file_12.zip
  18:59:05 STEP 1249 H= 312:15 +CPU= 21.367
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  18:59:37 STEP 1250 H= 312:30 +CPU= 15.392
  18:59:58 STEP 1251 H= 312:45 +CPU= 21.276
  19:00:20 STEP 1252 H= 313:00 +CPU= 21.709
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  19:01:01 STEP 1253 H= 313:15 +CPU= 20.137
  19:01:23 STEP 1254 H= 313:30 +CPU= 21.274
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  19:02:04 STEP 1255 H= 313:45 +CPU= 20.054
  19:02:25 STEP 1256 H= 314:00 +CPU= 21.543
  19:02:47 STEP 1257 H= 314:15 +CPU= 21.285
  19:03:08 STEP 1258 H= 314:30 +CPU= 21.453
  19:03:30 STEP 1259 H= 314:45 +CPU= 21.526
  19:03:56 STEP 1260 H= 315:00 +CPU= 26.241
Moving to projects directory: /home/robbie/slots/18/ICMGGh7zg+001260
Moving to projects directory: /home/robbie/slots/18/ICMSHh7zg+001260
Moving to projects directory: /home/robbie/slots/18/ICMUAh7zg+001260
  19:04:18 STEP 1261 H= 315:15 +CPU= 21.469
  19:04:39 STEP 1262 H= 315:30 +CPU= 21.507
  19:05:01 STEP 1263 H= 315:45 +CPU= 21.382
  19:05:22 STEP 1264 H= 316:00 +CPU= 21.871
  19:05:44 STEP 1265 H= 316:15 +CPU= 21.518
  19:06:05 STEP 1266 H= 316:30 +CPU= 21.290
  19:06:27 STEP 1267 H= 316:45 +CPU= 21.387
  19:06:48 STEP 1268 H= 317:00 +CPU= 21.728
  19:07:10 STEP 1269 H= 317:15 +CPU= 21.390
  19:07:31 STEP 1270 H= 317:30 +CPU= 21.194
  19:07:52 STEP 1271 H= 317:45 +CPU= 21.262
  19:08:19 STEP 1272 H= 318:00 +CPU= 26.562
Moving to projects directory: /home/robbie/slots/18/ICMGGh7zg+001272
Moving to projects directory: /home/robbie/slots/18/ICMSHh7zg+001272
Moving to projects directory: /home/robbie/slots/18/ICMUAh7zg+001272
  19:08:40 STEP 1273 H= 318:15 +CPU= 21.328
  19:09:02 STEP 1274 H= 318:30 +CPU= 21.410
  19:09:23 STEP 1275 H= 318:45 +CPU= 21.343
  19:09:45 STEP 1276 H= 319:00 +CPU= 21.825
  19:10:06 STEP 1277 H= 319:15 +CPU= 21.466
  19:10:28 STEP 1278 H= 319:30 +CPU= 21.330
  19:10:49 STEP 1279 H= 319:45 +CPU= 21.436
  19:11:11 STEP 1280 H= 320:00 +CPU= 21.806
  19:11:32 STEP 1281 H= 320:15 +CPU= 21.240
  19:11:54 STEP 1282 H= 320:30 +CPU= 21.259
  19:12:15 STEP 1283 H= 320:45 +CPU= 21.410
  19:12:42 STEP 1284 H= 321:00 +CPU= 26.613
Moving to projects directory: /home/robbie/slots/18/ICMGGh7zg+001284
Moving to projects directory: /home/robbie/slots/18/ICMSHh7zg+001284
Moving to projects directory: /home/robbie/slots/18/ICMUAh7zg+001284
  19:13:03 STEP 1285 H= 321:15 +CPU= 21.272
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  19:13:45 STEP 1286 H= 321:30 +CPU= 20.943
  19:14:06 STEP 1287 H= 321:45 +CPU= 21.348
  19:14:28 STEP 1288 H= 322:00 +CPU= 21.713
  19:14:49 STEP 1289 H= 322:15 +CPU= 21.202
  19:15:11 STEP 1290 H= 322:30 +CPU= 21.310
  19:15:32 STEP 1291 H= 322:45 +CPU= 21.302
  19:15:54 STEP 1292 H= 323:00 +CPU= 21.822
  19:16:15 STEP 1293 H= 323:15 +CPU= 21.289
  19:16:37 STEP 1294 H= 323:30 +CPU= 21.341
  19:16:58 STEP 1295 H= 323:45 +CPU= 21.539
  19:17:29 STEP 1296 H= 324:00 +CPU= 30.551
Moving to projects directory: /home/robbie/slots/18/ICMGGh7zg+001296
Moving to projects directory: /home/robbie/slots/18/ICMSHh7zg+001296
Moving to projects directory: /home/robbie/slots/18/ICMUAh7zg+001296
  19:17:50 STEP 1297 H= 324:15 +CPU= 21.376
  19:18:12 STEP 1298 H= 324:30 +CPU= 21.313
  19:18:33 STEP 1299 H= 324:45 +CPU= 21.387
  19:18:55 STEP 1300 H= 325:00 +CPU= 21.802
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  19:19:36 STEP 1301 H= 325:15 +CPU= 20.645
  19:19:58 STEP 1302 H= 325:30 +CPU= 21.543
  19:20:19 STEP 1303 H= 325:45 +CPU= 21.444
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  19:21:01 STEP 1304 H= 326:00 +CPU= 20.425
  19:21:22 STEP 1305 H= 326:15 +CPU= 21.341
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  19:22:05 STEP 1306 H= 326:30 +CPU= 21.134
  19:22:26 STEP 1307 H= 326:45 +CPU= 21.124
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  19:23:13 STEP 1308 H= 327:00 +CPU= 25.666
Moving to projects directory: /home/robbie/slots/18/ICMGGh7zg+001308
Moving to projects directory: /home/robbie/slots/18/ICMSHh7zg+001308
Moving to projects directory: /home/robbie/slots/18/ICMUAh7zg+001308
  19:23:33 STEP 1309 H= 327:15 +CPU= 20.851
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  19:24:15 STEP 1310 H= 327:30 +CPU= 20.972
  19:24:37 STEP 1311 H= 327:45 +CPU= 21.285
  19:24:58 STEP 1312 H= 328:00 +CPU= 21.766
  19:25:20 STEP 1313 H= 328:15 +CPU= 21.362
  19:25:41 STEP 1314 H= 328:30 +CPU= 21.373
  19:26:03 STEP 1315 H= 328:45 +CPU= 21.396
  19:26:24 STEP 1316 H= 329:00 +CPU= 21.780
  19:26:46 STEP 1317 H= 329:15 +CPU= 21.373
  19:27:07 STEP 1318 H= 329:30 +CPU= 21.410
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  19:27:49 STEP 1319 H= 329:45 +CPU= 21.108
  19:28:16 STEP 1320 H= 330:00 +CPU= 26.532
Moving to projects directory: /home/robbie/slots/18/ICMGGh7zg+001320
Moving to projects directory: /home/robbie/slots/18/ICMSHh7zg+001320
Moving to projects directory: /home/robbie/slots/18/ICMUAh7zg+001320
  19:28:37 STEP 1321 H= 330:15 +CPU= 21.274
  19:28:59 STEP 1322 H= 330:30 +CPU= 21.410
  19:29:20 STEP 1323 H= 330:45 +CPU= 21.526
  19:29:42 STEP 1324 H= 331:00 +CPU= 21.816
  19:30:04 STEP 1325 H= 331:15 +CPU= 21.654
  19:30:25 STEP 1326 H= 331:30 +CPU= 21.612
  19:30:47 STEP 1327 H= 331:45 +CPU= 21.414
  19:31:09 STEP 1328 H= 332:00 +CPU= 21.815
  19:31:30 STEP 1329 H= 332:15 +CPU= 21.343
  19:31:52 STEP 1330 H= 332:30 +CPU= 21.568
  19:32:13 STEP 1331 H= 332:45 +CPU= 21.451
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  19:33:01 STEP 1332 H= 333:00 +CPU= 26.189
Moving to projects directory: /home/robbie/slots/18/ICMGGh7zg+001332
Moving to projects directory: /home/robbie/slots/18/ICMSHh7zg+001332
Moving to projects directory: /home/robbie/slots/18/ICMUAh7zg+001332
  19:33:22 STEP 1333 H= 333:15 +CPU= 21.606
  19:33:44 STEP 1334 H= 333:30 +CPU= 21.625
  19:34:05 STEP 1335 H= 333:45 +CPU= 21.592
  19:34:28 STEP 1336 H= 334:00 +CPU= 22.116
  19:34:49 STEP 1337 H= 334:15 +CPU= 21.560
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  19:35:51 STEP 1338 H= 334:30 +CPU= 19.577
  19:36:12 STEP 1339 H= 334:45 +CPU= 21.308
  19:36:34 STEP 1340 H= 335:00 +CPU= 21.863
  19:36:55 STEP 1341 H= 335:15 +CPU= 21.351
  19:37:17 STEP 1342 H= 335:30 +CPU= 21.647
  19:37:39 STEP 1343 H= 335:45 +CPU= 21.598
  19:38:11 STEP 1344 H= 336:00 +CPU= 30.789
Moving to projects directory: /home/robbie/slots/18/ICMGGh7zg+001344
Moving to projects directory: /home/robbie/slots/18/ICMSHh7zg+001344
Moving to projects directory: /home/robbie/slots/18/ICMUAh7zg+001344
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMGGh7zg+001248
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMSHh7zg+001248
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMUAh7zg+001248
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMGGh7zg+001260
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMSHh7zg+001260
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMUAh7zg+001260
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMGGh7zg+001272
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMSHh7zg+001272
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMUAh7zg+001272
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMGGh7zg+001284
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMSHh7zg+001284
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMUAh7zg+001284
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMGGh7zg+001296
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMSHh7zg+001296
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMUAh7zg+001296
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMGGh7zg+001308
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMSHh7zg+001308
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMUAh7zg+001308
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMGGh7zg+001320
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMSHh7zg+001320
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMUAh7zg+001320
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMGGh7zg+001332
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMSHh7zg+001332
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMUAh7zg+001332
Zipping up the intermediate file: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_a10n_2016092300_20_1017_12283146_1_r906428035_13.zip
  19:38:32 STEP 1345 H= 336:15 +CPU= 21.605
Uploading the intermediate file: upload_file_13.zip
  19:38:56 STEP 1346 H= 336:30 +CPU= 21.710
  19:39:18 STEP 1347 H= 336:45 +CPU= 21.379
  19:39:40 STEP 1348 H= 337:00 +CPU= 22.208
  19:40:01 STEP 1349 H= 337:15 +CPU= 21.486
  19:40:23 STEP 1350 H= 337:30 +CPU= 21.592
  19:40:44 STEP 1351 H= 337:45 +CPU= 21.471
  19:41:06 STEP 1352 H= 338:00 +CPU= 22.002
  19:41:28 STEP 1353 H= 338:15 +CPU= 21.434
  19:41:49 STEP 1354 H= 338:30 +CPU= 21.389
  19:42:11 STEP 1355 H= 338:45 +CPU= 21.401
  19:42:37 STEP 1356 H= 339:00 +CPU= 26.603
Moving to projects directory: /home/robbie/slots/18/ICMGGh7zg+001356
Moving to projects directory: /home/robbie/slots/18/ICMSHh7zg+001356
Moving to projects directory: /home/robbie/slots/18/ICMUAh7zg+001356
  19:42:59 STEP 1357 H= 339:15 +CPU= 21.309
  19:43:20 STEP 1358 H= 339:30 +CPU= 21.539
  19:43:42 STEP 1359 H= 339:45 +CPU= 21.542
  19:44:04 STEP 1360 H= 340:00 +CPU= 21.834
  19:44:25 STEP 1361 H= 340:15 +CPU= 21.549
  19:44:47 STEP 1362 H= 340:30 +CPU= 21.404
  19:45:08 STEP 1363 H= 340:45 +CPU= 21.440
  19:45:30 STEP 1364 H= 341:00 +CPU= 21.939
  19:45:52 STEP 1365 H= 341:15 +CPU= 21.435
  19:46:13 STEP 1366 H= 341:30 +CPU= 21.426
  19:46:34 STEP 1367 H= 341:45 +CPU= 21.429
  19:47:01 STEP 1368 H= 342:00 +CPU= 26.719
Moving to projects directory: /home/robbie/slots/18/ICMGGh7zg+001368
Moving to projects directory: /home/robbie/slots/18/ICMSHh7zg+001368
Moving to projects directory: /home/robbie/slots/18/ICMUAh7zg+001368
  19:47:23 STEP 1369 H= 342:15 +CPU= 21.630
  19:47:44 STEP 1370 H= 342:30 +CPU= 21.544
  19:48:06 STEP 1371 H= 342:45 +CPU= 21.397
  19:48:28 STEP 1372 H= 343:00 +CPU= 22.138
  19:48:50 STEP 1373 H= 343:15 +CPU= 21.631
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  19:49:32 STEP 1374 H= 343:30 +CPU= 21.165
  19:49:54 STEP 1375 H= 343:45 +CPU= 21.608
  19:50:16 STEP 1376 H= 344:00 +CPU= 21.723
  19:50:37 STEP 1377 H= 344:15 +CPU= 21.459
  19:50:59 STEP 1378 H= 344:30 +CPU= 21.405
  19:51:20 STEP 1379 H= 344:45 +CPU= 21.430
  19:51:47 STEP 1380 H= 345:00 +CPU= 26.616
Moving to projects directory: /home/robbie/slots/18/ICMGGh7zg+001380
Moving to projects directory: /home/robbie/slots/18/ICMSHh7zg+001380
Moving to projects directory: /home/robbie/slots/18/ICMUAh7zg+001380
  19:52:08 STEP 1381 H= 345:15 +CPU= 21.511
  19:52:30 STEP 1382 H= 345:30 +CPU= 21.403
  19:52:51 STEP 1383 H= 345:45 +CPU= 21.422
  19:53:13 STEP 1384 H= 346:00 +CPU= 21.895
  19:53:34 STEP 1385 H= 346:15 +CPU= 21.475
  19:53:56 STEP 1386 H= 346:30 +CPU= 21.639
  19:54:18 STEP 1387 H= 346:45 +CPU= 21.424
  19:54:39 STEP 1388 H= 347:00 +CPU= 21.852
  19:55:01 STEP 1389 H= 347:15 +CPU= 21.466
  19:55:22 STEP 1390 H= 347:30 +CPU= 21.349
  19:55:44 STEP 1391 H= 347:45 +CPU= 21.517
  19:56:15 STEP 1392 H= 348:00 +CPU= 30.709
Moving to projects directory: /home/robbie/slots/18/ICMGGh7zg+001392
Moving to projects directory: /home/robbie/slots/18/ICMSHh7zg+001392
Moving to projects directory: /home/robbie/slots/18/ICMUAh7zg+001392
  19:56:36 STEP 1393 H= 348:15 +CPU= 21.495
  19:56:58 STEP 1394 H= 348:30 +CPU= 21.522
  19:57:19 STEP 1395 H= 348:45 +CPU= 21.426
  19:57:41 STEP 1396 H= 349:00 +CPU= 21.818
  19:58:03 STEP 1397 H= 349:15 +CPU= 21.451
  19:58:24 STEP 1398 H= 349:30 +CPU= 21.466
  19:58:46 STEP 1399 H= 349:45 +CPU= 21.470
  19:59:08 STEP 1400 H= 350:00 +CPU= 21.960
  19:59:29 STEP 1401 H= 350:15 +CPU= 21.538
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  20:00:10 STEP 1402 H= 350:30 +CPU= 20.332
  20:00:32 STEP 1403 H= 350:45 +CPU= 21.723
  20:00:59 STEP 1404 H= 351:00 +CPU= 26.840
Moving to projects directory: /home/robbie/slots/18/ICMGGh7zg+001404
Moving to projects directory: /home/robbie/slots/18/ICMSHh7zg+001404
Moving to projects directory: /home/robbie/slots/18/ICMUAh7zg+001404
  20:01:21 STEP 1405 H= 351:15 +CPU= 21.567
  20:01:42 STEP 1406 H= 351:30 +CPU= 21.514
  20:02:04 STEP 1407 H= 351:45 +CPU= 21.591
  20:02:26 STEP 1408 H= 352:00 +CPU= 21.919
  20:02:47 STEP 1409 H= 352:15 +CPU= 21.492
  20:03:09 STEP 1410 H= 352:30 +CPU= 21.540
  20:03:31 STEP 1411 H= 352:45 +CPU= 21.594
  20:03:53 STEP 1412 H= 353:00 +CPU= 22.102
  20:04:14 STEP 1413 H= 353:15 +CPU= 21.663
  20:04:36 STEP 1414 H= 353:30 +CPU= 21.590
  20:04:58 STEP 1415 H= 353:45 +CPU= 21.646
  20:05:25 STEP 1416 H= 354:00 +CPU= 26.887
Moving to projects directory: /home/robbie/slots/18/ICMGGh7zg+001416
Moving to projects directory: /home/robbie/slots/18/ICMSHh7zg+001416
Moving to projects directory: /home/robbie/slots/18/ICMUAh7zg+001416
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  20:06:06 STEP 1417 H= 354:15 +CPU= 20.405
  20:06:27 STEP 1418 H= 354:30 +CPU= 21.374
  20:06:49 STEP 1419 H= 354:45 +CPU= 21.339
  20:07:11 STEP 1420 H= 355:00 +CPU= 21.889
  20:07:32 STEP 1421 H= 355:15 +CPU= 21.674
  20:07:54 STEP 1422 H= 355:30 +CPU= 21.613
  20:08:15 STEP 1423 H= 355:45 +CPU= 21.498
  20:08:37 STEP 1424 H= 356:00 +CPU= 21.815
  20:08:59 STEP 1425 H= 356:15 +CPU= 21.439
  20:09:20 STEP 1426 H= 356:30 +CPU= 21.647
  20:09:42 STEP 1427 H= 356:45 +CPU= 21.582
  20:10:09 STEP 1428 H= 357:00 +CPU= 26.884
Moving to projects directory: /home/robbie/slots/18/ICMGGh7zg+001428
Moving to projects directory: /home/robbie/slots/18/ICMSHh7zg+001428
Moving to projects directory: /home/robbie/slots/18/ICMUAh7zg+001428
  20:10:31 STEP 1429 H= 357:15 +CPU= 21.590
  20:10:52 STEP 1430 H= 357:30 +CPU= 21.591
  20:11:14 STEP 1431 H= 357:45 +CPU= 21.483
  20:11:36 STEP 1432 H= 358:00 +CPU= 21.976
  20:11:57 STEP 1433 H= 358:15 +CPU= 21.410
  20:12:19 STEP 1434 H= 358:30 +CPU= 21.444
  20:12:40 STEP 1435 H= 358:45 +CPU= 21.642
  20:13:02 STEP 1436 H= 359:00 +CPU= 21.942
  20:13:24 STEP 1437 H= 359:15 +CPU= 21.571
  20:13:45 STEP 1438 H= 359:30 +CPU= 21.427
  20:14:07 STEP 1439 H= 359:45 +CPU= 21.371
  20:14:36 STEP 1440 H= 360:00 +CPU= 28.914
..The child process terminated with status: 0
>>> Printing last 70 lines from file: NODE.001_01

I   VARIABLE   I      Initial value         I        Current value        I
I-------------------------------------------------------------------------I
I  OMEGA (P/S) I     -0.7662921008E-21      I      -0.1621695100E-05      I
ICloud fractionI      0.0000000000E+00      I       0.4500980945E-01      I
I  Relat. Hum. I      0.5661153643E-01      I       0.1941811681E+00      I
I  PASS 01     I      0.0000000000E+00      I       0.4354816817E+03      I
DDH------------------------------------------------------------------------
  
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
 NSTEP =  1440 SCAN2M_HPOS  P
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ICMGGh7zg+001440 MODE=a
 IO-STREAM CLOSED - ICMGGh7zg+001440
 MPI-TASK:   1 -    312596468 BYTES IN      7 RECORDS TRANSFERRED IN    0.0010 SECONDS312596.5 Mbytes/s, TOTAL TIME=    0.0097(10.3%)
 20:14:36 STEP 1440 H= 360:00 +CPU= 28.914
 END CNT3
 NSTEP =  1440 CNT0   000000000  20.199  20.199  20.260   0.997               0               0               0               0               0
IO-STREAM STATISTICS, TOTAL NO OF REC/BYTES READ            0/           0 WRITTEN        43246/************
 -TASK-OPENED-OPEN-RECS IN -KBYTE IN    -RECS OUT-KBYTE OUT-WALL   -WALL IN-WALL
  OU-TOT IN -TOT OUT
   1    758    0        0            0    43246 ************   119.7     0.0   119.7     0.0   182.6
===-=== START OF TIMING STATISTICS ===-===
 
STATS FOR ALL TASKS
 NUM ROUTINE                                     CALLS  MEAN(ms)   MAX(ms)   FRAC(%)  UNBAL(%)
   0 CNT0     - COMPLETE EXECUTION                   1 ********* *********    100.00      0.00
   1 CNT4     - FORWARD INTEGRATION                  1 ********* *********     99.92      0.00
   8 SCAN2M - GRID-POINT DYNAMICS                 1562    3616.2    3616.2     17.17      0.00
   9 SPCM     - SPECTRAL COMP.                    1440     670.6     670.6      2.93      0.00
  10 SCAN2M - PHYSICS                             1441   10490.1   10490.1     45.94      0.00
  11 IOPACK   - OUTPUT P.P. RESULTS                121    1154.8    1154.8      0.42      0.00
  12 SPNORM   - SPECTRAL NORM COMP.                 63      28.5      28.5      0.01      0.00
  14 SUINIF                                          1   25001.7   25001.7      0.08      0.00
  17 GRIDFPOS IN CNT4                              121      22.7      22.7      0.01      0.00
  18 SUSPECG                                         1    2238.0    2238.0      0.01      0.00
  19 SUSPEC                                          1    2240.7    2240.7      0.01      0.00
  24 SUGRIDU                                         1   22502.0   22502.0      0.07      0.00
  25 SPECRT                                          1     183.2     183.2      0.00      0.00
  26 SUGRIDF                                         1      75.8      75.8      0.00      0.00
  27 RESTART FILES - WRITING                        30    4657.7    4657.7      0.42      0.00
  28 RESTART FILES - READING                         1       0.0       0.0      0.00      0.00
  29 SU4FPOS IN CNT4                               121       0.0       0.0      0.00      0.00
  30 DYNFPOS IN CNT4                               121    4723.9    4723.9      1.74      0.00
  31 POSDDH IN STEPO                                 7       2.1       2.1      0.00      0.00
  37 CPGLAG   - SL COMPUTATIONS                   1441    4869.6    4869.6     21.32      0.00
  39 SU0YOMB                                         1     344.8     344.8      0.00      0.00
  51 SCAN2M   - SL COMM. PART 1                   1441      21.4      21.4      0.09      0.00
  54 SPCM     - M TO S/S TO M TRANSP.             1440     356.7     356.7      1.56      0.00
  55 SPCIMPF  - S TO M/M TO S TRANSP.             1440      37.6      37.6      0.16      0.00
  56 SPNORM   - SPECTRAL NORM COMM.                 63       0.2       0.2      0.00      0.00
 102 LTINV_CTL   - INVERSE LEGENDRE TRANSFORM     3125     352.9     352.9      3.35      0.00
 103 LTDIR_CTL   - DIRECT LEGENDRE TRANSFORM      3006     216.3     216.3      1.98      0.00
 106 FTDIR_CTL   - DIRECT FOURIER TRANSFORM       3006      77.8      77.8      0.71      0.00
 107 FTINV_CTL   - INVERSE FOURIER TRANSFORM      3125     121.6     121.6      1.15      0.00
 140 SULEG       - COMP. OF LEGENDRE POL.            1      33.9      33.9      0.00      0.00
 152 LTINV_CTL   - M TO L TRANSPOSITION           3125      34.0      34.0      0.32      0.00
 153 LTDIR_CTL   - L TO M TRANSPOSITION           3006      33.4      33.4      0.31      0.00
 157 FTINV_CTL   - L TO G TRANSPOSITION           3125      77.3      77.3      0.73      0.00
 158 FTDIR_CTL   - G TO L TRANSPOSITION           3006      18.2      18.2      0.17      0.00
 400 GSTATS                                     167734       0.0       0.0      0.00      0.00
 401 GSTATS HOOK                                156066       0.0       0.0      0.00      0.00
TOTAL MEASURED IMBALANCE =       0.0 SECONDS,  0.0 PERCENT
TOTAL WALLCLOCK TIME    32906.7 CPU TIME   31109.6 VECTOR TIME    31109.6


===-=== END   OF TIMING STATISTICS ===-===
FORECAST DAYS PER DAY       39.4
  *** END CNT0 *** 
------------------------------------------------
Moving to projects directory: /home/robbie/slots/18/ICMGGh7zg+001440
Moving to projects directory: /home/robbie/slots/18/ICMSHh7zg+001440
Moving to projects directory: /home/robbie/slots/18/ICMUAh7zg+001440
Adding to the zip: /home/robbie/slots/18/NODE.001_01
Adding to the zip: /home/robbie/slots/18/ifs.stat
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMGGh7zg+001416
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMGGh7zg+001368
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMGGh7zg+001440
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMGGh7zg+001428
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMUAh7zg+001428
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMUAh7zg+001404
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMSHh7zg+001380
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMSHh7zg+001428
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMSHh7zg+001344
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMGGh7zg+001392
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMGGh7zg+001380
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMSHh7zg+001356
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMGGh7zg+001404
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMUAh7zg+001380
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMUAh7zg+001440
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMGGh7zg+001356
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMUAh7zg+001356
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMUAh7zg+001392
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMGGh7zg+001344
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMSHh7zg+001440
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMSHh7zg+001416
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMUAh7zg+001416
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMSHh7zg+001404
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMUAh7zg+001368
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMSHh7zg+001392
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMUAh7zg+001344
Adding to the zip: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_12283146/ICMSHh7zg+001368
Zipping up the final file: /home/robbie/projects/climateprediction.net/oifs_43r3_bl_a10n_2016092300_20_1017_12283146_1_r906428035_14.zip
Uploading the final file: upload_file_14.zip
Uploading trickle at timestep: 1295100
20:18:06 (2277): called boinc_finish(0)

</stderr_txt>
<message>
upload failure: <file_xfer_error>
  <file_name>oifs_43r3_bl_a10n_2016092300_20_1017_12283146_1_r906428035_15.zip</file_name>
  <error_code>-161 (not found)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>oifs_43r3_bl_a10n_2016092300_20_1017_12283146_1_r906428035_16.zip</file_name>
  <error_code>-161 (not found)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>oifs_43r3_bl_a10n_2016092300_20_1017_12283146_1_r906428035_17.zip</file_name>
  <error_code>-161 (not found)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>oifs_43r3_bl_a10n_2016092300_20_1017_12283146_1_r906428035_18.zip</file_name>
  <error_code>-161 (not found)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>oifs_43r3_bl_a10n_2016092300_20_1017_12283146_1_r906428035_19.zip</file_name>
  <error_code>-161 (not found)</error_code>
</file_xfer_error>
</message>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
08 Jun 2024 20:05:28 1548614 22440819 oifs_43r3_bl_a10n_2016092300_20_1017_12283146_1 1,295,100 31,109 0.0240
08 Jun 2024 17:08:27 1548614 22440819 oifs_43r3_bl_a10n_2016092300_20_1017_12283146_1 864,000 20,644 0.0239


©2024 climateprediction.net