climateprediction.net home page
Task 22436563

Task 22436563

Name oifs_43r3_bl_a2fd_2016092300_20_1017_12284972_0
Workunit 12284972
Created 7 Jun 2024, 15:23:36 UTC
Sent 8 Jun 2024, 1:04:54 UTC
Report deadline 7 Aug 2024, 1:04:54 UTC
Received 8 Jun 2024, 19:18:44 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 0 (0x00000000)
Computer ID 1507812
Run time 14 hours 29 min 10 sec
CPU time 12 hours 39 min 14 sec
Validate state Invalid
Credit 1,976.32
Device peak FLOPS 5.39 GFLOPS
Application version OpenIFS 43r3 Baroclinic Lifecycle v1.13
x86_64-pc-linux-gnu
Peak working set size 5,529.45 MB
Peak swap size 5,980.80 MB
Peak disk usage 2,092.74 MB
Stderr
<core_client_version>7.24.1</core_client_version>
<![CDATA[
<stderr_txt>
= 237:15 +CPU= 30.183
  15:36:59 STEP  950 H= 237:30 +CPU= 30.017
  15:37:36 STEP  951 H= 237:45 +CPU= 30.058
  15:38:09 STEP  952 H= 238:00 +CPU= 29.644
  15:38:44 STEP  953 H= 238:15 +CPU= 30.534
  15:39:20 STEP  954 H= 238:30 +CPU= 31.277
  15:39:53 STEP  955 H= 238:45 +CPU= 29.694
  15:40:28 STEP  956 H= 239:00 +CPU= 30.228
  15:41:01 STEP  957 H= 239:15 +CPU= 28.420
  15:41:33 STEP  958 H= 239:30 +CPU= 27.805
  15:42:05 STEP  959 H= 239:45 +CPU= 27.646
  15:43:13 STEP  960 H= 240:00 +CPU= 39.018
Moving to projects directory: /var/lib/boinc/slots/1/ICMGGh7zg+000960
Moving to projects directory: /var/lib/boinc/slots/1/ICMSHh7zg+000960
Moving to projects directory: /var/lib/boinc/slots/1/ICMUAh7zg+000960
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMGGh7zg+000864
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMSHh7zg+000864
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMUAh7zg+000864
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMGGh7zg+000876
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMSHh7zg+000876
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMUAh7zg+000876
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMGGh7zg+000888
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMSHh7zg+000888
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMUAh7zg+000888
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMGGh7zg+000900
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMSHh7zg+000900
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMUAh7zg+000900
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMGGh7zg+000912
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMSHh7zg+000912
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMUAh7zg+000912
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMGGh7zg+000924
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMSHh7zg+000924
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMUAh7zg+000924
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMGGh7zg+000936
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMSHh7zg+000936
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMUAh7zg+000936
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMGGh7zg+000948
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMSHh7zg+000948
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMUAh7zg+000948
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_a2fd_2016092300_20_1017_12284972_0_r2132692060_9.zip
Uploading the intermediate file: upload_file_9.zip
  15:43:51 STEP  961 H= 240:15 +CPU= 29.816
  15:44:25 STEP  962 H= 240:30 +CPU= 29.769
  15:44:58 STEP  963 H= 240:45 +CPU= 29.205
  15:45:34 STEP  964 H= 241:00 +CPU= 30.482
  15:46:09 STEP  965 H= 241:15 +CPU= 30.431
  15:46:46 STEP  966 H= 241:30 +CPU= 30.491
  15:47:22 STEP  967 H= 241:45 +CPU= 30.195
  15:47:56 STEP  968 H= 242:00 +CPU= 30.094
  15:48:30 STEP  969 H= 242:15 +CPU= 29.059
  15:49:04 STEP  970 H= 242:30 +CPU= 29.450
  15:49:42 STEP  971 H= 242:45 +CPU= 31.548
  15:50:29 STEP  972 H= 243:00 +CPU= 39.627
Moving to projects directory: /var/lib/boinc/slots/1/ICMGGh7zg+000972
Moving to projects directory: /var/lib/boinc/slots/1/ICMSHh7zg+000972
Moving to projects directory: /var/lib/boinc/slots/1/ICMUAh7zg+000972
  15:51:02 STEP  973 H= 243:15 +CPU= 29.326
  15:51:30 STEP  974 H= 243:30 +CPU= 25.689
  15:52:05 STEP  975 H= 243:45 +CPU= 28.972
  15:52:39 STEP  976 H= 244:00 +CPU= 29.625
  15:53:17 STEP  977 H= 244:15 +CPU= 32.133
  15:53:55 STEP  978 H= 244:30 +CPU= 32.489
  15:54:30 STEP  979 H= 244:45 +CPU= 30.547
  15:55:02 STEP  980 H= 245:00 +CPU= 29.564
  15:55:34 STEP  981 H= 245:15 +CPU= 28.662
  15:56:06 STEP  982 H= 245:30 +CPU= 27.864
  15:56:40 STEP  983 H= 245:45 +CPU= 28.451
  15:57:22 STEP  984 H= 246:00 +CPU= 34.668
Moving to projects directory: /var/lib/boinc/slots/1/ICMGGh7zg+000984
Moving to projects directory: /var/lib/boinc/slots/1/ICMSHh7zg+000984
Moving to projects directory: /var/lib/boinc/slots/1/ICMUAh7zg+000984
  15:57:54 STEP  985 H= 246:15 +CPU= 27.922
  15:58:28 STEP  986 H= 246:30 +CPU= 29.078
  15:59:01 STEP  987 H= 246:45 +CPU= 28.250
  15:59:38 STEP  988 H= 247:00 +CPU= 30.470
  16:00:14 STEP  989 H= 247:15 +CPU= 30.143
  16:00:48 STEP  990 H= 247:30 +CPU= 29.218
  16:01:24 STEP  991 H= 247:45 +CPU= 29.549
  16:01:55 STEP  992 H= 248:00 +CPU= 28.361
  16:02:29 STEP  993 H= 248:15 +CPU= 29.052
  16:03:02 STEP  994 H= 248:30 +CPU= 28.212
  16:03:36 STEP  995 H= 248:45 +CPU= 28.741
  16:04:21 STEP  996 H= 249:00 +CPU= 36.834
Moving to projects directory: /var/lib/boinc/slots/1/ICMGGh7zg+000996
Moving to projects directory: /var/lib/boinc/slots/1/ICMSHh7zg+000996
Moving to projects directory: /var/lib/boinc/slots/1/ICMUAh7zg+000996
  16:04:57 STEP  997 H= 249:15 +CPU= 30.003
  16:05:31 STEP  998 H= 249:30 +CPU= 30.105
  16:06:07 STEP  999 H= 249:45 +CPU= 29.343
  16:06:45 STEP 1000 H= 250:00 +CPU= 30.655
  16:07:21 STEP 1001 H= 250:15 +CPU= 29.658
  16:07:55 STEP 1002 H= 250:30 +CPU= 29.181
  16:08:29 STEP 1003 H= 250:45 +CPU= 28.221
  16:09:03 STEP 1004 H= 251:00 +CPU= 28.469
  16:09:38 STEP 1005 H= 251:15 +CPU= 28.584
  16:10:09 STEP 1006 H= 251:30 +CPU= 26.762
  16:10:42 STEP 1007 H= 251:45 +CPU= 28.043
  16:12:00 STEP 1008 H= 252:00 +CPU= 38.926
Moving to projects directory: /var/lib/boinc/slots/1/ICMGGh7zg+001008
Moving to projects directory: /var/lib/boinc/slots/1/ICMSHh7zg+001008
Moving to projects directory: /var/lib/boinc/slots/1/ICMUAh7zg+001008
  16:12:36 STEP 1009 H= 252:15 +CPU= 29.425
  16:13:11 STEP 1010 H= 252:30 +CPU= 29.453
  16:13:48 STEP 1011 H= 252:45 +CPU= 29.579
  16:14:21 STEP 1012 H= 253:00 +CPU= 28.146
  16:14:53 STEP 1013 H= 253:15 +CPU= 27.825
  16:15:26 STEP 1014 H= 253:30 +CPU= 27.956
  16:15:58 STEP 1015 H= 253:45 +CPU= 27.867
  16:16:31 STEP 1016 H= 254:00 +CPU= 28.142
  16:17:04 STEP 1017 H= 254:15 +CPU= 28.180
  16:17:39 STEP 1018 H= 254:30 +CPU= 29.420
  16:18:13 STEP 1019 H= 254:45 +CPU= 29.395
  16:18:55 STEP 1020 H= 255:00 +CPU= 35.718
Moving to projects directory: /var/lib/boinc/slots/1/ICMGGh7zg+001020
Moving to projects directory: /var/lib/boinc/slots/1/ICMSHh7zg+001020
Moving to projects directory: /var/lib/boinc/slots/1/ICMUAh7zg+001020
  16:19:30 STEP 1021 H= 255:15 +CPU= 28.797
  16:20:05 STEP 1022 H= 255:30 +CPU= 30.116
  16:20:41 STEP 1023 H= 255:45 +CPU= 28.755
  16:21:17 STEP 1024 H= 256:00 +CPU= 29.230
  16:21:51 STEP 1025 H= 256:15 +CPU= 28.479
  16:22:25 STEP 1026 H= 256:30 +CPU= 28.739
  16:22:57 STEP 1027 H= 256:45 +CPU= 27.917
  16:23:32 STEP 1028 H= 257:00 +CPU= 30.117
  16:24:08 STEP 1029 H= 257:15 +CPU= 29.670
  16:24:44 STEP 1030 H= 257:30 +CPU= 29.548
  16:25:19 STEP 1031 H= 257:45 +CPU= 29.707
  16:26:03 STEP 1032 H= 258:00 +CPU= 37.137
Moving to projects directory: /var/lib/boinc/slots/1/ICMGGh7zg+001032
Moving to projects directory: /var/lib/boinc/slots/1/ICMSHh7zg+001032
Moving to projects directory: /var/lib/boinc/slots/1/ICMUAh7zg+001032
  16:26:38 STEP 1033 H= 258:15 +CPU= 28.762
  16:27:15 STEP 1034 H= 258:30 +CPU= 30.565
  16:27:50 STEP 1035 H= 258:45 +CPU= 28.177
  16:28:25 STEP 1036 H= 259:00 +CPU= 29.909
  16:28:58 STEP 1037 H= 259:15 +CPU= 27.489
  16:29:32 STEP 1038 H= 259:30 +CPU= 28.491
  16:30:05 STEP 1039 H= 259:45 +CPU= 27.866
  16:30:40 STEP 1040 H= 260:00 +CPU= 28.852
  16:31:16 STEP 1041 H= 260:15 +CPU= 30.117
  16:31:49 STEP 1042 H= 260:30 +CPU= 28.144
  16:32:24 STEP 1043 H= 260:45 +CPU= 30.066
  16:33:07 STEP 1044 H= 261:00 +CPU= 37.256
Moving to projects directory: /var/lib/boinc/slots/1/ICMGGh7zg+001044
Moving to projects directory: /var/lib/boinc/slots/1/ICMSHh7zg+001044
Moving to projects directory: /var/lib/boinc/slots/1/ICMUAh7zg+001044
  16:33:43 STEP 1045 H= 261:15 +CPU= 29.643
  16:34:17 STEP 1046 H= 261:30 +CPU= 29.684
  16:34:51 STEP 1047 H= 261:45 +CPU= 29.079
  16:35:25 STEP 1048 H= 262:00 +CPU= 29.876
  16:35:58 STEP 1049 H= 262:15 +CPU= 28.192
  16:36:32 STEP 1050 H= 262:30 +CPU= 28.928
  16:37:05 STEP 1051 H= 262:45 +CPU= 28.659
  16:37:39 STEP 1052 H= 263:00 +CPU= 28.950
  16:38:11 STEP 1053 H= 263:15 +CPU= 27.661
  16:38:42 STEP 1054 H= 263:30 +CPU= 26.818
  16:39:18 STEP 1055 H= 263:45 +CPU= 29.857
  16:40:23 STEP 1056 H= 264:00 +CPU= 34.039
Moving to projects directory: /var/lib/boinc/slots/1/ICMGGh7zg+001056
Moving to projects directory: /var/lib/boinc/slots/1/ICMSHh7zg+001056
Moving to projects directory: /var/lib/boinc/slots/1/ICMUAh7zg+001056
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMGGh7zg+000960
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMSHh7zg+000960
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMUAh7zg+000960
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMGGh7zg+000972
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMSHh7zg+000972
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMUAh7zg+000972
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMGGh7zg+000984
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMSHh7zg+000984
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMUAh7zg+000984
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMGGh7zg+000996
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMSHh7zg+000996
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMUAh7zg+000996
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMGGh7zg+001008
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMSHh7zg+001008
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMUAh7zg+001008
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMGGh7zg+001020
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMSHh7zg+001020
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMUAh7zg+001020
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMGGh7zg+001032
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMSHh7zg+001032
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMUAh7zg+001032
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMGGh7zg+001044
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMSHh7zg+001044
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMUAh7zg+001044
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_a2fd_2016092300_20_1017_12284972_0_r2132692060_10.zip
Uploading the intermediate file: upload_file_10.zip
  16:40:59 STEP 1057 H= 264:15 +CPU= 28.804
  16:41:33 STEP 1058 H= 264:30 +CPU= 27.632
  16:42:04 STEP 1059 H= 264:45 +CPU= 27.790
  16:42:36 STEP 1060 H= 265:00 +CPU= 28.397
  16:43:08 STEP 1061 H= 265:15 +CPU= 27.917
  16:43:40 STEP 1062 H= 265:30 +CPU= 27.906
  16:44:11 STEP 1063 H= 265:45 +CPU= 27.980
  16:44:45 STEP 1064 H= 266:00 +CPU= 29.979
  16:45:19 STEP 1065 H= 266:15 +CPU= 29.308
  16:45:52 STEP 1066 H= 266:30 +CPU= 29.629
  16:46:26 STEP 1067 H= 266:45 +CPU= 29.730
  16:47:09 STEP 1068 H= 267:00 +CPU= 37.673
Moving to projects directory: /var/lib/boinc/slots/1/ICMGGh7zg+001068
Moving to projects directory: /var/lib/boinc/slots/1/ICMSHh7zg+001068
Moving to projects directory: /var/lib/boinc/slots/1/ICMUAh7zg+001068
  16:47:40 STEP 1069 H= 267:15 +CPU= 26.682
  16:48:09 STEP 1070 H= 267:30 +CPU= 26.027
  16:48:42 STEP 1071 H= 267:45 +CPU= 28.383
  16:49:16 STEP 1072 H= 268:00 +CPU= 29.100
  16:49:49 STEP 1073 H= 268:15 +CPU= 28.838
  16:50:24 STEP 1074 H= 268:30 +CPU= 29.418
  16:50:58 STEP 1075 H= 268:45 +CPU= 29.823
  16:51:31 STEP 1076 H= 269:00 +CPU= 29.701
  16:52:07 STEP 1077 H= 269:15 +CPU= 30.180
  16:52:41 STEP 1078 H= 269:30 +CPU= 29.515
  16:53:14 STEP 1079 H= 269:45 +CPU= 29.920
  16:53:54 STEP 1080 H= 270:00 +CPU= 35.651
Moving to projects directory: /var/lib/boinc/slots/1/ICMGGh7zg+001080
Moving to projects directory: /var/lib/boinc/slots/1/ICMSHh7zg+001080
Moving to projects directory: /var/lib/boinc/slots/1/ICMUAh7zg+001080
  16:54:26 STEP 1081 H= 270:15 +CPU= 28.166
  16:55:00 STEP 1082 H= 270:30 +CPU= 29.257
  16:55:33 STEP 1083 H= 270:45 +CPU= 29.290
  16:56:08 STEP 1084 H= 271:00 +CPU= 30.082
  16:56:44 STEP 1085 H= 271:15 +CPU= 29.515
  16:57:18 STEP 1086 H= 271:30 +CPU= 29.457
  16:57:51 STEP 1087 H= 271:45 +CPU= 29.499
  16:58:26 STEP 1088 H= 272:00 +CPU= 30.524
  16:59:01 STEP 1089 H= 272:15 +CPU= 30.166
  16:59:37 STEP 1090 H= 272:30 +CPU= 30.375
  17:00:10 STEP 1091 H= 272:45 +CPU= 29.340
  17:00:51 STEP 1092 H= 273:00 +CPU= 36.498
Moving to projects directory: /var/lib/boinc/slots/1/ICMGGh7zg+001092
Moving to projects directory: /var/lib/boinc/slots/1/ICMSHh7zg+001092
Moving to projects directory: /var/lib/boinc/slots/1/ICMUAh7zg+001092
  17:01:24 STEP 1093 H= 273:15 +CPU= 28.687
  17:01:59 STEP 1094 H= 273:30 +CPU= 29.377
  17:02:33 STEP 1095 H= 273:45 +CPU= 29.340
  17:03:08 STEP 1096 H= 274:00 +CPU= 29.821
  17:03:42 STEP 1097 H= 274:15 +CPU= 29.316
  17:04:17 STEP 1098 H= 274:30 +CPU= 29.869
  17:04:52 STEP 1099 H= 274:45 +CPU= 30.131
  17:05:28 STEP 1100 H= 275:00 +CPU= 30.577
  17:06:01 STEP 1101 H= 275:15 +CPU= 28.901
  17:06:36 STEP 1102 H= 275:30 +CPU= 29.131
  17:07:10 STEP 1103 H= 275:45 +CPU= 29.527
  17:08:10 STEP 1104 H= 276:00 +CPU= 37.714
Moving to projects directory: /var/lib/boinc/slots/1/ICMGGh7zg+001104
Moving to projects directory: /var/lib/boinc/slots/1/ICMSHh7zg+001104
Moving to projects directory: /var/lib/boinc/slots/1/ICMUAh7zg+001104
  17:08:41 STEP 1105 H= 276:15 +CPU= 27.082
  17:09:14 STEP 1106 H= 276:30 +CPU= 29.181
  17:09:48 STEP 1107 H= 276:45 +CPU= 29.270
  17:10:22 STEP 1108 H= 277:00 +CPU= 29.881
  17:10:55 STEP 1109 H= 277:15 +CPU= 29.508
  17:11:30 STEP 1110 H= 277:30 +CPU= 29.694
  17:12:06 STEP 1111 H= 277:45 +CPU= 30.151
  17:12:43 STEP 1112 H= 278:00 +CPU= 31.119
  17:13:17 STEP 1113 H= 278:15 +CPU= 29.110
  17:13:50 STEP 1114 H= 278:30 +CPU= 28.931
  17:14:26 STEP 1115 H= 278:45 +CPU= 29.332
  17:15:09 STEP 1116 H= 279:00 +CPU= 36.085
Moving to projects directory: /var/lib/boinc/slots/1/ICMGGh7zg+001116
Moving to projects directory: /var/lib/boinc/slots/1/ICMSHh7zg+001116
Moving to projects directory: /var/lib/boinc/slots/1/ICMUAh7zg+001116
  17:15:44 STEP 1117 H= 279:15 +CPU= 29.233
  17:16:15 STEP 1118 H= 279:30 +CPU= 27.487
  17:16:52 STEP 1119 H= 279:45 +CPU= 29.756
  17:17:30 STEP 1120 H= 280:00 +CPU= 30.198
  17:18:06 STEP 1121 H= 280:15 +CPU= 29.826
  17:18:41 STEP 1122 H= 280:30 +CPU= 30.228
  17:19:18 STEP 1123 H= 280:45 +CPU= 30.566
  17:19:54 STEP 1124 H= 281:00 +CPU= 30.299
  17:20:30 STEP 1125 H= 281:15 +CPU= 29.498
  17:21:03 STEP 1126 H= 281:30 +CPU= 28.778
  17:21:37 STEP 1127 H= 281:45 +CPU= 29.041
  17:22:20 STEP 1128 H= 282:00 +CPU= 36.325
Moving to projects directory: /var/lib/boinc/slots/1/ICMGGh7zg+001128
Moving to projects directory: /var/lib/boinc/slots/1/ICMSHh7zg+001128
Moving to projects directory: /var/lib/boinc/slots/1/ICMUAh7zg+001128
  17:22:55 STEP 1129 H= 282:15 +CPU= 29.275
  17:23:32 STEP 1130 H= 282:30 +CPU= 29.424
  17:24:05 STEP 1131 H= 282:45 +CPU= 29.257
  17:24:39 STEP 1132 H= 283:00 +CPU= 30.094
  17:25:15 STEP 1133 H= 283:15 +CPU= 29.964
  17:25:50 STEP 1134 H= 283:30 +CPU= 29.973
  17:26:24 STEP 1135 H= 283:45 +CPU= 29.341
  17:26:57 STEP 1136 H= 284:00 +CPU= 28.989
  17:27:32 STEP 1137 H= 284:15 +CPU= 28.945
  17:28:07 STEP 1138 H= 284:30 +CPU= 28.992
  17:28:42 STEP 1139 H= 284:45 +CPU= 29.103
  17:29:26 STEP 1140 H= 285:00 +CPU= 36.580
Moving to projects directory: /var/lib/boinc/slots/1/ICMGGh7zg+001140
Moving to projects directory: /var/lib/boinc/slots/1/ICMSHh7zg+001140
Moving to projects directory: /var/lib/boinc/slots/1/ICMUAh7zg+001140
  17:30:02 STEP 1141 H= 285:15 +CPU= 29.811
  17:30:39 STEP 1142 H= 285:30 +CPU= 29.241
  17:31:13 STEP 1143 H= 285:45 +CPU= 29.306
  17:31:49 STEP 1144 H= 286:00 +CPU= 30.413
  17:32:27 STEP 1145 H= 286:15 +CPU= 30.703
  17:33:03 STEP 1146 H= 286:30 +CPU= 30.280
  17:33:39 STEP 1147 H= 286:45 +CPU= 29.547
  17:34:16 STEP 1148 H= 287:00 +CPU= 29.639
  17:34:49 STEP 1149 H= 287:15 +CPU= 27.917
  17:35:24 STEP 1150 H= 287:30 +CPU= 28.501
  17:35:59 STEP 1151 H= 287:45 +CPU= 28.533
  17:37:09 STEP 1152 H= 288:00 +CPU= 38.477
Moving to projects directory: /var/lib/boinc/slots/1/ICMGGh7zg+001152
Moving to projects directory: /var/lib/boinc/slots/1/ICMSHh7zg+001152
Moving to projects directory: /var/lib/boinc/slots/1/ICMUAh7zg+001152
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMGGh7zg+001056
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMSHh7zg+001056
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMUAh7zg+001056
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMGGh7zg+001068
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMSHh7zg+001068
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMUAh7zg+001068
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMGGh7zg+001080
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMSHh7zg+001080
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMUAh7zg+001080
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMGGh7zg+001092
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMSHh7zg+001092
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMUAh7zg+001092
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMGGh7zg+001104
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMSHh7zg+001104
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMUAh7zg+001104
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMGGh7zg+001116
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMSHh7zg+001116
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMUAh7zg+001116
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMGGh7zg+001128
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMSHh7zg+001128
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMUAh7zg+001128
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMGGh7zg+001140
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMSHh7zg+001140
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMUAh7zg+001140
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_a2fd_2016092300_20_1017_12284972_0_r2132692060_11.zip
  17:37:36 STEP 1153 H= 288:15 +CPU= 24.741
Uploading the intermediate file: upload_file_11.zip
  17:38:11 STEP 1154 H= 288:30 +CPU= 29.109
  17:38:48 STEP 1155 H= 288:45 +CPU= 29.337
  17:39:25 STEP 1156 H= 289:00 +CPU= 31.084
  17:40:01 STEP 1157 H= 289:15 +CPU= 29.784
  17:40:37 STEP 1158 H= 289:30 +CPU= 29.451
  17:41:11 STEP 1159 H= 289:45 +CPU= 29.372
  17:41:47 STEP 1160 H= 290:00 +CPU= 29.921
  17:42:21 STEP 1161 H= 290:15 +CPU= 29.277
  17:42:56 STEP 1162 H= 290:30 +CPU= 29.121
  17:43:32 STEP 1163 H= 290:45 +CPU= 29.633
  17:44:18 STEP 1164 H= 291:00 +CPU= 36.963
Moving to projects directory: /var/lib/boinc/slots/1/ICMGGh7zg+001164
Moving to projects directory: /var/lib/boinc/slots/1/ICMSHh7zg+001164
Moving to projects directory: /var/lib/boinc/slots/1/ICMUAh7zg+001164
  17:44:54 STEP 1165 H= 291:15 +CPU= 30.058
  17:45:27 STEP 1166 H= 291:30 +CPU= 28.423
  17:46:03 STEP 1167 H= 291:45 +CPU= 30.514
  17:46:40 STEP 1168 H= 292:00 +CPU= 30.638
  17:47:13 STEP 1169 H= 292:15 +CPU= 28.851
  17:47:47 STEP 1170 H= 292:30 +CPU= 29.539
  17:48:21 STEP 1171 H= 292:45 +CPU= 29.011
  17:48:58 STEP 1172 H= 293:00 +CPU= 30.179
  17:49:32 STEP 1173 H= 293:15 +CPU= 29.555
  17:50:08 STEP 1174 H= 293:30 +CPU= 29.394
  17:50:43 STEP 1175 H= 293:45 +CPU= 29.698
  17:51:26 STEP 1176 H= 294:00 +CPU= 36.651
Moving to projects directory: /var/lib/boinc/slots/1/ICMGGh7zg+001176
Moving to projects directory: /var/lib/boinc/slots/1/ICMSHh7zg+001176
Moving to projects directory: /var/lib/boinc/slots/1/ICMUAh7zg+001176
  17:52:03 STEP 1177 H= 294:15 +CPU= 30.432
  17:52:39 STEP 1178 H= 294:30 +CPU= 30.661
  17:53:14 STEP 1179 H= 294:45 +CPU= 30.210
  17:53:50 STEP 1180 H= 295:00 +CPU= 30.095
  17:54:25 STEP 1181 H= 295:15 +CPU= 29.761
  17:55:00 STEP 1182 H= 295:30 +CPU= 29.406
  17:55:35 STEP 1183 H= 295:45 +CPU= 29.769
  17:56:11 STEP 1184 H= 296:00 +CPU= 29.843
  17:56:48 STEP 1185 H= 296:15 +CPU= 29.415
  17:57:23 STEP 1186 H= 296:30 +CPU= 29.289
  17:57:59 STEP 1187 H= 296:45 +CPU= 29.818
  17:58:45 STEP 1188 H= 297:00 +CPU= 37.533
Moving to projects directory: /var/lib/boinc/slots/1/ICMGGh7zg+001188
Moving to projects directory: /var/lib/boinc/slots/1/ICMSHh7zg+001188
Moving to projects directory: /var/lib/boinc/slots/1/ICMUAh7zg+001188
  17:59:22 STEP 1189 H= 297:15 +CPU= 30.603
  17:59:59 STEP 1190 H= 297:30 +CPU= 30.710
  18:00:35 STEP 1191 H= 297:45 +CPU= 29.596
  18:01:13 STEP 1192 H= 298:00 +CPU= 30.071
  18:01:49 STEP 1193 H= 298:15 +CPU= 29.398
  18:02:24 STEP 1194 H= 298:30 +CPU= 29.358
  18:03:01 STEP 1195 H= 298:45 +CPU= 29.471
  18:03:39 STEP 1196 H= 299:00 +CPU= 29.897
  18:04:14 STEP 1197 H= 299:15 +CPU= 29.206
  18:04:50 STEP 1198 H= 299:30 +CPU= 29.724
  18:05:27 STEP 1199 H= 299:45 +CPU= 30.116
  18:06:31 STEP 1200 H= 300:00 +CPU= 40.945
Moving to projects directory: /var/lib/boinc/slots/1/ICMGGh7zg+001200
Moving to projects directory: /var/lib/boinc/slots/1/ICMSHh7zg+001200
Moving to projects directory: /var/lib/boinc/slots/1/ICMUAh7zg+001200
  18:07:04 STEP 1201 H= 300:15 +CPU= 28.138
  18:07:34 STEP 1202 H= 300:30 +CPU= 26.451
  18:08:08 STEP 1203 H= 300:45 +CPU= 29.053
  18:08:45 STEP 1204 H= 301:00 +CPU= 30.087
  18:09:20 STEP 1205 H= 301:15 +CPU= 29.617
  18:09:56 STEP 1206 H= 301:30 +CPU= 29.686
  18:10:32 STEP 1207 H= 301:45 +CPU= 29.757
  18:11:09 STEP 1208 H= 302:00 +CPU= 30.331
  18:11:47 STEP 1209 H= 302:15 +CPU= 30.109
  18:12:23 STEP 1210 H= 302:30 +CPU= 30.384
  18:13:01 STEP 1211 H= 302:45 +CPU= 30.617
  18:13:49 STEP 1212 H= 303:00 +CPU= 37.992
Moving to projects directory: /var/lib/boinc/slots/1/ICMGGh7zg+001212
Moving to projects directory: /var/lib/boinc/slots/1/ICMSHh7zg+001212
Moving to projects directory: /var/lib/boinc/slots/1/ICMUAh7zg+001212
  18:14:27 STEP 1213 H= 303:15 +CPU= 29.553
  18:15:02 STEP 1214 H= 303:30 +CPU= 28.252
  18:15:36 STEP 1215 H= 303:45 +CPU= 28.085
  18:16:13 STEP 1216 H= 304:00 +CPU= 30.144
  18:16:51 STEP 1217 H= 304:15 +CPU= 29.457
  18:17:27 STEP 1218 H= 304:30 +CPU= 29.758
  18:18:05 STEP 1219 H= 304:45 +CPU= 30.110
  18:18:41 STEP 1220 H= 305:00 +CPU= 30.531
  18:19:17 STEP 1221 H= 305:15 +CPU= 30.138
  18:19:52 STEP 1222 H= 305:30 +CPU= 30.129
  18:20:28 STEP 1223 H= 305:45 +CPU= 30.860
  18:21:12 STEP 1224 H= 306:00 +CPU= 37.761
Moving to projects directory: /var/lib/boinc/slots/1/ICMGGh7zg+001224
Moving to projects directory: /var/lib/boinc/slots/1/ICMSHh7zg+001224
Moving to projects directory: /var/lib/boinc/slots/1/ICMUAh7zg+001224
  18:21:47 STEP 1225 H= 306:15 +CPU= 29.511
  18:22:22 STEP 1226 H= 306:30 +CPU= 29.588
  18:22:56 STEP 1227 H= 306:45 +CPU= 29.506
  18:23:29 STEP 1228 H= 307:00 +CPU= 29.781
  18:24:02 STEP 1229 H= 307:15 +CPU= 29.529
  18:24:37 STEP 1230 H= 307:30 +CPU= 29.845
  18:25:10 STEP 1231 H= 307:45 +CPU= 29.782
  18:25:44 STEP 1232 H= 308:00 +CPU= 30.460
  18:26:18 STEP 1233 H= 308:15 +CPU= 30.089
  18:26:55 STEP 1234 H= 308:30 +CPU= 30.841
  18:27:31 STEP 1235 H= 308:45 +CPU= 29.730
  18:28:14 STEP 1236 H= 309:00 +CPU= 37.126
Moving to projects directory: /var/lib/boinc/slots/1/ICMGGh7zg+001236
Moving to projects directory: /var/lib/boinc/slots/1/ICMSHh7zg+001236
Moving to projects directory: /var/lib/boinc/slots/1/ICMUAh7zg+001236
  18:28:48 STEP 1237 H= 309:15 +CPU= 29.546
  18:29:23 STEP 1238 H= 309:30 +CPU= 29.655
  18:29:57 STEP 1239 H= 309:45 +CPU= 29.493
  18:30:31 STEP 1240 H= 310:00 +CPU= 29.983
  18:31:05 STEP 1241 H= 310:15 +CPU= 29.745
  18:31:40 STEP 1242 H= 310:30 +CPU= 29.676
  18:32:14 STEP 1243 H= 310:45 +CPU= 30.158
  18:32:51 STEP 1244 H= 311:00 +CPU= 30.906
  18:33:23 STEP 1245 H= 311:15 +CPU= 28.969
  18:33:57 STEP 1246 H= 311:30 +CPU= 29.670
  18:34:33 STEP 1247 H= 311:45 +CPU= 29.177
  18:35:32 STEP 1248 H= 312:00 +CPU= 39.336
Moving to projects directory: /var/lib/boinc/slots/1/ICMGGh7zg+001248
Moving to projects directory: /var/lib/boinc/slots/1/ICMSHh7zg+001248
Moving to projects directory: /var/lib/boinc/slots/1/ICMUAh7zg+001248
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMGGh7zg+001152
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMSHh7zg+001152
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMUAh7zg+001152
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMGGh7zg+001164
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMSHh7zg+001164
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMUAh7zg+001164
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMGGh7zg+001176
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMSHh7zg+001176
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMUAh7zg+001176
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMGGh7zg+001188
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMSHh7zg+001188
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMUAh7zg+001188
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMGGh7zg+001200
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMSHh7zg+001200
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMUAh7zg+001200
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMGGh7zg+001212
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMSHh7zg+001212
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMUAh7zg+001212
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMGGh7zg+001224
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMSHh7zg+001224
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMUAh7zg+001224
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMGGh7zg+001236
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMSHh7zg+001236
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMUAh7zg+001236
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_a2fd_2016092300_20_1017_12284972_0_r2132692060_12.zip
  18:36:06 STEP 1249 H= 312:15 +CPU= 28.415
Uploading the intermediate file: upload_file_12.zip
  18:36:38 STEP 1250 H= 312:30 +CPU= 27.921
  18:37:13 STEP 1251 H= 312:45 +CPU= 29.036
  18:37:50 STEP 1252 H= 313:00 +CPU= 29.990
  18:38:24 STEP 1253 H= 313:15 +CPU= 29.744
  18:39:00 STEP 1254 H= 313:30 +CPU= 30.104
  18:39:36 STEP 1255 H= 313:45 +CPU= 30.614
  18:40:13 STEP 1256 H= 314:00 +CPU= 30.947
  18:40:47 STEP 1257 H= 314:15 +CPU= 29.766
  18:41:22 STEP 1258 H= 314:30 +CPU= 29.577
  18:41:55 STEP 1259 H= 314:45 +CPU= 29.502
  18:42:38 STEP 1260 H= 315:00 +CPU= 37.304
Moving to projects directory: /var/lib/boinc/slots/1/ICMGGh7zg+001260
Moving to projects directory: /var/lib/boinc/slots/1/ICMSHh7zg+001260
Moving to projects directory: /var/lib/boinc/slots/1/ICMUAh7zg+001260
  18:43:13 STEP 1261 H= 315:15 +CPU= 29.627
  18:43:46 STEP 1262 H= 315:30 +CPU= 29.429
  18:44:14 STEP 1263 H= 315:45 +CPU= 25.992
  18:44:51 STEP 1264 H= 316:00 +CPU= 30.253
  18:45:27 STEP 1265 H= 316:15 +CPU= 30.123
  18:46:04 STEP 1266 H= 316:30 +CPU= 30.627
  18:46:40 STEP 1267 H= 316:45 +CPU= 30.559
  18:47:16 STEP 1268 H= 317:00 +CPU= 29.962
  18:47:51 STEP 1269 H= 317:15 +CPU= 29.528
  18:48:24 STEP 1270 H= 317:30 +CPU= 29.631
  18:48:59 STEP 1271 H= 317:45 +CPU= 29.547
  18:49:42 STEP 1272 H= 318:00 +CPU= 37.189
Moving to projects directory: /var/lib/boinc/slots/1/ICMGGh7zg+001272
Moving to projects directory: /var/lib/boinc/slots/1/ICMSHh7zg+001272
Moving to projects directory: /var/lib/boinc/slots/1/ICMUAh7zg+001272
  18:50:18 STEP 1273 H= 318:15 +CPU= 29.732
  18:50:52 STEP 1274 H= 318:30 +CPU= 29.644
  18:51:26 STEP 1275 H= 318:45 +CPU= 29.810
  18:52:01 STEP 1276 H= 319:00 +CPU= 30.268
  18:52:37 STEP 1277 H= 319:15 +CPU= 30.401
  18:53:12 STEP 1278 H= 319:30 +CPU= 30.929
  18:53:47 STEP 1279 H= 319:45 +CPU= 29.815
  18:54:21 STEP 1280 H= 320:00 +CPU= 29.812
  18:54:55 STEP 1281 H= 320:15 +CPU= 29.340
  18:55:29 STEP 1282 H= 320:30 +CPU= 29.647
  18:56:03 STEP 1283 H= 320:45 +CPU= 29.554
  18:56:48 STEP 1284 H= 321:00 +CPU= 37.482
Moving to projects directory: /var/lib/boinc/slots/1/ICMGGh7zg+001284
Moving to projects directory: /var/lib/boinc/slots/1/ICMSHh7zg+001284
Moving to projects directory: /var/lib/boinc/slots/1/ICMUAh7zg+001284
  18:57:22 STEP 1285 H= 321:15 +CPU= 29.672
  18:57:57 STEP 1286 H= 321:30 +CPU= 29.740
  18:58:32 STEP 1287 H= 321:45 +CPU= 30.085
  18:59:07 STEP 1288 H= 322:00 +CPU= 30.665
  18:59:44 STEP 1289 H= 322:15 +CPU= 30.904
  19:00:18 STEP 1290 H= 322:30 +CPU= 29.506
  19:00:52 STEP 1291 H= 322:45 +CPU= 29.352
  19:01:27 STEP 1292 H= 323:00 +CPU= 29.461
  19:01:58 STEP 1293 H= 323:15 +CPU= 27.814
  19:02:33 STEP 1294 H= 323:30 +CPU= 29.127
  19:03:06 STEP 1295 H= 323:45 +CPU= 29.524
  19:04:09 STEP 1296 H= 324:00 +CPU= 40.298
Moving to projects directory: /var/lib/boinc/slots/1/ICMGGh7zg+001296
Moving to projects directory: /var/lib/boinc/slots/1/ICMSHh7zg+001296
Moving to projects directory: /var/lib/boinc/slots/1/ICMUAh7zg+001296
  19:04:38 STEP 1297 H= 324:15 +CPU= 27.425
  19:05:09 STEP 1298 H= 324:30 +CPU= 28.097
  19:05:39 STEP 1299 H= 324:45 +CPU= 27.040
  19:06:15 STEP 1300 H= 325:00 +CPU= 32.481
  19:06:53 STEP 1301 H= 325:15 +CPU= 31.579
  19:07:32 STEP 1302 H= 325:30 +CPU= 31.890
  19:08:08 STEP 1303 H= 325:45 +CPU= 31.050
  19:08:41 STEP 1304 H= 326:00 +CPU= 29.837
  19:09:13 STEP 1305 H= 326:15 +CPU= 29.191
  19:09:47 STEP 1306 H= 326:30 +CPU= 29.913
  19:10:24 STEP 1307 H= 326:45 +CPU= 32.233
  19:11:12 STEP 1308 H= 327:00 +CPU= 40.381
Moving to projects directory: /var/lib/boinc/slots/1/ICMGGh7zg+001308
Moving to projects directory: /var/lib/boinc/slots/1/ICMSHh7zg+001308
Moving to projects directory: /var/lib/boinc/slots/1/ICMUAh7zg+001308
  19:11:48 STEP 1309 H= 327:15 +CPU= 31.382
  19:12:21 STEP 1310 H= 327:30 +CPU= 30.717
  19:12:54 STEP 1311 H= 327:45 +CPU= 29.275
  19:13:26 STEP 1312 H= 328:00 +CPU= 27.448
  19:13:59 STEP 1313 H= 328:15 +CPU= 28.558
  19:14:30 STEP 1314 H= 328:30 +CPU= 27.696
  19:15:00 STEP 1315 H= 328:45 +CPU= 27.356
  19:15:31 STEP 1316 H= 329:00 +CPU= 27.846
  19:16:00 STEP 1317 H= 329:15 +CPU= 26.210
  19:16:28 STEP 1318 H= 329:30 +CPU= 25.494
  19:16:58 STEP 1319 H= 329:45 +CPU= 26.940
  19:17:36 STEP 1320 H= 330:00 +CPU= 33.771
Moving to projects directory: /var/lib/boinc/slots/1/ICMGGh7zg+001320
Moving to projects directory: /var/lib/boinc/slots/1/ICMSHh7zg+001320
Moving to projects directory: /var/lib/boinc/slots/1/ICMUAh7zg+001320
  19:18:06 STEP 1321 H= 330:15 +CPU= 26.726
  19:18:38 STEP 1322 H= 330:30 +CPU= 27.900
  19:19:13 STEP 1323 H= 330:45 +CPU= 29.335
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_bl_1.13_x86_64-pc-linux-gnu
(argv1) start_date: 2016092300
(argv2) exptid: h7zg
(argv3) unique_member_id: a2fd
(argv4) batchid: 1017
(argv5) wuid: 12284972
(argv6) fclen: 20
(argv7) app_name: oifs_43r3_bl
(argv8) nthreads: 1
Working directory is: /var/lib/boinc/slots/1
Project directory is: /var/lib/boinc/projects/climateprediction.net/
app name: oifs_43r3_bl
version: 1.13
Location of temp folder: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972
..mkdir for temp folder for results failed
Copying: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip to: /var/lib/boinc/slots/1/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc/slots/1/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_52180969630a64e05db59958497ae7a2 to: /var/lib/boinc/slots/1/oifs_43r3_bl_a2fd_2016092300_20_1017_12284972.zip
Unzipping the namelist zip file: /var/lib/boinc/slots/1/oifs_43r3_bl_a2fd_2016092300_20_1017_12284972.zip
ic_ancil_file: ic_ancil_12284972
ifsdata_file: ifsdata_12284972
climate_data_file: clim_data_12284972
horiz_resolution: 159
vert_resolution: 91
grid_type: l_2
upload_interval: 96
utstep: 900
nfrres: restart dump frequency (steps) 48
Copying IC ancils from: ../../projects/climateprediction.net/jf_f7ecfd27aa1e68bbe7e8d19800e25c73 to: /var/lib/boinc/slots/1/ic_ancil_12284972.zip
Unzipping the IC ancils zip file: /var/lib/boinc/slots/1/ic_ancil_12284972.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_8046f8eb7f39fbf20a6c330521ca5404 to: /var/lib/boinc/slots/1/ifsdata/ifsdata_12284972.zip
Unzipping the ifsdata_zip file: /var/lib/boinc/slots/1/ifsdata/ifsdata_12284972.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc/slots/1/159l_2/clim_data_12284972.zip
Unzipping the climate data zip file: /var/lib/boinc/slots/1/159l_2/clim_data_12284972.zip
Checking for progress XML file: /var/lib/boinc/slots/1/progress_file_12284972.xml
Opened progress file ok : /var/lib/boinc/slots/1/progress_file_12284972.xml
-- Model is restarting --
Adjusting last_iter, 1324, to previous model restart step.
Creating progress file: /var/lib/boinc/slots/1/progress_file_12284972.xml
last_cpu_time: 41117
upload_file_number: 13
last_iter: 1297
last_upload: 1123200
model_completed: 0
total_length_of_simulation: 1728000
result_base_name: oifs_43r3_bl_a2fd_2016092300_20_1017_12284972_0_r2132692060
The child process has been launched with process id: 1905241
Executing the command: /var/lib/boinc/slots/1/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:zion.faskatech.net:1:1:1905241:1905241] [20240608:192056:1717867256:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2091] fp = 0x314f440
[EC_DRHOOK:zion.faskatech.net:1:1:1905241:1905241] [20240608:192056:1717867256:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:zion.faskatech.net:1:1:1905241:1905241] [20240608:192056:1717867256:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:zion.faskatech.net:1:1:1905241:1905241] [20240608:192056:1717867256:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:zion.faskatech.net:1:1:1905241:1905241] [20240608:192056:1717867256:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:zion.faskatech.net:1:1:1905241:1905241] [20240608:192056:1717867256:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:zion.faskatech.net:1:1:1905241:1905241] [20240608:192056:1717867256:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:zion.faskatech.net:1:1:1905241:1905241] [20240608:192056:1717867256:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:zion.faskatech.net:1:1:1905241:1905241] [20240608:192056:1717867256:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:zion.faskatech.net:1:1:1905241:1905241] [20240608:192056:1717867256:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:zion.faskatech.net:1:1:1905241:1905241] [20240608:192056:1717867256:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:zion.faskatech.net:1:1:1905241:1905241] [20240608:192056:1717867256:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:zion.faskatech.net:1:1:1905241:1905241] [20240608:192056:1717867256:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:zion.faskatech.net:1:1:1905241:1905241] [20240608:192056:1717867256:0.000] [ignore_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:zion.faskatech.net:1:1:1905241:1905241] [20240608:192056:1717867256:0.000] [restore_default_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:zion.faskatech.net:1:1:1905241:1905241] [20240608:192056:1717867256:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:zion.faskatech.net:1:1:1905241:1905241] [20240608:192056:1717867256:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:zion.faskatech.net:1:1:1905241:1905241] [20240608:192056:1717867256:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:zion.faskatech.net:1:1:1905241:1905241] [20240608:192056:1717867256:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x820cf0 (old at (nil))
[EC_DRHOOK:zion.faskatech.net:1:1:1905241:1905241] [20240608:192056:1717867256:0.000] [trapfpe_treatment@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:zion.faskatech.net:1:1:1905241:1905241] [20240608:192056:1717867256:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:zion.faskatech.net:1:1:1905241:1905241] [20240608:192056:1717867256:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:zion.faskatech.net:1:1:1905241:1905241] [20240608:192056:1717867256:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x820cf0 (old at (nil))
[EC_DRHOOK:zion.faskatech.net:1:1:1905241:1905241] [20240608:192056:1717867256:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:zion.faskatech.net:1:1:1905241:1905241] [20240608:192056:1717867256:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:zion.faskatech.net:1:1:1905241:1905241] [20240608:192056:1717867256:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:zion.faskatech.net:1:1:1905241:1905241] [20240608:192056:1717867256:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x820cf0 (old at (nil))
[EC_DRHOOK:zion.faskatech.net:1:1:1905241:1905241] [20240608:192056:1717867256:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x820cf0 (old at (nil))
[EC_DRHOOK:zion.faskatech.net:1:1:1905241:1905241] [20240608:192056:1717867256:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:zion.faskatech.net:1:1:1905241:1905241] [20240608:192056:1717867256:0.001] [catch_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  19:22:06 STEP 1296 H= 324:00 +CPU= 30.739
  19:22:40 STEP 1297 H= 324:15 +CPU= 29.849
  19:23:14 STEP 1298 H= 324:30 +CPU= 29.984
  19:23:50 STEP 1299 H= 324:45 +CPU= 30.655
  19:24:24 STEP 1300 H= 325:00 +CPU= 30.409
  19:24:58 STEP 1301 H= 325:15 +CPU= 29.588
  19:25:33 STEP 1302 H= 325:30 +CPU= 29.896
  19:26:06 STEP 1303 H= 325:45 +CPU= 29.421
  19:26:40 STEP 1304 H= 326:00 +CPU= 29.791
  19:27:13 STEP 1305 H= 326:15 +CPU= 29.002
  19:27:48 STEP 1306 H= 326:30 +CPU= 30.014
  19:28:22 STEP 1307 H= 326:45 +CPU= 29.636
  19:29:06 STEP 1308 H= 327:00 +CPU= 37.841
Moving to projects directory: /var/lib/boinc/slots/1/ICMGGh7zg+001308
Moving to projects directory: /var/lib/boinc/slots/1/ICMSHh7zg+001308
Moving to projects directory: /var/lib/boinc/slots/1/ICMUAh7zg+001308
  19:29:42 STEP 1309 H= 327:15 +CPU= 30.553
  19:30:17 STEP 1310 H= 327:30 +CPU= 31.047
  19:30:51 STEP 1311 H= 327:45 +CPU= 29.861
  19:31:26 STEP 1312 H= 328:00 +CPU= 30.194
  19:32:00 STEP 1313 H= 328:15 +CPU= 29.931
  19:32:34 STEP 1314 H= 328:30 +CPU= 29.938
  19:33:09 STEP 1315 H= 328:45 +CPU= 30.121
  19:33:44 STEP 1316 H= 329:00 +CPU= 30.119
  19:34:17 STEP 1317 H= 329:15 +CPU= 29.649
  19:34:52 STEP 1318 H= 329:30 +CPU= 29.943
  19:35:27 STEP 1319 H= 329:45 +CPU= 30.918
  19:36:11 STEP 1320 H= 330:00 +CPU= 37.969
Moving to projects directory: /var/lib/boinc/slots/1/ICMGGh7zg+001320
Moving to projects directory: /var/lib/boinc/slots/1/ICMSHh7zg+001320
Moving to projects directory: /var/lib/boinc/slots/1/ICMUAh7zg+001320
  19:36:45 STEP 1321 H= 330:15 +CPU= 30.451
  19:37:20 STEP 1322 H= 330:30 +CPU= 29.976
  19:37:55 STEP 1323 H= 330:45 +CPU= 30.009
  19:38:29 STEP 1324 H= 331:00 +CPU= 30.027
  19:39:02 STEP 1325 H= 331:15 +CPU= 29.686
  19:39:37 STEP 1326 H= 331:30 +CPU= 29.702
  19:40:11 STEP 1327 H= 331:45 +CPU= 29.821
  19:40:46 STEP 1328 H= 332:00 +CPU= 30.593
  19:41:21 STEP 1329 H= 332:15 +CPU= 30.192
  19:41:56 STEP 1330 H= 332:30 +CPU= 30.505
  19:42:31 STEP 1331 H= 332:45 +CPU= 30.511
  19:43:15 STEP 1332 H= 333:00 +CPU= 38.610
Moving to projects directory: /var/lib/boinc/slots/1/ICMGGh7zg+001332
Moving to projects directory: /var/lib/boinc/slots/1/ICMSHh7zg+001332
Moving to projects directory: /var/lib/boinc/slots/1/ICMUAh7zg+001332
  19:43:50 STEP 1333 H= 333:15 +CPU= 30.024
  19:44:24 STEP 1334 H= 333:30 +CPU= 29.847
  19:44:58 STEP 1335 H= 333:45 +CPU= 29.814
  19:45:32 STEP 1336 H= 334:00 +CPU= 30.348
  19:46:07 STEP 1337 H= 334:15 +CPU= 29.621
  19:46:41 STEP 1338 H= 334:30 +CPU= 29.821
  19:47:16 STEP 1339 H= 334:45 +CPU= 29.720
  19:47:51 STEP 1340 H= 335:00 +CPU= 30.258
  19:48:22 STEP 1341 H= 335:15 +CPU= 28.312
  19:48:57 STEP 1342 H= 335:30 +CPU= 30.092
  19:49:25 STEP 1343 H= 335:45 +CPU= 25.563
  19:50:45 STEP 1344 H= 336:00 +CPU= 38.757
Moving to projects directory: /var/lib/boinc/slots/1/ICMGGh7zg+001344
Moving to projects directory: /var/lib/boinc/slots/1/ICMSHh7zg+001344
Moving to projects directory: /var/lib/boinc/slots/1/ICMUAh7zg+001344
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMGGh7zg+001248
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMSHh7zg+001248
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMUAh7zg+001248
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMGGh7zg+001260
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMSHh7zg+001260
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMUAh7zg+001260
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMGGh7zg+001272
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMSHh7zg+001272
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMUAh7zg+001272
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMGGh7zg+001284
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMSHh7zg+001284
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMUAh7zg+001284
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMGGh7zg+001296
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMSHh7zg+001296
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMUAh7zg+001296
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMGGh7zg+001308
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMSHh7zg+001308
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMUAh7zg+001308
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMGGh7zg+001320
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMSHh7zg+001320
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMUAh7zg+001320
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMGGh7zg+001332
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMSHh7zg+001332
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMUAh7zg+001332
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_a2fd_2016092300_20_1017_12284972_0_r2132692060_13.zip
Uploading the intermediate file: upload_file_13.zip
  19:51:22 STEP 1345 H= 336:15 +CPU= 29.100
  19:51:57 STEP 1346 H= 336:30 +CPU= 29.803
  19:52:32 STEP 1347 H= 336:45 +CPU= 29.755
  19:53:07 STEP 1348 H= 337:00 +CPU= 30.316
  19:53:42 STEP 1349 H= 337:15 +CPU= 30.030
  19:54:16 STEP 1350 H= 337:30 +CPU= 30.021
  19:54:51 STEP 1351 H= 337:45 +CPU= 30.203
  19:55:28 STEP 1352 H= 338:00 +CPU= 31.220
  19:56:04 STEP 1353 H= 338:15 +CPU= 31.119
  19:56:40 STEP 1354 H= 338:30 +CPU= 30.401
  19:57:16 STEP 1355 H= 338:45 +CPU= 29.783
  19:58:00 STEP 1356 H= 339:00 +CPU= 37.307
Moving to projects directory: /var/lib/boinc/slots/1/ICMGGh7zg+001356
Moving to projects directory: /var/lib/boinc/slots/1/ICMSHh7zg+001356
Moving to projects directory: /var/lib/boinc/slots/1/ICMUAh7zg+001356
  19:58:35 STEP 1357 H= 339:15 +CPU= 29.952
  19:59:11 STEP 1358 H= 339:30 +CPU= 30.178
  19:59:45 STEP 1359 H= 339:45 +CPU= 29.566
  20:00:21 STEP 1360 H= 340:00 +CPU= 30.765
  20:00:56 STEP 1361 H= 340:15 +CPU= 30.194
  20:01:31 STEP 1362 H= 340:30 +CPU= 30.442
  20:02:07 STEP 1363 H= 340:45 +CPU= 30.588
  20:02:44 STEP 1364 H= 341:00 +CPU= 31.547
  20:03:19 STEP 1365 H= 341:15 +CPU= 30.185
  20:03:56 STEP 1366 H= 341:30 +CPU= 30.130
  20:04:31 STEP 1367 H= 341:45 +CPU= 29.308
  20:05:15 STEP 1368 H= 342:00 +CPU= 37.032
Moving to projects directory: /var/lib/boinc/slots/1/ICMGGh7zg+001368
Moving to projects directory: /var/lib/boinc/slots/1/ICMSHh7zg+001368
Moving to projects directory: /var/lib/boinc/slots/1/ICMUAh7zg+001368
  20:05:51 STEP 1369 H= 342:15 +CPU= 29.937
  20:06:26 STEP 1370 H= 342:30 +CPU= 30.115
  20:07:01 STEP 1371 H= 342:45 +CPU= 30.119
  20:07:37 STEP 1372 H= 343:00 +CPU= 30.670
  20:08:13 STEP 1373 H= 343:15 +CPU= 30.361
  20:08:47 STEP 1374 H= 343:30 +CPU= 30.321
  20:09:24 STEP 1375 H= 343:45 +CPU= 30.986
  20:09:59 STEP 1376 H= 344:00 +CPU= 30.442
  20:10:33 STEP 1377 H= 344:15 +CPU= 29.835
  20:11:06 STEP 1378 H= 344:30 +CPU= 29.696
  20:11:41 STEP 1379 H= 344:45 +CPU= 29.667
  20:12:24 STEP 1380 H= 345:00 +CPU= 36.869
Moving to projects directory: /var/lib/boinc/slots/1/ICMGGh7zg+001380
Moving to projects directory: /var/lib/boinc/slots/1/ICMSHh7zg+001380
Moving to projects directory: /var/lib/boinc/slots/1/ICMUAh7zg+001380
  20:12:57 STEP 1381 H= 345:15 +CPU= 29.542
  20:13:32 STEP 1382 H= 345:30 +CPU= 29.865
  20:14:06 STEP 1383 H= 345:45 +CPU= 30.107
  20:14:42 STEP 1384 H= 346:00 +CPU= 30.751
  20:15:17 STEP 1385 H= 346:15 +CPU= 30.398
  20:15:53 STEP 1386 H= 346:30 +CPU= 31.039
  20:16:28 STEP 1387 H= 346:45 +CPU= 30.080
  20:16:59 STEP 1388 H= 347:00 +CPU= 27.924
  20:17:33 STEP 1389 H= 347:15 +CPU= 28.530
  20:18:04 STEP 1390 H= 347:30 +CPU= 27.268
  20:18:37 STEP 1391 H= 347:45 +CPU= 28.579
  20:19:48 STEP 1392 H= 348:00 +CPU= 40.699
Moving to projects directory: /var/lib/boinc/slots/1/ICMGGh7zg+001392
Moving to projects directory: /var/lib/boinc/slots/1/ICMSHh7zg+001392
Moving to projects directory: /var/lib/boinc/slots/1/ICMUAh7zg+001392
  20:20:23 STEP 1393 H= 348:15 +CPU= 30.307
  20:20:58 STEP 1394 H= 348:30 +CPU= 30.355
  20:21:35 STEP 1395 H= 348:45 +CPU= 31.053
  20:22:12 STEP 1396 H= 349:00 +CPU= 31.576
  20:22:48 STEP 1397 H= 349:15 +CPU= 30.683
  20:23:22 STEP 1398 H= 349:30 +CPU= 29.804
  20:23:56 STEP 1399 H= 349:45 +CPU= 30.014
  20:24:32 STEP 1400 H= 350:00 +CPU= 30.544
  20:25:07 STEP 1401 H= 350:15 +CPU= 30.058
  20:25:41 STEP 1402 H= 350:30 +CPU= 29.451
  20:26:16 STEP 1403 H= 350:45 +CPU= 30.046
  20:27:02 STEP 1404 H= 351:00 +CPU= 38.028
Moving to projects directory: /var/lib/boinc/slots/1/ICMGGh7zg+001404
Moving to projects directory: /var/lib/boinc/slots/1/ICMSHh7zg+001404
Moving to projects directory: /var/lib/boinc/slots/1/ICMUAh7zg+001404
  20:27:39 STEP 1405 H= 351:15 +CPU= 30.383
  20:28:16 STEP 1406 H= 351:30 +CPU= 30.755
  20:28:52 STEP 1407 H= 351:45 +CPU= 30.917
  20:29:28 STEP 1408 H= 352:00 +CPU= 31.012
  20:30:02 STEP 1409 H= 352:15 +CPU= 29.867
  20:30:37 STEP 1410 H= 352:30 +CPU= 29.903
  20:31:11 STEP 1411 H= 352:45 +CPU= 29.800
  20:31:46 STEP 1412 H= 353:00 +CPU= 30.525
  20:32:20 STEP 1413 H= 353:15 +CPU= 29.813
  20:32:54 STEP 1414 H= 353:30 +CPU= 30.018
  20:33:29 STEP 1415 H= 353:45 +CPU= 30.082
  20:34:13 STEP 1416 H= 354:00 +CPU= 38.365
Moving to projects directory: /var/lib/boinc/slots/1/ICMGGh7zg+001416
Moving to projects directory: /var/lib/boinc/slots/1/ICMSHh7zg+001416
Moving to projects directory: /var/lib/boinc/slots/1/ICMUAh7zg+001416
  20:34:48 STEP 1417 H= 354:15 +CPU= 30.715
  20:35:24 STEP 1418 H= 354:30 +CPU= 30.948
  20:35:59 STEP 1419 H= 354:45 +CPU= 30.130
  20:36:32 STEP 1420 H= 355:00 +CPU= 29.967
  20:37:05 STEP 1421 H= 355:15 +CPU= 29.661
  20:37:40 STEP 1422 H= 355:30 +CPU= 29.655
  20:38:15 STEP 1423 H= 355:45 +CPU= 30.096
  20:38:50 STEP 1424 H= 356:00 +CPU= 30.652
  20:39:25 STEP 1425 H= 356:15 +CPU= 30.296
  20:39:59 STEP 1426 H= 356:30 +CPU= 30.060
  20:40:35 STEP 1427 H= 356:45 +CPU= 30.577
  20:41:20 STEP 1428 H= 357:00 +CPU= 38.270
Moving to projects directory: /var/lib/boinc/slots/1/ICMGGh7zg+001428
Moving to projects directory: /var/lib/boinc/slots/1/ICMSHh7zg+001428
Moving to projects directory: /var/lib/boinc/slots/1/ICMUAh7zg+001428
  20:41:56 STEP 1429 H= 357:15 +CPU= 30.973
  20:42:31 STEP 1430 H= 357:30 +CPU= 30.150
  20:43:06 STEP 1431 H= 357:45 +CPU= 29.926
  20:43:40 STEP 1432 H= 358:00 +CPU= 30.366
  20:44:15 STEP 1433 H= 358:15 +CPU= 29.883
  20:44:50 STEP 1434 H= 358:30 +CPU= 29.849
  20:45:24 STEP 1435 H= 358:45 +CPU= 29.992
  20:45:58 STEP 1436 H= 359:00 +CPU= 29.544
  20:46:29 STEP 1437 H= 359:15 +CPU= 26.977
  20:46:53 STEP 1438 H= 359:30 +CPU= 22.833
  20:47:25 STEP 1439 H= 359:45 +CPU= 28.488
  20:48:28 STEP 1440 H= 360:00 +CPU= 29.224
..The child process terminated with status: 0
>>> Printing last 70 lines from file: NODE.001_01

 
DDH------------------------------------------------------------------------
I                                                                         I
I TIME =  360.000H DDH * Diagnostic Var.,     AREA                GLOBAL  I
I                                                                         I
I-------------------------------------------------------------------------I
I   VARIABLE   I      Initial value         I        Current value        I
I-------------------------------------------------------------------------I
I  OMEGA (P/S) I     -0.7385730483E-21      I       0.4102741026E-06      I
ICloud fractionI      0.0000000000E+00      I       0.3823295650E-01      I
I  Relat. Hum. I      0.4613525791E+00      I       0.4998657830E+00      I
I  PASS 01     I      0.0000000000E+00      I       0.1950899131E+03      I
DDH------------------------------------------------------------------------
  
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
 NSTEP =  1440 SCAN2M_HPOS  P
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ICMGGh7zg+001440 MODE=a
 IO-STREAM CLOSED - ICMGGh7zg+001440
 MPI-TASK:   1 -    859783988 BYTES IN      7 RECORDS TRANSFERRED IN    0.0003 SECONDS******** Mbytes/s, TOTAL TIME=    0.0090( 3.3%)
 20:48:28 STEP 1440 H= 360:00 +CPU= 29.224
 END CNT3
 NSTEP =  1440 CNT0   000000000  21.331  21.331  23.320   0.915               0               0               0               0               0
IO-STREAM STATISTICS, TOTAL NO OF REC/BYTES READ           46/  1069125708 WRITTEN         4290/************
 -TASK-OPENED-OPEN-RECS IN -KBYTE IN    -RECS OUT-KBYTE OUT-WALL   -WALL IN-WALL
  OU-TOT IN -TOT OUT
   1     78    0       46      1044068     4290 ************    92.7    33.3    59.4    33.8    87.3
===-=== START OF TIMING STATISTICS ===-===
 
STATS FOR ALL TASKS
 NUM ROUTINE                                     CALLS  MEAN(ms)   MAX(ms)   FRAC(%)  UNBAL(%)
   0 CNT0     - COMPLETE EXECUTION                   1 5252347.2 5252347.2    100.00      0.00
   1 CNT4     - FORWARD INTEGRATION                  1 5217204.6 5217204.6     99.33      0.00
   8 SCAN2M - GRID-POINT DYNAMICS                  157    6074.8    6074.8     18.16      0.00
   9 SPCM     - SPECTRAL COMP.                     144    1088.9    1088.9      2.99      0.00
  10 SCAN2M - PHYSICS                              145   15502.6   15502.6     42.80      0.00
  11 IOPACK   - OUTPUT P.P. RESULTS                 12    6710.2    6710.2      1.53      0.00
  12 SPNORM   - SPECTRAL NORM COMP.                  7      30.8      30.8      0.00      0.00
  17 GRIDFPOS IN CNT4                               12      33.2      33.2      0.01      0.00
  27 RESTART FILES - WRITING                         3   26609.0   26609.0      1.52      0.00
  28 RESTART FILES - READING                         1   34417.6   34417.6      0.66      0.00
  29 SU4FPOS IN CNT4                                12       0.1       0.1      0.00      0.00
  30 DYNFPOS IN CNT4                                12    8320.1    8320.1      1.90      0.00
  31 POSDDH IN STEPO                                 1      81.5      81.5      0.00      0.00
  37 CPGLAG   - SL COMPUTATIONS                    145    7765.3    7765.3     21.44      0.00
  39 SU0YOMB                                         1     180.6     180.6      0.00      0.00
  51 SCAN2M   - SL COMM. PART 1                    145      36.2      36.2      0.10      0.00
  54 SPCM     - M TO S/S TO M TRANSP.              144     510.4     510.4      1.40      0.00
  55 SPCIMPF  - S TO M/M TO S TRANSP.              144     145.2     145.2      0.40      0.00
  56 SPNORM   - SPECTRAL NORM COMM.                  7       0.2       0.2      0.00      0.00
 102 LTINV_CTL   - INVERSE LEGENDRE TRANSFORM      316     565.9     565.9      3.40      0.00
 103 LTDIR_CTL   - DIRECT LEGENDRE TRANSFORM       300     362.1     362.1      2.07      0.00
 106 FTDIR_CTL   - DIRECT FOURIER TRANSFORM        300     116.2     116.2      0.66      0.00
 107 FTINV_CTL   - INVERSE FOURIER TRANSFORM       316     178.6     178.6      1.07      0.00
 140 SULEG       - COMP. OF LEGENDRE POL.            1      40.2      40.2      0.00      0.00
 152 LTINV_CTL   - M TO L TRANSPOSITION            316      84.9      84.9      0.51      0.00
 153 LTDIR_CTL   - L TO M TRANSPOSITION            300      44.1      44.1      0.25      0.00
 157 FTINV_CTL   - L TO G TRANSPOSITION            316      67.2      67.2      0.40      0.00
 158 FTDIR_CTL   - G TO L TRANSPOSITION            300      30.0      30.0      0.17      0.00
 400 GSTATS                                      17763       0.0       0.0      0.00      0.00
 401 GSTATS HOOK                                 16473       0.0       0.0      0.00      0.00
TOTAL MEASURED IMBALANCE =       0.0 SECONDS,  0.0 PERCENT
TOTAL WALLCLOCK TIME     5252.3 CPU TIME    4437.8 VECTOR TIME     4437.8


===-=== END   OF TIMING STATISTICS ===-===
FORECAST DAYS PER DAY      248.4
  *** END CNT0 *** 
------------------------------------------------
Moving to projects directory: /var/lib/boinc/slots/1/ICMGGh7zg+001440
Moving to projects directory: /var/lib/boinc/slots/1/ICMSHh7zg+001440
Moving to projects directory: /var/lib/boinc/slots/1/ICMUAh7zg+001440
Adding to the zip: /var/lib/boinc/slots/1/NODE.001_01
Adding to the zip: /var/lib/boinc/slots/1/ifs.stat
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMGGh7zg+001344
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMSHh7zg+001344
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMUAh7zg+001344
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMGGh7zg+001356
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMSHh7zg+001356
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMUAh7zg+001356
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMGGh7zg+001368
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMSHh7zg+001368
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMUAh7zg+001368
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMGGh7zg+001380
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMSHh7zg+001380
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMUAh7zg+001380
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMGGh7zg+001392
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMSHh7zg+001392
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMUAh7zg+001392
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMGGh7zg+001404
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMSHh7zg+001404
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMUAh7zg+001404
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMGGh7zg+001416
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMSHh7zg+001416
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMUAh7zg+001416
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMGGh7zg+001428
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMSHh7zg+001428
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMUAh7zg+001428
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMGGh7zg+001440
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMSHh7zg+001440
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12284972/ICMUAh7zg+001440
Zipping up the final file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_a2fd_2016092300_20_1017_12284972_0_r2132692060_14.zip
Uploading the final file: upload_file_14.zip
Uploading trickle at timestep: 1295100
20:52:05 (1905195): called boinc_finish(0)

</stderr_txt>
<message>
upload failure: <file_xfer_error>
  <file_name>oifs_43r3_bl_a2fd_2016092300_20_1017_12284972_0_r2132692060_15.zip</file_name>
  <error_code>-161 (not found)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>oifs_43r3_bl_a2fd_2016092300_20_1017_12284972_0_r2132692060_16.zip</file_name>
  <error_code>-161 (not found)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>oifs_43r3_bl_a2fd_2016092300_20_1017_12284972_0_r2132692060_17.zip</file_name>
  <error_code>-161 (not found)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>oifs_43r3_bl_a2fd_2016092300_20_1017_12284972_0_r2132692060_18.zip</file_name>
  <error_code>-161 (not found)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>oifs_43r3_bl_a2fd_2016092300_20_1017_12284972_0_r2132692060_19.zip</file_name>
  <error_code>-161 (not found)</error_code>
</file_xfer_error>
</message>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
08 Jun 2024 19:19:53 1507812 22436563 oifs_43r3_bl_a2fd_2016092300_20_1017_12284972_0 1,295,100 45,554 0.0352


©2024 cpdn.org