climateprediction.net home page
Task 22270216

Task 22270216

Name oifs_43r3_ps_0295_1999050100_123_968_12184939_0
Workunit 12184939
Created 22 Dec 2022, 18:27:06 UTC
Sent 5 Jan 2023, 21:00:25 UTC
Report deadline 4 Feb 2023, 21:00:25 UTC
Received 7 Jan 2023, 8:46:09 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 196 (0x000000C4) EXIT_DISK_LIMIT_EXCEEDED
Computer ID 1507457
Run time 10 hours 12 min 17 sec
CPU time 8 hours 43 min 55 sec
Validate state Invalid
Credit 7,741.44
Device peak FLOPS 5.98 GFLOPS
Application version OpenIFS 43r3 Perturbed Surface v1.05
x86_64-pc-linux-gnu
Peak working set size 4,760.14 MB
Peak swap size 4,978.08 MB
Peak disk usage 7,792.10 MB
Stderr
<core_client_version>7.16.16</core_client_version>
<![CDATA[
<message>
Disk usage limit exceeded</message>
<stderr_txt>
 1141 H=1141:00 +CPU= 15.442
  18:17:49 STEP 1142 H=1142:00 +CPU= 15.634
  18:18:14 STEP 1143 H=1143:00 +CPU= 23.916
  18:18:30 STEP 1144 H=1144:00 +CPU= 15.681
  18:18:46 STEP 1145 H=1145:00 +CPU= 15.321
  18:19:12 STEP 1146 H=1146:00 +CPU= 24.530
  18:19:28 STEP 1147 H=1147:00 +CPU= 15.205
  18:19:44 STEP 1148 H=1148:00 +CPU= 15.699
  18:20:10 STEP 1149 H=1149:00 +CPU= 24.390
  18:20:27 STEP 1150 H=1150:00 +CPU= 15.652
  18:20:43 STEP 1151 H=1151:00 +CPU= 15.555
  18:21:11 STEP 1152 H=1152:00 +CPU= 27.645
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMGGhq0f+001152
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMSHhq0f+001152
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMUAhq0f+001152
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMGGhq0f+001128
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMSHhq0f+001128
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMUAhq0f+001128
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMGGhq0f+001140
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMSHhq0f+001140
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMUAhq0f+001140
Zipping up the intermediate file: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_0295_1999050100_123_968_12184939_0_r1638477040_46.zip
Uploading the intermediate file: upload_file_46.zip
  18:21:27 STEP 1153 H=1153:00 +CPU= 15.126
  18:21:43 STEP 1154 H=1154:00 +CPU= 15.491
  18:22:08 STEP 1155 H=1155:00 +CPU= 23.938
  18:22:25 STEP 1156 H=1156:00 +CPU= 15.902
  18:22:41 STEP 1157 H=1157:00 +CPU= 15.170
  18:23:06 STEP 1158 H=1158:00 +CPU= 24.560
  18:23:22 STEP 1159 H=1159:00 +CPU= 15.238
  18:23:39 STEP 1160 H=1160:00 +CPU= 15.700
  18:24:04 STEP 1161 H=1161:00 +CPU= 24.204
  18:24:20 STEP 1162 H=1162:00 +CPU= 15.668
  18:24:35 STEP 1163 H=1163:00 +CPU= 15.186
  18:25:03 STEP 1164 H=1164:00 +CPU= 26.478
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMGGhq0f+001164
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMSHhq0f+001164
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMUAhq0f+001164
  18:25:19 STEP 1165 H=1165:00 +CPU= 15.264
  18:25:35 STEP 1166 H=1166:00 +CPU= 15.515
  18:26:00 STEP 1167 H=1167:00 +CPU= 24.139
  18:26:16 STEP 1168 H=1168:00 +CPU= 15.588
  18:26:33 STEP 1169 H=1169:00 +CPU= 15.376
  18:26:59 STEP 1170 H=1170:00 +CPU= 24.573
  18:27:16 STEP 1171 H=1171:00 +CPU= 15.457
  18:27:32 STEP 1172 H=1172:00 +CPU= 15.485
  18:27:57 STEP 1173 H=1173:00 +CPU= 23.833
  18:28:13 STEP 1174 H=1174:00 +CPU= 15.576
  18:28:29 STEP 1175 H=1175:00 +CPU= 15.576
  18:28:58 STEP 1176 H=1176:00 +CPU= 27.766
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMGGhq0f+001176
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMSHhq0f+001176
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMUAhq0f+001176
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMGGhq0f+001152
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMSHhq0f+001152
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMUAhq0f+001152
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMGGhq0f+001164
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMSHhq0f+001164
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMUAhq0f+001164
Zipping up the intermediate file: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_0295_1999050100_123_968_12184939_0_r1638477040_47.zip
Uploading the intermediate file: upload_file_47.zip
  18:29:14 STEP 1177 H=1177:00 +CPU= 15.341
  18:29:31 STEP 1178 H=1178:00 +CPU= 15.629
  18:29:56 STEP 1179 H=1179:00 +CPU= 24.119
  18:30:12 STEP 1180 H=1180:00 +CPU= 15.539
  18:30:27 STEP 1181 H=1181:00 +CPU= 15.034
  18:30:52 STEP 1182 H=1182:00 +CPU= 24.254
  18:31:08 STEP 1183 H=1183:00 +CPU= 15.163
  18:31:24 STEP 1184 H=1184:00 +CPU= 15.549
  18:31:49 STEP 1185 H=1185:00 +CPU= 24.062
  18:32:05 STEP 1186 H=1186:00 +CPU= 15.587
  18:32:21 STEP 1187 H=1187:00 +CPU= 15.523
  18:32:49 STEP 1188 H=1188:00 +CPU= 26.518
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMGGhq0f+001188
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMSHhq0f+001188
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMUAhq0f+001188
  18:33:04 STEP 1189 H=1189:00 +CPU= 15.184
  18:33:21 STEP 1190 H=1190:00 +CPU= 15.813
  18:33:47 STEP 1191 H=1191:00 +CPU= 24.189
  18:34:03 STEP 1192 H=1192:00 +CPU= 15.612
  18:34:19 STEP 1193 H=1193:00 +CPU= 15.166
  18:34:45 STEP 1194 H=1194:00 +CPU= 24.584
  18:35:00 STEP 1195 H=1195:00 +CPU= 15.217
  18:35:17 STEP 1196 H=1196:00 +CPU= 15.620
  18:35:41 STEP 1197 H=1197:00 +CPU= 23.994
  18:35:58 STEP 1198 H=1198:00 +CPU= 15.603
  18:36:13 STEP 1199 H=1199:00 +CPU= 15.428
  18:36:42 STEP 1200 H=1200:00 +CPU= 27.747
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMGGhq0f+001200
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMSHhq0f+001200
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMUAhq0f+001200
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMGGhq0f+001176
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMSHhq0f+001176
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMUAhq0f+001176
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMGGhq0f+001188
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMSHhq0f+001188
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMUAhq0f+001188
Zipping up the intermediate file: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_0295_1999050100_123_968_12184939_0_r1638477040_48.zip
Uploading the intermediate file: upload_file_48.zip
  18:36:58 STEP 1201 H=1201:00 +CPU= 15.185
  18:37:14 STEP 1202 H=1202:00 +CPU= 15.526
  18:37:39 STEP 1203 H=1203:00 +CPU= 24.022
  18:37:55 STEP 1204 H=1204:00 +CPU= 15.676
  18:38:12 STEP 1205 H=1205:00 +CPU= 15.390
  18:38:37 STEP 1206 H=1206:00 +CPU= 24.613
  18:38:53 STEP 1207 H=1207:00 +CPU= 15.151
  18:39:09 STEP 1208 H=1208:00 +CPU= 15.541
  18:39:34 STEP 1209 H=1209:00 +CPU= 24.077
  18:39:50 STEP 1210 H=1210:00 +CPU= 15.707
  18:40:06 STEP 1211 H=1211:00 +CPU= 15.186
  18:40:33 STEP 1212 H=1212:00 +CPU= 26.325
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMGGhq0f+001212
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMSHhq0f+001212
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMUAhq0f+001212
  18:40:48 STEP 1213 H=1213:00 +CPU= 15.085
  18:41:05 STEP 1214 H=1214:00 +CPU= 15.600
  18:41:30 STEP 1215 H=1215:00 +CPU= 24.093
  18:41:46 STEP 1216 H=1216:00 +CPU= 15.618
  18:42:02 STEP 1217 H=1217:00 +CPU= 15.234
  18:42:27 STEP 1218 H=1218:00 +CPU= 24.501
  18:42:44 STEP 1219 H=1219:00 +CPU= 15.367
  18:43:00 STEP 1220 H=1220:00 +CPU= 15.691
  18:43:26 STEP 1221 H=1221:00 +CPU= 24.380
  18:43:42 STEP 1222 H=1222:00 +CPU= 15.766
  18:43:59 STEP 1223 H=1223:00 +CPU= 15.688
  18:44:29 STEP 1224 H=1224:00 +CPU= 27.854
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMGGhq0f+001224
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMSHhq0f+001224
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMUAhq0f+001224
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMGGhq0f+001200
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMSHhq0f+001200
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMUAhq0f+001200
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMGGhq0f+001212
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMSHhq0f+001212
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMUAhq0f+001212
Zipping up the intermediate file: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_0295_1999050100_123_968_12184939_0_r1638477040_49.zip
Uploading the intermediate file: upload_file_49.zip
  18:44:45 STEP 1225 H=1225:00 +CPU= 15.285
  18:45:02 STEP 1226 H=1226:00 +CPU= 15.697
  18:45:27 STEP 1227 H=1227:00 +CPU= 24.217
  18:45:44 STEP 1228 H=1228:00 +CPU= 15.855
  18:46:00 STEP 1229 H=1229:00 +CPU= 15.231
  18:46:26 STEP 1230 H=1230:00 +CPU= 24.634
  18:46:42 STEP 1231 H=1231:00 +CPU= 15.376
  18:46:59 STEP 1232 H=1232:00 +CPU= 15.901
  18:47:24 STEP 1233 H=1233:00 +CPU= 24.162
  18:47:41 STEP 1234 H=1234:00 +CPU= 15.818
  18:47:58 STEP 1235 H=1235:00 +CPU= 15.549
  18:48:27 STEP 1236 H=1236:00 +CPU= 26.793
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMGGhq0f+001236
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMSHhq0f+001236
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMUAhq0f+001236
  18:48:44 STEP 1237 H=1237:00 +CPU= 15.694
  18:49:01 STEP 1238 H=1238:00 +CPU= 16.118
  18:49:27 STEP 1239 H=1239:00 +CPU= 24.363
  18:49:43 STEP 1240 H=1240:00 +CPU= 15.613
  18:49:59 STEP 1241 H=1241:00 +CPU= 15.236
  18:50:25 STEP 1242 H=1242:00 +CPU= 24.601
  18:50:41 STEP 1243 H=1243:00 +CPU= 15.213
  18:50:57 STEP 1244 H=1244:00 +CPU= 15.632
  18:51:23 STEP 1245 H=1245:00 +CPU= 24.194
  18:51:39 STEP 1246 H=1246:00 +CPU= 15.626
  18:51:55 STEP 1247 H=1247:00 +CPU= 15.594
  18:52:23 STEP 1248 H=1248:00 +CPU= 27.547
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMGGhq0f+001248
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMSHhq0f+001248
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMUAhq0f+001248
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMGGhq0f+001224
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMSHhq0f+001224
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMUAhq0f+001224
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMGGhq0f+001236
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMSHhq0f+001236
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMUAhq0f+001236
Zipping up the intermediate file: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_0295_1999050100_123_968_12184939_0_r1638477040_50.zip
Uploading the intermediate file: upload_file_50.zip
  18:52:39 STEP 1249 H=1249:00 +CPU= 15.175
  18:52:55 STEP 1250 H=1250:00 +CPU= 15.435
  18:53:20 STEP 1251 H=1251:00 +CPU= 24.085
  18:53:37 STEP 1252 H=1252:00 +CPU= 15.715
  18:53:54 STEP 1253 H=1253:00 +CPU= 15.589
  18:54:20 STEP 1254 H=1254:00 +CPU= 25.144
  18:54:37 STEP 1255 H=1255:00 +CPU= 15.545
  18:54:53 STEP 1256 H=1256:00 +CPU= 15.617
  18:55:18 STEP 1257 H=1257:00 +CPU= 24.154
  18:55:35 STEP 1258 H=1258:00 +CPU= 15.769
  18:55:51 STEP 1259 H=1259:00 +CPU= 15.481
  18:56:21 STEP 1260 H=1260:00 +CPU= 27.018
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMGGhq0f+001260
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMSHhq0f+001260
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMUAhq0f+001260
  18:56:37 STEP 1261 H=1261:00 +CPU= 15.340
  18:56:54 STEP 1262 H=1262:00 +CPU= 15.775
  18:57:19 STEP 1263 H=1263:00 +CPU= 24.095
  18:57:37 STEP 1264 H=1264:00 +CPU= 15.905
  18:57:57 STEP 1265 H=1265:00 +CPU= 16.025
  18:58:24 STEP 1266 H=1266:00 +CPU= 24.807
  18:58:40 STEP 1267 H=1267:00 +CPU= 15.368
  18:58:57 STEP 1268 H=1268:00 +CPU= 15.561
  18:59:22 STEP 1269 H=1269:00 +CPU= 24.112
  18:59:39 STEP 1270 H=1270:00 +CPU= 15.871
  18:59:57 STEP 1271 H=1271:00 +CPU= 16.074
  19:00:26 STEP 1272 H=1272:00 +CPU= 27.953
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMGGhq0f+001272
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMSHhq0f+001272
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMUAhq0f+001272
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMGGhq0f+001248
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMSHhq0f+001248
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMUAhq0f+001248
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMGGhq0f+001260
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMSHhq0f+001260
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMUAhq0f+001260
Zipping up the intermediate file: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_0295_1999050100_123_968_12184939_0_r1638477040_51.zip
Uploading the intermediate file: upload_file_51.zip
  19:00:44 STEP 1273 H=1273:00 +CPU= 15.727
  19:01:01 STEP 1274 H=1274:00 +CPU= 15.924
  19:01:28 STEP 1275 H=1275:00 +CPU= 24.567
  19:01:44 STEP 1276 H=1276:00 +CPU= 15.734
  19:02:00 STEP 1277 H=1277:00 +CPU= 15.197
  19:02:26 STEP 1278 H=1278:00 +CPU= 24.591
  19:02:42 STEP 1279 H=1279:00 +CPU= 15.274
  19:02:58 STEP 1280 H=1280:00 +CPU= 15.690
  19:03:24 STEP 1281 H=1281:00 +CPU= 24.404
  19:03:41 STEP 1282 H=1282:00 +CPU= 15.897
  19:03:58 STEP 1283 H=1283:00 +CPU= 15.731
  19:04:28 STEP 1284 H=1284:00 +CPU= 27.067
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMGGhq0f+001284
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMSHhq0f+001284
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMUAhq0f+001284
  19:04:44 STEP 1285 H=1285:00 +CPU= 15.423
  19:05:01 STEP 1286 H=1286:00 +CPU= 15.669
  19:05:26 STEP 1287 H=1287:00 +CPU= 24.332
  19:05:43 STEP 1288 H=1288:00 +CPU= 15.687
  19:06:00 STEP 1289 H=1289:00 +CPU= 15.468
  19:06:27 STEP 1290 H=1290:00 +CPU= 24.901
  19:06:44 STEP 1291 H=1291:00 +CPU= 15.665
  19:07:01 STEP 1292 H=1292:00 +CPU= 15.811
  19:07:28 STEP 1293 H=1293:00 +CPU= 24.895
  19:07:45 STEP 1294 H=1294:00 +CPU= 15.784
  19:08:02 STEP 1295 H=1295:00 +CPU= 15.807
  19:08:31 STEP 1296 H=1296:00 +CPU= 27.936
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMGGhq0f+001296
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMSHhq0f+001296
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMUAhq0f+001296
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMGGhq0f+001272
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMSHhq0f+001272
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMUAhq0f+001272
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMGGhq0f+001284
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMSHhq0f+001284
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMUAhq0f+001284
Zipping up the intermediate file: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_0295_1999050100_123_968_12184939_0_r1638477040_52.zip
Uploading the intermediate file: upload_file_52.zip
  19:08:47 STEP 1297 H=1297:00 +CPU= 15.196
Uploading trickle at timestep: 4665600
  19:09:02 STEP 1298 H=1298:00 +CPU= 15.451
  19:09:28 STEP 1299 H=1299:00 +CPU= 23.994
  19:09:45 STEP 1300 H=1300:00 +CPU= 16.094
  19:10:01 STEP 1301 H=1301:00 +CPU= 15.538
  19:10:27 STEP 1302 H=1302:00 +CPU= 24.663
  19:10:44 STEP 1303 H=1303:00 +CPU= 15.549
  19:11:00 STEP 1304 H=1304:00 +CPU= 15.647
  19:11:27 STEP 1305 H=1305:00 +CPU= 24.903
  19:11:44 STEP 1306 H=1306:00 +CPU= 15.825
  19:12:01 STEP 1307 H=1307:00 +CPU= 15.390
  19:12:28 STEP 1308 H=1308:00 +CPU= 26.414
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMGGhq0f+001308
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMSHhq0f+001308
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMUAhq0f+001308
  19:12:44 STEP 1309 H=1309:00 +CPU= 15.131
  19:13:00 STEP 1310 H=1310:00 +CPU= 15.536
  19:13:25 STEP 1311 H=1311:00 +CPU= 23.958
  19:13:41 STEP 1312 H=1312:00 +CPU= 15.637
  19:13:57 STEP 1313 H=1313:00 +CPU= 15.088
  19:14:23 STEP 1314 H=1314:00 +CPU= 24.575
  19:14:38 STEP 1315 H=1315:00 +CPU= 15.104
  19:14:55 STEP 1316 H=1316:00 +CPU= 15.694
  19:15:21 STEP 1317 H=1317:00 +CPU= 24.261
  19:15:38 STEP 1318 H=1318:00 +CPU= 15.772
  19:15:55 STEP 1319 H=1319:00 +CPU= 15.729
  19:16:23 STEP 1320 H=1320:00 +CPU= 27.488
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMGGhq0f+001320
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMSHhq0f+001320
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMUAhq0f+001320
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMGGhq0f+001296
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMSHhq0f+001296
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMUAhq0f+001296
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMGGhq0f+001308
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMSHhq0f+001308
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMUAhq0f+001308
Zipping up the intermediate file: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_0295_1999050100_123_968_12184939_0_r1638477040_53.zip
Uploading the intermediate file: upload_file_53.zip
  19:16:38 STEP 1321 H=1321:00 +CPU= 15.009
  19:16:54 STEP 1322 H=1322:00 +CPU= 15.450
  19:17:20 STEP 1323 H=1323:00 +CPU= 24.113
  19:17:38 STEP 1324 H=1324:00 +CPU= 16.194
  19:17:55 STEP 1325 H=1325:00 +CPU= 15.605
  19:18:22 STEP 1326 H=1326:00 +CPU= 25.160
  19:18:39 STEP 1327 H=1327:00 +CPU= 15.451
  19:18:55 STEP 1328 H=1328:00 +CPU= 15.912
  19:19:21 STEP 1329 H=1329:00 +CPU= 24.237
  19:19:39 STEP 1330 H=1330:00 +CPU= 16.106
  19:19:56 STEP 1331 H=1331:00 +CPU= 15.392
  19:20:23 STEP 1332 H=1332:00 +CPU= 26.413
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMGGhq0f+001332
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMSHhq0f+001332
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMUAhq0f+001332
  19:20:39 STEP 1333 H=1333:00 +CPU= 15.306
  19:20:55 STEP 1334 H=1334:00 +CPU= 15.473
  19:21:21 STEP 1335 H=1335:00 +CPU= 24.081
  19:21:37 STEP 1336 H=1336:00 +CPU= 15.612
  19:21:53 STEP 1337 H=1337:00 +CPU= 15.061
  19:22:18 STEP 1338 H=1338:00 +CPU= 24.406
  19:22:34 STEP 1339 H=1339:00 +CPU= 15.114
  19:22:50 STEP 1340 H=1340:00 +CPU= 15.438
  19:23:14 STEP 1341 H=1341:00 +CPU= 23.760
  19:23:30 STEP 1342 H=1342:00 +CPU= 15.548
  19:23:46 STEP 1343 H=1343:00 +CPU= 15.559
  19:24:15 STEP 1344 H=1344:00 +CPU= 27.636
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMGGhq0f+001344
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMSHhq0f+001344
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMUAhq0f+001344
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMGGhq0f+001320
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMSHhq0f+001320
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMUAhq0f+001320
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMGGhq0f+001332
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMSHhq0f+001332
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMUAhq0f+001332
Zipping up the intermediate file: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_0295_1999050100_123_968_12184939_0_r1638477040_54.zip
Uploading the intermediate file: upload_file_54.zip
  19:24:31 STEP 1345 H=1345:00 +CPU= 15.240
  19:24:47 STEP 1346 H=1346:00 +CPU= 15.558
  19:25:12 STEP 1347 H=1347:00 +CPU= 23.827
  19:25:28 STEP 1348 H=1348:00 +CPU= 15.564
  19:25:45 STEP 1349 H=1349:00 +CPU= 15.417
  19:26:10 STEP 1350 H=1350:00 +CPU= 24.364
  19:26:25 STEP 1351 H=1351:00 +CPU= 15.079
  19:26:41 STEP 1352 H=1352:00 +CPU= 15.516
  19:27:06 STEP 1353 H=1353:00 +CPU= 23.845
  19:27:21 STEP 1354 H=1354:00 +CPU= 15.469
  19:27:37 STEP 1355 H=1355:00 +CPU= 15.199
  19:28:04 STEP 1356 H=1356:00 +CPU= 26.183
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMGGhq0f+001356
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMSHhq0f+001356
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMUAhq0f+001356
  19:28:19 STEP 1357 H=1357:00 +CPU= 15.023
  19:28:35 STEP 1358 H=1358:00 +CPU= 15.482
  19:28:59 STEP 1359 H=1359:00 +CPU= 23.762
  19:29:15 STEP 1360 H=1360:00 +CPU= 15.458
  19:29:30 STEP 1361 H=1361:00 +CPU= 15.018
  19:29:55 STEP 1362 H=1362:00 +CPU= 24.242
  19:30:10 STEP 1363 H=1363:00 +CPU= 15.040
  19:30:26 STEP 1364 H=1364:00 +CPU= 15.387
  19:30:50 STEP 1365 H=1365:00 +CPU= 23.793
  19:31:06 STEP 1366 H=1366:00 +CPU= 15.479
  19:31:22 STEP 1367 H=1367:00 +CPU= 15.472
  19:31:50 STEP 1368 H=1368:00 +CPU= 27.426
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMGGhq0f+001368
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMSHhq0f+001368
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMUAhq0f+001368
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMGGhq0f+001344
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMSHhq0f+001344
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMUAhq0f+001344
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMGGhq0f+001356
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMSHhq0f+001356
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMUAhq0f+001356
Zipping up the intermediate file: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_0295_1999050100_123_968_12184939_0_r1638477040_55.zip
Uploading the intermediate file: upload_file_55.zip
  19:32:05 STEP 1369 H=1369:00 +CPU= 15.010
  19:32:21 STEP 1370 H=1370:00 +CPU= 15.330
  19:32:45 STEP 1371 H=1371:00 +CPU= 23.696
  19:33:01 STEP 1372 H=1372:00 +CPU= 15.519
  19:33:17 STEP 1373 H=1373:00 +CPU= 15.108
  19:33:43 STEP 1374 H=1374:00 +CPU= 24.485
  19:33:59 STEP 1375 H=1375:00 +CPU= 15.191
  19:34:14 STEP 1376 H=1376:00 +CPU= 15.340
  19:34:38 STEP 1377 H=1377:00 +CPU= 23.652
  19:34:54 STEP 1378 H=1378:00 +CPU= 15.430
  19:35:10 STEP 1379 H=1379:00 +CPU= 15.212
  19:35:37 STEP 1380 H=1380:00 +CPU= 26.066
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMGGhq0f+001380
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMSHhq0f+001380
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMUAhq0f+001380
  19:35:52 STEP 1381 H=1381:00 +CPU= 15.055
  19:36:08 STEP 1382 H=1382:00 +CPU= 15.385
  19:36:32 STEP 1383 H=1383:00 +CPU= 23.638
  19:36:48 STEP 1384 H=1384:00 +CPU= 15.418
  19:37:03 STEP 1385 H=1385:00 +CPU= 14.894
  19:37:28 STEP 1386 H=1386:00 +CPU= 24.139
  19:37:44 STEP 1387 H=1387:00 +CPU= 14.998
  19:37:59 STEP 1388 H=1388:00 +CPU= 15.326
  19:38:23 STEP 1389 H=1389:00 +CPU= 23.641
  19:38:39 STEP 1390 H=1390:00 +CPU= 15.427
  19:38:55 STEP 1391 H=1391:00 +CPU= 15.438
  19:39:23 STEP 1392 H=1392:00 +CPU= 27.384
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMGGhq0f+001392
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMSHhq0f+001392
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMUAhq0f+001392
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMGGhq0f+001368
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMSHhq0f+001368
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMUAhq0f+001368
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMGGhq0f+001380
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMSHhq0f+001380
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMUAhq0f+001380
Zipping up the intermediate file: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_0295_1999050100_123_968_12184939_0_r1638477040_56.zip
Uploading the intermediate file: upload_file_56.zip
  19:39:39 STEP 1393 H=1393:00 +CPU= 14.969
  19:39:54 STEP 1394 H=1394:00 +CPU= 15.307
  19:40:19 STEP 1395 H=1395:00 +CPU= 23.695
  19:40:34 STEP 1396 H=1396:00 +CPU= 15.446
  19:40:50 STEP 1397 H=1397:00 +CPU= 14.898
  19:41:15 STEP 1398 H=1398:00 +CPU= 24.191
  19:41:30 STEP 1399 H=1399:00 +CPU= 15.003
  19:41:46 STEP 1400 H=1400:00 +CPU= 15.326
  19:42:10 STEP 1401 H=1401:00 +CPU= 23.752
  19:42:26 STEP 1402 H=1402:00 +CPU= 15.434
  19:42:41 STEP 1403 H=1403:00 +CPU= 15.099
  19:43:08 STEP 1404 H=1404:00 +CPU= 26.089
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMGGhq0f+001404
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMSHhq0f+001404
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMUAhq0f+001404
  19:43:24 STEP 1405 H=1405:00 +CPU= 14.975
  19:43:39 STEP 1406 H=1406:00 +CPU= 15.323
  19:44:04 STEP 1407 H=1407:00 +CPU= 23.641
  19:44:19 STEP 1408 H=1408:00 +CPU= 15.388
  19:44:35 STEP 1409 H=1409:00 +CPU= 14.864
  19:44:59 STEP 1410 H=1410:00 +CPU= 24.154
  19:45:15 STEP 1411 H=1411:00 +CPU= 14.938
  19:45:30 STEP 1412 H=1412:00 +CPU= 15.286
  19:45:55 STEP 1413 H=1413:00 +CPU= 23.654
  19:46:10 STEP 1414 H=1414:00 +CPU= 15.359
  19:46:26 STEP 1415 H=1415:00 +CPU= 15.360
  19:46:54 STEP 1416 H=1416:00 +CPU= 27.364
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMGGhq0f+001416
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMSHhq0f+001416
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMUAhq0f+001416
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMGGhq0f+001392
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMSHhq0f+001392
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMUAhq0f+001392
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMGGhq0f+001404
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMSHhq0f+001404
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMUAhq0f+001404
Zipping up the intermediate file: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_0295_1999050100_123_968_12184939_0_r1638477040_57.zip
Uploading the intermediate file: upload_file_57.zip
  19:47:10 STEP 1417 H=1417:00 +CPU= 14.997
  19:47:25 STEP 1418 H=1418:00 +CPU= 15.332
  19:47:49 STEP 1419 H=1419:00 +CPU= 23.462
  19:48:05 STEP 1420 H=1420:00 +CPU= 15.379
  19:48:20 STEP 1421 H=1421:00 +CPU= 14.883
  19:48:45 STEP 1422 H=1422:00 +CPU= 24.129
  19:49:01 STEP 1423 H=1423:00 +CPU= 15.007
  19:49:16 STEP 1424 H=1424:00 +CPU= 15.328
  19:49:40 STEP 1425 H=1425:00 +CPU= 23.473
  19:49:56 STEP 1426 H=1426:00 +CPU= 15.416
  19:50:12 STEP 1427 H=1427:00 +CPU= 15.070
  19:50:38 STEP 1428 H=1428:00 +CPU= 26.052
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMGGhq0f+001428
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMSHhq0f+001428
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMUAhq0f+001428
  19:50:54 STEP 1429 H=1429:00 +CPU= 15.001
  19:51:10 STEP 1430 H=1430:00 +CPU= 15.334
  19:51:34 STEP 1431 H=1431:00 +CPU= 23.644
  19:51:50 STEP 1432 H=1432:00 +CPU= 15.386
  19:52:05 STEP 1433 H=1433:00 +CPU= 14.866
  19:52:30 STEP 1434 H=1434:00 +CPU= 24.113
  19:52:45 STEP 1435 H=1435:00 +CPU= 14.946
  19:53:01 STEP 1436 H=1436:00 +CPU= 15.283
  19:53:25 STEP 1437 H=1437:00 +CPU= 23.665
  19:53:40 STEP 1438 H=1438:00 +CPU= 15.362
  19:53:56 STEP 1439 H=1439:00 +CPU= 15.310
  19:54:24 STEP 1440 H=1440:00 +CPU= 27.454
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMGGhq0f+001440
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMSHhq0f+001440
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMUAhq0f+001440
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMGGhq0f+001416
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMSHhq0f+001416
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMUAhq0f+001416
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMGGhq0f+001428
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMSHhq0f+001428
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMUAhq0f+001428
Zipping up the intermediate file: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_0295_1999050100_123_968_12184939_0_r1638477040_58.zip
Uploading the intermediate file: upload_file_58.zip
  19:54:40 STEP 1441 H=1441:00 +CPU= 14.916
  19:54:55 STEP 1442 H=1442:00 +CPU= 15.311
  19:55:20 STEP 1443 H=1443:00 +CPU= 23.711
  19:55:35 STEP 1444 H=1444:00 +CPU= 15.412
  19:55:51 STEP 1445 H=1445:00 +CPU= 14.890
  19:56:16 STEP 1446 H=1446:00 +CPU= 24.254
  19:56:31 STEP 1447 H=1447:00 +CPU= 14.972
  19:56:47 STEP 1448 H=1448:00 +CPU= 15.356
  19:57:11 STEP 1449 H=1449:00 +CPU= 23.742
  19:57:27 STEP 1450 H=1450:00 +CPU= 15.441
  19:57:42 STEP 1451 H=1451:00 +CPU= 15.008
  19:58:09 STEP 1452 H=1452:00 +CPU= 26.067
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMGGhq0f+001452
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMSHhq0f+001452
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMUAhq0f+001452
  19:58:25 STEP 1453 H=1453:00 +CPU= 15.074
  19:58:41 STEP 1454 H=1454:00 +CPU= 15.434
  19:59:06 STEP 1455 H=1455:00 +CPU= 23.964
  19:59:21 STEP 1456 H=1456:00 +CPU= 15.416
  19:59:37 STEP 1457 H=1457:00 +CPU= 14.901
  20:00:02 STEP 1458 H=1458:00 +CPU= 24.315
  20:00:17 STEP 1459 H=1459:00 +CPU= 15.013
  20:00:33 STEP 1460 H=1460:00 +CPU= 15.365
  20:00:58 STEP 1461 H=1461:00 +CPU= 23.705
  20:01:14 STEP 1462 H=1462:00 +CPU= 15.474
  20:01:30 STEP 1463 H=1463:00 +CPU= 15.436
  20:01:58 STEP 1464 H=1464:00 +CPU= 27.578
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMGGhq0f+001464
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMSHhq0f+001464
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMUAhq0f+001464
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMGGhq0f+001440
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMSHhq0f+001440
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMUAhq0f+001440
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMGGhq0f+001452
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMSHhq0f+001452
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMUAhq0f+001452
Zipping up the intermediate file: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_0295_1999050100_123_968_12184939_0_r1638477040_59.zip
Uploading the intermediate file: upload_file_59.zip
  20:02:13 STEP 1465 H=1465:00 +CPU= 14.824
  20:02:29 STEP 1466 H=1466:00 +CPU= 15.304
  20:02:53 STEP 1467 H=1467:00 +CPU= 23.694
  20:03:09 STEP 1468 H=1468:00 +CPU= 15.384
  20:03:24 STEP 1469 H=1469:00 +CPU= 14.830
  20:03:49 STEP 1470 H=1470:00 +CPU= 24.250
  20:04:04 STEP 1471 H=1471:00 +CPU= 14.932
  20:04:20 STEP 1472 H=1472:00 +CPU= 15.322
  20:04:44 STEP 1473 H=1473:00 +CPU= 23.687
  20:05:00 STEP 1474 H=1474:00 +CPU= 15.430
  20:05:16 STEP 1475 H=1475:00 +CPU= 14.973
  20:05:42 STEP 1476 H=1476:00 +CPU= 26.016
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMGGhq0f+001476
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMSHhq0f+001476
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMUAhq0f+001476
  20:05:58 STEP 1477 H=1477:00 +CPU= 14.961
  20:06:13 STEP 1478 H=1478:00 +CPU= 15.294
  20:06:38 STEP 1479 H=1479:00 +CPU= 23.613
  20:06:53 STEP 1480 H=1480:00 +CPU= 15.388
  20:07:09 STEP 1481 H=1481:00 +CPU= 14.822
  20:07:33 STEP 1482 H=1482:00 +CPU= 24.111
  20:07:49 STEP 1483 H=1483:00 +CPU= 14.930
  20:08:04 STEP 1484 H=1484:00 +CPU= 15.244
  20:08:29 STEP 1485 H=1485:00 +CPU= 23.580
  20:08:44 STEP 1486 H=1486:00 +CPU= 15.318
  20:09:00 STEP 1487 H=1487:00 +CPU= 15.268
  20:09:28 STEP 1488 H=1488:00 +CPU= 27.462
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMGGhq0f+001488
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMSHhq0f+001488
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMUAhq0f+001488
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMGGhq0f+001464
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMSHhq0f+001464
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMUAhq0f+001464
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMGGhq0f+001476
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMSHhq0f+001476
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMUAhq0f+001476
Zipping up the intermediate file: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_0295_1999050100_123_968_12184939_0_r1638477040_60.zip
Uploading the intermediate file: upload_file_60.zip
  20:09:43 STEP 1489 H=1489:00 +CPU= 14.821
  20:09:59 STEP 1490 H=1490:00 +CPU= 15.337
  20:10:23 STEP 1491 H=1491:00 +CPU= 23.664
  20:10:39 STEP 1492 H=1492:00 +CPU= 15.418
  20:10:54 STEP 1493 H=1493:00 +CPU= 14.883
  20:11:19 STEP 1494 H=1494:00 +CPU= 24.266
  20:11:35 STEP 1495 H=1495:00 +CPU= 14.948
  20:11:50 STEP 1496 H=1496:00 +CPU= 15.363
  20:12:15 STEP 1497 H=1497:00 +CPU= 23.703
  20:12:30 STEP 1498 H=1498:00 +CPU= 15.449
  20:12:46 STEP 1499 H=1499:00 +CPU= 14.995
  20:13:13 STEP 1500 H=1500:00 +CPU= 26.050
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMGGhq0f+001500
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMSHhq0f+001500
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMUAhq0f+001500
  20:13:28 STEP 1501 H=1501:00 +CPU= 15.004
  20:13:44 STEP 1502 H=1502:00 +CPU= 15.331
  20:14:08 STEP 1503 H=1503:00 +CPU= 23.687
  20:14:24 STEP 1504 H=1504:00 +CPU= 15.392
  20:14:39 STEP 1505 H=1505:00 +CPU= 14.877
  20:15:04 STEP 1506 H=1506:00 +CPU= 24.187
  20:15:19 STEP 1507 H=1507:00 +CPU= 14.964
  20:15:35 STEP 1508 H=1508:00 +CPU= 15.312
  20:15:59 STEP 1509 H=1509:00 +CPU= 23.693
  20:16:15 STEP 1510 H=1510:00 +CPU= 15.371
  20:16:31 STEP 1511 H=1511:00 +CPU= 15.307
  20:16:59 STEP 1512 H=1512:00 +CPU= 27.472
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMGGhq0f+001512
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMSHhq0f+001512
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMUAhq0f+001512
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMGGhq0f+001488
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMSHhq0f+001488
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMUAhq0f+001488
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMGGhq0f+001500
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMSHhq0f+001500
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMUAhq0f+001500
Zipping up the intermediate file: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_0295_1999050100_123_968_12184939_0_r1638477040_61.zip
Uploading the intermediate file: upload_file_61.zip
  20:17:14 STEP 1513 H=1513:00 +CPU= 14.824
  20:17:30 STEP 1514 H=1514:00 +CPU= 15.312
  20:17:54 STEP 1515 H=1515:00 +CPU= 23.686
  20:18:10 STEP 1516 H=1516:00 +CPU= 15.390
  20:18:25 STEP 1517 H=1517:00 +CPU= 14.816
  20:18:50 STEP 1518 H=1518:00 +CPU= 24.320
  20:19:05 STEP 1519 H=1519:00 +CPU= 14.972
  20:19:21 STEP 1520 H=1520:00 +CPU= 15.353
  20:19:45 STEP 1521 H=1521:00 +CPU= 23.721
  20:20:01 STEP 1522 H=1522:00 +CPU= 15.447
  20:20:16 STEP 1523 H=1523:00 +CPU= 14.973
  20:20:43 STEP 1524 H=1524:00 +CPU= 26.049
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMGGhq0f+001524
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMSHhq0f+001524
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMUAhq0f+001524
  20:20:58 STEP 1525 H=1525:00 +CPU= 14.987
  20:21:14 STEP 1526 H=1526:00 +CPU= 15.330
  20:21:38 STEP 1527 H=1527:00 +CPU= 23.650
  20:21:54 STEP 1528 H=1528:00 +CPU= 15.394
  20:22:09 STEP 1529 H=1529:00 +CPU= 14.842
  20:22:34 STEP 1530 H=1530:00 +CPU= 24.137
  20:22:49 STEP 1531 H=1531:00 +CPU= 14.947
  20:23:05 STEP 1532 H=1532:00 +CPU= 15.293
  20:23:29 STEP 1533 H=1533:00 +CPU= 23.638
  20:23:45 STEP 1534 H=1534:00 +CPU= 15.364
  20:24:01 STEP 1535 H=1535:00 +CPU= 15.290
  20:24:29 STEP 1536 H=1536:00 +CPU= 27.505
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMGGhq0f+001536
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMSHhq0f+001536
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMUAhq0f+001536
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMGGhq0f+001512
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMSHhq0f+001512
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMUAhq0f+001512
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMGGhq0f+001524
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMSHhq0f+001524
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMUAhq0f+001524
Zipping up the intermediate file: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_0295_1999050100_123_968_12184939_0_r1638477040_62.zip
Uploading the intermediate file: upload_file_62.zip
  20:24:44 STEP 1537 H=1537:00 +CPU= 14.847
Uploading trickle at timestep: 5529600
  20:25:00 STEP 1538 H=1538:00 +CPU= 15.331
  20:25:24 STEP 1539 H=1539:00 +CPU= 23.696
  20:25:40 STEP 1540 H=1540:00 +CPU= 15.416
  20:25:55 STEP 1541 H=1541:00 +CPU= 14.873
  20:26:20 STEP 1542 H=1542:00 +CPU= 24.241
  20:26:36 STEP 1543 H=1543:00 +CPU= 14.968
  20:26:51 STEP 1544 H=1544:00 +CPU= 15.327
  20:27:16 STEP 1545 H=1545:00 +CPU= 23.711
  20:27:31 STEP 1546 H=1546:00 +CPU= 15.442
  20:27:48 STEP 1547 H=1547:00 +CPU= 15.322
  20:28:17 STEP 1548 H=1548:00 +CPU= 26.896
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMGGhq0f+001548
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMSHhq0f+001548
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMUAhq0f+001548
  20:28:38 STEP 1549 H=1549:00 +CPU= 15.757
  20:28:54 STEP 1550 H=1550:00 +CPU= 15.545
  20:29:19 STEP 1551 H=1551:00 +CPU= 23.919
  20:29:35 STEP 1552 H=1552:00 +CPU= 15.569
  20:29:51 STEP 1553 H=1553:00 +CPU= 15.026
  20:30:17 STEP 1554 H=1554:00 +CPU= 24.679
  20:30:34 STEP 1555 H=1555:00 +CPU= 15.246
  20:30:56 STEP 1556 H=1556:00 +CPU= 16.208
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  20:32:07 STEP 1557 H=1557:00 +CPU= 25.720
  20:32:28 STEP 1558 H=1558:00 +CPU= 16.081
  20:32:55 STEP 1559 H=1559:00 +CPU= 16.519
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  20:42:44 STEP 1560 H=1560:00 +CPU= 29.846
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMGGhq0f+001560
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMSHhq0f+001560
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMUAhq0f+001560
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMGGhq0f+001536
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMSHhq0f+001536
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMUAhq0f+001536
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMGGhq0f+001548
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMSHhq0f+001548
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMUAhq0f+001548
Zipping up the intermediate file: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_0295_1999050100_123_968_12184939_0_r1638477040_63.zip
Uploading the intermediate file: upload_file_63.zip
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  20:43:24 STEP 1561 H=1561:00 +CPU= 15.897
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  20:45:54 STEP 1562 H=1562:00 +CPU= 16.287
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  21:05:45 STEP 1563 H=1563:00 +CPU= 25.870
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  21:06:22 STEP 1564 H=1564:00 +CPU= 16.561
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  21:11:02 STEP 1565 H=1565:00 +CPU= 16.257
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  21:33:49 STEP 1566 H=1566:00 +CPU= 26.378
  21:34:08 STEP 1567 H=1567:00 +CPU= 15.734
  21:34:25 STEP 1568 H=1568:00 +CPU= 15.770
[EC_DRHOOK:debian-desktop:1:1:32377:32377] [20230106:213437:1673037277:23206.642] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1538] Received signal#15 (SIGTERM) :: 4749MB (heap), 4780MB (maxrss), 0MB (maxstack), 0 (paging), nsigs = 1
[EC_DRHOOK:debian-desktop:1:1:32377:32377] [20230106:213437:1673037277:23206.699] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1542] Also activating Harakiri-alarm (SIGALRM=14) to expire after 500s elapsed to prevent hangs, nsigs = 1
[EC_DRHOOK:debian-desktop:1:1:32377:32377] [20230106:213437:1673037277:23206.699] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1544] Harakiri signal handler 'signal_harakiri' for signal#14 (SIGALRM) installed at 0x81f3c0 (old at (nil))
[EC_DRHOOK:debian-desktop:1:1:32377:32377] [20230106:213437:1673037277:23206.699] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1617] Signal#15 was caused by unrecognized si_code [memaddr=0x1], nsigs = 1
[EC_DRHOOK:debian-desktop:1:1:32377:32377] [20230106:213437:1673037277:23206.699] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1686] Starting DrHook backtrace for signal#15, nsigs = 1
[EC_DRHOOK:debian-desktop:1:1:32377:32377] [20230106:213437:1673037277:23206.699] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3843] 4749 MB (maxheap), 4780 MB (maxrss), 0 MB (maxstack)
[EC_DRHOOK:debian-desktop:1:1:32377:32377] [20230106:213437:1673037277:23206.699] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :  MASTER 
[EC_DRHOOK:debian-desktop:1:1:32377:32377] [20230106:213437:1673037277:23206.702] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :   CNT0 
[EC_DRHOOK:debian-desktop:1:1:32377:32377] [20230106:213437:1673037277:23206.702] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :    CNT1 
[EC_DRHOOK:debian-desktop:1:1:32377:32377] [20230106:213437:1673037277:23206.702] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :     CNT2 
[EC_DRHOOK:debian-desktop:1:1:32377:32377] [20230106:213437:1673037277:23206.702] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :      CNT3 
[EC_DRHOOK:debian-desktop:1:1:32377:32377] [20230106:213437:1673037277:23206.702] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :       CNT4 
[EC_DRHOOK:debian-desktop:1:1:32377:32377] [20230106:213437:1673037277:23206.702] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :        STEPO 
[EC_DRHOOK:debian-desktop:1:1:32377:32377] [20230106:213437:1673037277:23206.702] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :         SCAN2M 
[EC_DRHOOK:debian-desktop:1:1:32377:32377] [20230106:213437:1673037277:23206.702] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :          GP_MODEL_HEAP 
[EC_DRHOOK:debian-desktop:1:1:32377:32377] [20230106:213437:1673037277:23206.702] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :           GP_MODEL 
[EC_DRHOOK:debian-desktop:1:1:32377:32377] [20230106:213437:1673037277:23206.702] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :            CALL_SL_HEAP 
[EC_DRHOOK:debian-desktop:1:1:32377:32377] [20230106:213437:1673037277:23206.702] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :             CALL_SL 
[EC_DRHOOK:debian-desktop:1:1:32377:32377] [20230106:213437:1673037277:23206.702] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :              >OMP-CALL_SL     1        (1004) 
[EC_DRHOOK:debian-desktop:1:1:32377:32377] [20230106:213437:1673037277:23206.702] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :               LAPINEA 
[EC_DRHOOK:debian-desktop:1:1:32377:32377] [20230106:213437:1673037277:23206.702] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :                LARMES 
[EC_DRHOOK:debian-desktop:1:1:32377:32377] [20230106:213438:1673037278:23207.703] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1734] DrHook backtrace done for signal#15, nsigs = 1
[EC_DRHOOK:debian-desktop:1:1:32377:32377] [20230106:213438:1673037278:23207.703] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1785] Calling previous signal handler at 0x1ce8bb0 for signal#15, nsigs = 1
forrtl: error (78): process killed (SIGTERM)
Image              PC                Routine            Line        Source             
oifs_43r3_model.e  0000000001CE916B  Unknown               Unknown  Unknown
oifs_43r3_model.e  000000000081FFF1  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001DC9090  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000000DE9AA1  larmes_                   543  larmes.F90
oifs_43r3_model.e  0000000000DE50FD  lapinea_                  270  lapinea.F90
oifs_43r3_model.e  0000000000D7A4EC  call_sl_                  300  call_sl.F90
oifs_43r3_model.e  0000000000D784FD  call_sl_heap_              94  call_sl_heap.F90
oifs_43r3_model.e  0000000000BF9958  gp_model_                 533  gp_model.F90
oifs_43r3_model.e  00000000012E9482  gp_model_heap_             74  gp_model_heap.F90
oifs_43r3_model.e  0000000000BBC1CE  scan2m_                   535  scan2m.F90
oifs_43r3_model.e  000000000057A664  stepo_                    327  stepo.F90
oifs_43r3_model.e  000000000055DEEC  cnt4_                    1133  cnt4.F90
oifs_43r3_model.e  00000000005471C9  cnt3_                     267  cnt3.F90
oifs_43r3_model.e  0000000000546412  cnt2_                      88  cnt2.F90
oifs_43r3_model.e  0000000000545E28  cnt1_                      92  cnt1.F90
oifs_43r3_model.e  000000000040708D  cnt0_                     146  cnt0.F90
oifs_43r3_model.e  000000000040220F  MAIN__                     96  master.F90
oifs_43r3_model.e  00000000004021A2  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001DCA390  Unknown               Unknown  Unknown
oifs_43r3_model.e  000000000040206E  Unknown               Unknown  Unknown
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 1999050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0295
(argv4) batchid: 968
(argv5) wuid: 12184939
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /media/nathan/512gb/boinc/slots/2
Project directory is: /media/nathan/512gb/boinc/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939
..mkdir for temp folder for results failed
Copying: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /media/nathan/512gb/boinc/slots/2/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /media/nathan/512gb/boinc/slots/2/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_54bf02c1412c15e982b0b488cc1aa3eb to: /media/nathan/512gb/boinc/slots/2/oifs_43r3_ps_0295_1999050100_123_968_12184939.zip
Unzipping the namelist zip file: /media/nathan/512gb/boinc/slots/2/oifs_43r3_ps_0295_1999050100_123_968_12184939.zip
ic_ancil_file: ic_ancil_12184939
ifsdata_file: ifsdata_12184939
climate_data_file: clim_data_12184939
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_f1080bdf2b1996b707436f54dd417dc6 to: /media/nathan/512gb/boinc/slots/2/ic_ancil_12184939.zip
Unzipping the IC ancils zip file: /media/nathan/512gb/boinc/slots/2/ic_ancil_12184939.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_5dd13dd0090c7af59b127a5f4c714c2b to: /media/nathan/512gb/boinc/slots/2/ifsdata/ifsdata_12184939.zip
Unzipping the ifsdata_zip file: /media/nathan/512gb/boinc/slots/2/ifsdata/ifsdata_12184939.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /media/nathan/512gb/boinc/slots/2/159l_2/clim_data_12184939.zip
Unzipping the climate data zip file: /media/nathan/512gb/boinc/slots/2/159l_2/clim_data_12184939.zip
Checking for progress XML file: /media/nathan/512gb/boinc/slots/2/progress_file_12184939.xml
Opened progress file ok : /media/nathan/512gb/boinc/slots/2/progress_file_12184939.xml
-- Model is restarting --
Adjusting last_iter, 1569, to previous model restart step.
Creating progress file: /media/nathan/512gb/boinc/slots/2/progress_file_12184939.xml
last_cpu_time: 30856
upload_file_number: 64
last_iter: 1559
last_upload: 5616000
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0295_1999050100_123_968_12184939_0_r1638477040
The child process has been launched with process id: 3257
Executing the command: /media/nathan/512gb/boinc/slots/2/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:debian-desktop:1:1:3257:3257] [20220807:152605:1659878765:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:debian-desktop:1:1:3257:3257] [20220807:152605:1659878765:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:debian-desktop:1:1:3257:3257] [20220807:152605:1659878765:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:debian-desktop:1:1:3257:3257] [20220807:152605:1659878765:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:debian-desktop:1:1:3257:3257] [20220807:152605:1659878765:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:debian-desktop:1:1:3257:3257] [20220807:152605:1659878765:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:debian-desktop:1:1:3257:3257] [20220807:152605:1659878765:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:debian-desktop:1:1:3257:3257] [20220807:152605:1659878765:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:debian-desktop:1:1:3257:3257] [20220807:152605:1659878765:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:debian-desktop:1:1:3257:3257] [20220807:152605:1659878765:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:debian-desktop:1:1:3257:3257] [20220807:152605:1659878765:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:debian-desktop:1:1:3257:3257] [20220807:152605:1659878765:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:debian-desktop:1:1:3257:3257] [20220807:152605:1659878765:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:debian-desktop:1:1:3257:3257] [20220807:152605:1659878765:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:debian-desktop:1:1:3257:3257] [20220807:152605:1659878765:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:debian-desktop:1:1:3257:3257] [20220807:152605:1659878765:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:debian-desktop:1:1:3257:3257] [20220807:152605:1659878765:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:debian-desktop:1:1:3257:3257] [20220807:152605:1659878765:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:debian-desktop:1:1:3257:3257] [20220807:152605:1659878765:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:debian-desktop:1:1:3257:3257] [20220807:152605:1659878765:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:debian-desktop:1:1:3257:3257] [20220807:152605:1659878765:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:debian-desktop:1:1:3257:3257] [20220807:152605:1659878765:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:debian-desktop:1:1:3257:3257] [20220807:152605:1659878765:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:debian-desktop:1:1:3257:3257] [20220807:152605:1659878765:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:debian-desktop:1:1:3257:3257] [20220807:152605:1659878765:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:debian-desktop:1:1:3257:3257] [20220807:152605:1659878765:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:debian-desktop:1:1:3257:3257] [20220807:152605:1659878765:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:debian-desktop:1:1:3257:3257] [20220807:152605:1659878765:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:debian-desktop:1:1:3257:3257] [20220807:152605:1659878765:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:debian-desktop:1:1:3257:3257] [20220807:152605:1659878765:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  08:42:01 STEP 1560 H=1560:00 +CPU= 27.026
  08:42:17 STEP 1561 H=1561:00 +CPU= 15.195
  08:42:33 STEP 1562 H=1562:00 +CPU= 15.627
  08:42:58 STEP 1563 H=1563:00 +CPU= 23.973
  08:43:14 STEP 1564 H=1564:00 +CPU= 15.706
  08:43:29 STEP 1565 H=1565:00 +CPU= 14.993
  08:43:54 STEP 1566 H=1566:00 +CPU= 24.245
  08:44:09 STEP 1567 H=1567:00 +CPU= 15.201
  08:44:25 STEP 1568 H=1568:00 +CPU= 15.456
  08:44:49 STEP 1569 H=1569:00 +CPU= 23.756
  08:45:04 STEP 1570 H=1570:00 +CPU= 14.602
  08:45:17 STEP 1571 H=1571:00 +CPU= 13.587
  08:45:42 STEP 1572 H=1572:00 +CPU= 24.194
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMGGhq0f+001572
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMSHhq0f+001572
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMUAhq0f+001572
  08:45:56 STEP 1573 H=1573:00 +CPU= 13.614
  08:46:11 STEP 1574 H=1574:00 +CPU= 14.683
  08:46:34 STEP 1575 H=1575:00 +CPU= 23.295
  08:46:50 STEP 1576 H=1576:00 +CPU= 15.751
  08:47:06 STEP 1577 H=1577:00 +CPU= 15.520
  08:47:31 STEP 1578 H=1578:00 +CPU= 24.620
  08:47:47 STEP 1579 H=1579:00 +CPU= 15.703
  08:48:03 STEP 1580 H=1580:00 +CPU= 15.783
  08:48:26 STEP 1581 H=1581:00 +CPU= 23.102
  08:48:42 STEP 1582 H=1582:00 +CPU= 15.095
  08:48:57 STEP 1583 H=1583:00 +CPU= 15.019
  08:49:24 STEP 1584 H=1584:00 +CPU= 26.650
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMGGhq0f+001584
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMSHhq0f+001584
Moving to projects directory: /media/nathan/512gb/boinc/slots/2/ICMUAhq0f+001584
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMGGhq0f+001560
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMSHhq0f+001560
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMUAhq0f+001560
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMGGhq0f+001572
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMSHhq0f+001572
Adding to the zip: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_12184939/ICMUAhq0f+001572
Zipping up the intermediate file: /media/nathan/512gb/boinc/projects/climateprediction.net/oifs_43r3_ps_0295_1999050100_123_968_12184939_0_r1638477040_64.zip
Uploading the intermediate file: upload_file_64.zip
  08:49:39 STEP 1585 H=1585:00 +CPU= 14.602
  08:49:54 STEP 1586 H=1586:00 +CPU= 14.972
  08:50:17 STEP 1587 H=1587:00 +CPU= 23.198
  08:50:33 STEP 1588 H=1588:00 +CPU= 15.149
  08:50:47 STEP 1589 H=1589:00 +CPU= 14.594
  08:51:11 STEP 1590 H=1590:00 +CPU= 23.752
Abort request received from BOINC client, ending the child process

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 May 2023 16:10:48 1507457 22270216 oifs_43r3_ps_0295_1999050100_123_968_12184939_0 5,529,600 30,255 0.0055
09 May 2023 16:10:45 1507457 22270216 oifs_43r3_ps_0295_1999050100_123_968_12184939_0 4,665,600 25,833 0.0055
09 May 2023 16:10:42 1507457 22270216 oifs_43r3_ps_0295_1999050100_123_968_12184939_0 3,801,600 21,340 0.0056
09 May 2023 16:10:38 1507457 22270216 oifs_43r3_ps_0295_1999050100_123_968_12184939_0 2,937,600 16,795 0.0057


©2024 cpdn.org