Name | oifs_43r3_ps_0404_1999050100_123_968_12185048_1 |
Workunit | 12185048 |
Created | 5 Feb 2023, 18:44:09 UTC |
Sent | 5 Feb 2023, 18:45:26 UTC |
Report deadline | 6 Apr 2023, 18:45:26 UTC |
Received | 27 Feb 2023, 12:39:51 UTC |
Server state | Over |
Outcome | Computation error |
Client state | Compute error |
Exit status | 196 (0x000000C4) EXIT_DISK_LIMIT_EXCEEDED |
Computer ID | 1509518 |
Run time | 13 hours 57 min 37 sec |
CPU time | 6 hours 2 min 15 sec |
Validate state | Invalid |
Credit | 7,136.64 |
Device peak FLOPS | 4.20 GFLOPS |
Application version | OpenIFS 43r3 Perturbed Surface v1.05 x86_64-pc-linux-gnu |
Peak working set size | 4,419.80 MB |
Peak swap size | 4,978.16 MB |
Peak disk usage | 7,294.97 MB |
Stderr | <core_client_version>7.16.6</core_client_version> <![CDATA[ <message> Disk usage limit exceeded</message> <stderr_txt> 46 01:35:57 STEP 1266 H=1266:00 +CPU= 30.972 01:36:16 STEP 1267 H=1267:00 +CPU= 19.257 01:36:36 STEP 1268 H=1268:00 +CPU= 19.607 01:37:06 STEP 1269 H=1269:00 +CPU= 29.736 01:37:25 STEP 1270 H=1270:00 +CPU= 19.744 01:37:45 STEP 1271 H=1271:00 +CPU= 19.665 01:38:20 STEP 1272 H=1272:00 +CPU= 35.245 Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+001272 Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+001272 Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+001272 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185048/ICMGGhq0f+001248 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185048/ICMSHhq0f+001248 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185048/ICMUAhq0f+001248 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185048/ICMGGhq0f+001260 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185048/ICMSHhq0f+001260 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185048/ICMUAhq0f+001260 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0404_1999050100_123_968_12185048_1_r578666857_22.zip Uploading the intermediate file: upload_file_22.zip 01:38:41 STEP 1273 H=1273:00 +CPU= 20.420 Suspend request received from the BOINC client, suspending the child process Resuming the child process 01:39:29 STEP 1274 H=1274:00 +CPU= 25.915 01:40:04 STEP 1275 H=1275:00 +CPU= 34.145 01:40:25 STEP 1276 H=1276:00 +CPU= 20.653 01:40:49 STEP 1277 H=1277:00 +CPU= 23.499 01:41:30 STEP 1278 H=1278:00 +CPU= 38.739 01:41:56 STEP 1279 H=1279:00 +CPU= 25.558 01:42:23 STEP 1280 H=1280:00 +CPU= 25.876 01:42:59 STEP 1281 H=1281:00 +CPU= 35.138 01:43:21 STEP 1282 H=1282:00 +CPU= 21.586 01:43:41 STEP 1283 H=1283:00 +CPU= 20.098 Suspend request received from the BOINC client, suspending the child process Resuming the child process 01:44:39 STEP 1284 H=1284:00 +CPU= 43.307 Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+001284 Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+001284 Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+001284 01:45:04 STEP 1285 H=1285:00 +CPU= 24.412 01:45:32 STEP 1286 H=1286:00 +CPU= 26.655 01:46:07 STEP 1287 H=1287:00 +CPU= 34.006 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 01:47:31 STEP 1288 H=1288:00 +CPU= 29.419 01:47:54 STEP 1289 H=1289:00 +CPU= 22.777 01:48:30 STEP 1290 H=1290:00 +CPU= 34.782 Suspend request received from the BOINC client, suspending the child process Resuming the child process 01:49:18 STEP 1291 H=1291:00 +CPU= 25.254 01:49:39 STEP 1292 H=1292:00 +CPU= 21.142 Suspend request received from the BOINC client, suspending the child process Resuming the child process 01:50:29 STEP 1293 H=1293:00 +CPU= 37.524 Suspend request received from the BOINC client, suspending the child process Resuming the child process 01:51:05 STEP 1294 H=1294:00 +CPU= 24.624 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 01:52:30 STEP 1295 H=1295:00 +CPU= 29.938 Suspend request received from the BOINC client, suspending the child process Resuming the child process 01:53:30 STEP 1296 H=1296:00 +CPU= 38.172 Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+001296 Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+001296 Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+001296 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185048/ICMGGhq0f+001272 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185048/ICMSHhq0f+001272 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185048/ICMUAhq0f+001272 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185048/ICMGGhq0f+001284 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185048/ICMSHhq0f+001284 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185048/ICMUAhq0f+001284 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0404_1999050100_123_968_12185048_1_r578666857_23.zip Uploading the intermediate file: upload_file_23.zip 01:53:49 STEP 1297 H=1297:00 +CPU= 19.226 01:54:09 STEP 1298 H=1298:00 +CPU= 19.767 01:54:39 STEP 1299 H=1299:00 +CPU= 29.862 01:54:59 STEP 1300 H=1300:00 +CPU= 19.697 01:55:18 STEP 1301 H=1301:00 +CPU= 19.179 01:55:48 STEP 1302 H=1302:00 +CPU= 30.497 01:56:08 STEP 1303 H=1303:00 +CPU= 19.113 01:56:27 STEP 1304 H=1304:00 +CPU= 19.721 01:56:57 STEP 1305 H=1305:00 +CPU= 29.823 01:57:17 STEP 1306 H=1306:00 +CPU= 19.751 01:57:36 STEP 1307 H=1307:00 +CPU= 19.158 01:58:09 STEP 1308 H=1308:00 +CPU= 33.189 Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+001308 Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+001308 Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+001308 01:58:28 STEP 1309 H=1309:00 +CPU= 19.202 01:58:48 STEP 1310 H=1310:00 +CPU= 19.865 01:59:18 STEP 1311 H=1311:00 +CPU= 29.759 01:59:38 STEP 1312 H=1312:00 +CPU= 19.698 01:59:57 STEP 1313 H=1313:00 +CPU= 19.092 02:00:27 STEP 1314 H=1314:00 +CPU= 30.402 02:00:46 STEP 1315 H=1315:00 +CPU= 19.087 02:01:06 STEP 1316 H=1316:00 +CPU= 19.649 02:01:36 STEP 1317 H=1317:00 +CPU= 29.770 02:01:55 STEP 1318 H=1318:00 +CPU= 19.614 02:02:15 STEP 1319 H=1319:00 +CPU= 19.728 02:02:51 STEP 1320 H=1320:00 +CPU= 35.331 Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+001320 Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+001320 Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+001320 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185048/ICMGGhq0f+001296 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185048/ICMSHhq0f+001296 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185048/ICMUAhq0f+001296 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185048/ICMGGhq0f+001308 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185048/ICMSHhq0f+001308 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185048/ICMUAhq0f+001308 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0404_1999050100_123_968_12185048_1_r578666857_24.zip Uploading the intermediate file: upload_file_24.zip 02:03:10 STEP 1321 H=1321:00 +CPU= 19.203 02:03:29 STEP 1322 H=1322:00 +CPU= 19.709 02:03:59 STEP 1323 H=1323:00 +CPU= 29.791 02:04:19 STEP 1324 H=1324:00 +CPU= 19.695 02:04:38 STEP 1325 H=1325:00 +CPU= 19.090 02:05:09 STEP 1326 H=1326:00 +CPU= 30.483 02:05:28 STEP 1327 H=1327:00 +CPU= 19.181 02:05:47 STEP 1328 H=1328:00 +CPU= 19.715 02:06:17 STEP 1329 H=1329:00 +CPU= 29.803 02:06:37 STEP 1330 H=1330:00 +CPU= 19.710 02:06:56 STEP 1331 H=1331:00 +CPU= 19.145 02:07:29 STEP 1332 H=1332:00 +CPU= 33.151 Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+001332 Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+001332 Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+001332 02:07:48 STEP 1333 H=1333:00 +CPU= 19.135 02:08:08 STEP 1334 H=1334:00 +CPU= 19.701 02:08:38 STEP 1335 H=1335:00 +CPU= 29.889 02:08:58 STEP 1336 H=1336:00 +CPU= 19.681 02:09:17 STEP 1337 H=1337:00 +CPU= 19.107 02:09:47 STEP 1338 H=1338:00 +CPU= 30.432 02:10:06 STEP 1339 H=1339:00 +CPU= 19.151 02:10:26 STEP 1340 H=1340:00 +CPU= 19.724 02:10:56 STEP 1341 H=1341:00 +CPU= 29.787 02:11:16 STEP 1342 H=1342:00 +CPU= 19.698 02:11:35 STEP 1343 H=1343:00 +CPU= 19.681 02:12:11 STEP 1344 H=1344:00 +CPU= 35.239 Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+001344 Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+001344 Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+001344 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185048/ICMGGhq0f+001320 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185048/ICMSHhq0f+001320 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185048/ICMUAhq0f+001320 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185048/ICMGGhq0f+001332 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185048/ICMSHhq0f+001332 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185048/ICMUAhq0f+001332 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0404_1999050100_123_968_12185048_1_r578666857_25.zip Uploading the intermediate file: upload_file_25.zip 02:12:30 STEP 1345 H=1345:00 +CPU= 19.186 02:12:49 STEP 1346 H=1346:00 +CPU= 19.690 02:13:19 STEP 1347 H=1347:00 +CPU= 29.795 02:13:39 STEP 1348 H=1348:00 +CPU= 19.976 02:13:58 STEP 1349 H=1349:00 +CPU= 19.144 02:14:29 STEP 1350 H=1350:00 +CPU= 30.397 02:14:48 STEP 1351 H=1351:00 +CPU= 19.115 02:15:08 STEP 1352 H=1352:00 +CPU= 19.693 02:15:37 STEP 1353 H=1353:00 +CPU= 29.779 02:15:57 STEP 1354 H=1354:00 +CPU= 19.674 02:16:16 STEP 1355 H=1355:00 +CPU= 19.176 02:16:49 STEP 1356 H=1356:00 +CPU= 33.156 Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+001356 Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+001356 Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+001356 02:17:09 STEP 1357 H=1357:00 +CPU= 19.136 02:17:28 STEP 1358 H=1358:00 +CPU= 19.690 02:17:58 STEP 1359 H=1359:00 +CPU= 29.789 02:18:18 STEP 1360 H=1360:00 +CPU= 19.751 02:18:37 STEP 1361 H=1361:00 +CPU= 19.235 02:19:08 STEP 1362 H=1362:00 +CPU= 30.421 02:19:27 STEP 1363 H=1363:00 +CPU= 19.104 02:19:46 STEP 1364 H=1364:00 +CPU= 19.693 02:20:16 STEP 1365 H=1365:00 +CPU= 29.756 02:20:36 STEP 1366 H=1366:00 +CPU= 19.663 02:20:56 STEP 1367 H=1367:00 +CPU= 19.765 02:21:31 STEP 1368 H=1368:00 +CPU= 35.231 Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+001368 Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+001368 Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+001368 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185048/ICMGGhq0f+001344 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185048/ICMSHhq0f+001344 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185048/ICMUAhq0f+001344 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185048/ICMGGhq0f+001356 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185048/ICMSHhq0f+001356 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185048/ICMUAhq0f+001356 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0404_1999050100_123_968_12185048_1_r578666857_26.zip Uploading the intermediate file: upload_file_26.zip 02:21:50 STEP 1369 H=1369:00 +CPU= 19.214 02:22:10 STEP 1370 H=1370:00 +CPU= 19.726 02:22:40 STEP 1371 H=1371:00 +CPU= 29.789 02:22:59 STEP 1372 H=1372:00 +CPU= 19.697 02:23:18 STEP 1373 H=1373:00 +CPU= 19.212 02:23:52 STEP 1374 H=1374:00 +CPU= 33.122 Suspend request received from the BOINC client, suspending the child process Resuming the child process 02:24:25 STEP 1375 H=1375:00 +CPU= 21.930 02:24:51 STEP 1376 H=1376:00 +CPU= 24.719 02:25:26 STEP 1377 H=1377:00 +CPU= 34.325 Suspend request received from the BOINC client, suspending the child process Resuming the child process 02:26:26 STEP 1378 H=1378:00 +CPU= 26.117 02:26:50 STEP 1379 H=1379:00 +CPU= 23.078 Suspend request received from the BOINC client, suspending the child process Resuming the child process 02:27:55 STEP 1380 H=1380:00 +CPU= 42.291 Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+001380 Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+001380 Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+001380 02:28:18 STEP 1381 H=1381:00 +CPU= 22.341 Suspend request received from the BOINC client, suspending the child process Resuming the child process 02:28:57 STEP 1382 H=1382:00 +CPU= 27.051 02:29:31 STEP 1383 H=1383:00 +CPU= 33.303 Suspend request received from the BOINC client, suspending the child process Resuming the child process 02:30:08 STEP 1384 H=1384:00 +CPU= 24.946 Suspend request received from the BOINC client, suspending the child process Resuming the child process 02:31:37 STEP 1385 H=1385:00 +CPU= 26.240 02:32:12 STEP 1386 H=1386:00 +CPU= 33.709 Suspend request received from the BOINC client, suspending the child process Resuming the child process 02:32:46 STEP 1387 H=1387:00 +CPU= 22.661 02:33:08 STEP 1388 H=1388:00 +CPU= 21.321 02:33:40 STEP 1389 H=1389:00 +CPU= 31.491 Suspend request received from the BOINC client, suspending the child process Resuming the child process 02:34:14 STEP 1390 H=1390:00 +CPU= 22.787 02:34:37 STEP 1391 H=1391:00 +CPU= 22.194 Suspend request received from the BOINC client, suspending the child process Resuming the child process 02:35:28 STEP 1392 H=1392:00 +CPU= 39.773 Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+001392 Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+001392 Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+001392 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185048/ICMGGhq0f+001368 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185048/ICMSHhq0f+001368 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185048/ICMUAhq0f+001368 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185048/ICMGGhq0f+001380 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185048/ICMSHhq0f+001380 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185048/ICMUAhq0f+001380 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0404_1999050100_123_968_12185048_1_r578666857_27.zip Uploading the intermediate file: upload_file_27.zip 02:35:53 STEP 1393 H=1393:00 +CPU= 23.730 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 02:36:40 STEP 1394 H=1394:00 +CPU= 24.271 02:37:11 STEP 1395 H=1395:00 +CPU= 30.935 02:37:31 STEP 1396 H=1396:00 +CPU= 19.833 02:37:53 STEP 1397 H=1397:00 +CPU= 21.625 02:38:26 STEP 1398 H=1398:00 +CPU= 32.229 02:38:47 STEP 1399 H=1399:00 +CPU= 20.544 02:39:09 STEP 1400 H=1400:00 +CPU= 21.450 02:39:39 STEP 1401 H=1401:00 +CPU= 30.021 02:40:00 STEP 1402 H=1402:00 +CPU= 20.735 02:40:20 STEP 1403 H=1403:00 +CPU= 20.238 02:40:57 STEP 1404 H=1404:00 +CPU= 36.093 Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+001404 Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+001404 Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+001404 02:41:21 STEP 1405 H=1405:00 +CPU= 23.570 02:41:44 STEP 1406 H=1406:00 +CPU= 22.428 Suspend request received from the BOINC client, suspending the child process Resuming the child process 02:42:42 STEP 1407 H=1407:00 +CPU= 34.973 02:43:04 STEP 1408 H=1408:00 +CPU= 21.575 Suspend request received from the BOINC client, suspending the child process Resuming the child process 02:43:42 STEP 1409 H=1409:00 +CPU= 25.780 Suspend request received from the BOINC client, suspending the child process Resuming the child process 02:44:43 STEP 1410 H=1410:00 +CPU= 38.511 02:45:05 STEP 1411 H=1411:00 +CPU= 21.550 02:45:28 STEP 1412 H=1412:00 +CPU= 22.226 02:46:02 STEP 1413 H=1413:00 +CPU= 33.594 02:46:27 STEP 1414 H=1414:00 +CPU= 24.089 Suspend request received from the BOINC client, suspending the child process Resuming the child process 02:47:14 STEP 1415 H=1415:00 +CPU= 23.815 02:47:53 STEP 1416 H=1416:00 +CPU= 38.805 Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+001416 Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+001416 Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+001416 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185048/ICMGGhq0f+001392 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185048/ICMSHhq0f+001392 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185048/ICMUAhq0f+001392 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185048/ICMGGhq0f+001404 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185048/ICMSHhq0f+001404 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185048/ICMUAhq0f+001404 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0404_1999050100_123_968_12185048_1_r578666857_28.zip Uploading the intermediate file: upload_file_28.zip 02:48:14 STEP 1417 H=1417:00 +CPU= 20.662 Uploading trickle at timestep: 5097600 Suspend request received from the BOINC client, suspending the child process Resuming the child process 02:48:53 STEP 1418 H=1418:00 +CPU= 26.312 02:49:24 STEP 1419 H=1419:00 +CPU= 31.052 02:49:48 STEP 1420 H=1420:00 +CPU= 22.797 02:50:09 STEP 1421 H=1421:00 +CPU= 21.122 Suspend request received from the BOINC client, suspending the child process Resuming the child process 02:50:57 STEP 1422 H=1422:00 +CPU= 35.969 02:51:19 STEP 1423 H=1423:00 +CPU= 21.538 02:51:42 STEP 1424 H=1424:00 +CPU= 21.854 02:52:14 STEP 1425 H=1425:00 +CPU= 31.598 02:52:35 STEP 1426 H=1426:00 +CPU= 21.533 02:53:01 STEP 1427 H=1427:00 +CPU= 24.566 Suspend request received from the BOINC client, suspending the child process Resuming the child process 02:53:52 STEP 1428 H=1428:00 +CPU= 38.867 Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+001428 Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+001428 Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+001428 02:54:16 STEP 1429 H=1429:00 +CPU= 22.742 02:54:40 STEP 1430 H=1430:00 +CPU= 23.869 02:55:13 STEP 1431 H=1431:00 +CPU= 31.959 02:55:33 STEP 1432 H=1432:00 +CPU= 20.631 02:55:58 STEP 1433 H=1433:00 +CPU= 23.476 Suspend request received from the BOINC client, suspending the child process Resuming the child process 02:56:49 STEP 1434 H=1434:00 +CPU= 37.859 02:57:15 STEP 1435 H=1435:00 +CPU= 24.690 Suspend request received from the BOINC client, suspending the child process Resuming the child process 02:58:25 STEP 1436 H=1436:00 +CPU= 27.359 Suspend request received from the BOINC client, suspending the child process Resuming the child process 02:59:20 STEP 1437 H=1437:00 +CPU= 41.079 Suspend request received from the BOINC client, suspending the child process Resuming the child process 03:00:09 STEP 1438 H=1438:00 +CPU= 26.939 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 03:01:00 STEP 1439 H=1439:00 +CPU= 27.297 03:01:41 STEP 1440 H=1440:00 +CPU= 40.235 Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+001440 Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+001440 Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+001440 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185048/ICMGGhq0f+001416 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185048/ICMSHhq0f+001416 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185048/ICMUAhq0f+001416 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185048/ICMGGhq0f+001428 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185048/ICMSHhq0f+001428 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185048/ICMUAhq0f+001428 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0404_1999050100_123_968_12185048_1_r578666857_29.zip Uploading the intermediate file: upload_file_29.zip 03:02:02 STEP 1441 H=1441:00 +CPU= 20.581 03:02:23 STEP 1442 H=1442:00 +CPU= 20.801 Suspend request received from the BOINC client, suspending the child process Resuming the child process 03:03:16 STEP 1443 H=1443:00 +CPU= 39.704 03:03:42 STEP 1444 H=1444:00 +CPU= 25.224 [EC_DRHOOK:ruly-Inspiron-5558:1:1:1920:1920] [20230225:030359:1677323039:36338.782] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1538] Received signal#15 (SIGTERM) :: 3996MB (heap), 4419MB (maxrss), 0MB (maxstack), 0 (paging), nsigs = 1 [EC_DRHOOK:ruly-Inspiron-5558:1:1:1920:1920] [20230225:030359:1677323039:36338.782] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1542] Also activating Harakiri-alarm (SIGALRM=14) to expire after 500s elapsed to prevent hangs, nsigs = 1 [EC_DRHOOK:ruly-Inspiron-5558:1:1:1920:1920] [20230225:030359:1677323039:36338.782] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1544] Harakiri signal handler 'signal_harakiri' for signal#14 (SIGALRM) installed at 0x81f3c0 (old at (nil)) [EC_DRHOOK:ruly-Inspiron-5558:1:1:1920:1920] [20230225:030359:1677323039:36338.782] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1617] Signal#15 was caused by unrecognized si_code [memaddr=0x1], nsigs = 1 [EC_DRHOOK:ruly-Inspiron-5558:1:1:1920:1920] [20230225:030359:1677323039:36338.782] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1686] Starting DrHook backtrace for signal#15, nsigs = 1 [EC_DRHOOK:ruly-Inspiron-5558:1:1:1920:1920] [20230225:030359:1677323039:36338.782] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3843] 3996 MB (maxheap), 4419 MB (maxrss), 0 MB (maxstack) [EC_DRHOOK:ruly-Inspiron-5558:1:1:1920:1920] [20230225:030359:1677323039:36338.782] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : MASTER [EC_DRHOOK:ruly-Inspiron-5558:1:1:1920:1920] [20230225:030359:1677323039:36338.782] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT0 [EC_DRHOOK:ruly-Inspiron-5558:1:1:1920:1920] [20230225:030359:1677323039:36338.782] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT1 [EC_DRHOOK:ruly-Inspiron-5558:1:1:1920:1920] [20230225:030359:1677323039:36338.782] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT2 [EC_DRHOOK:ruly-Inspiron-5558:1:1:1920:1920] [20230225:030359:1677323039:36338.782] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT3 [EC_DRHOOK:ruly-Inspiron-5558:1:1:1920:1920] [20230225:030359:1677323039:36338.782] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT4 [EC_DRHOOK:ruly-Inspiron-5558:1:1:1920:1920] [20230225:030359:1677323039:36338.782] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : STEPO [EC_DRHOOK:ruly-Inspiron-5558:1:1:1920:1920] [20230225:030359:1677323039:36338.782] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : SCAN2M [EC_DRHOOK:ruly-Inspiron-5558:1:1:1920:1920] [20230225:030359:1677323039:36338.782] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : GP_MODEL_HEAP [EC_DRHOOK:ruly-Inspiron-5558:1:1:1920:1920] [20230225:030359:1677323039:36338.782] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : GP_MODEL [EC_DRHOOK:ruly-Inspiron-5558:1:1:1920:1920] [20230225:030359:1677323039:36338.782] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : EC_PHYS_DRV [EC_DRHOOK:ruly-Inspiron-5558:1:1:1920:1920] [20230225:030359:1677323039:36338.782] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : >OMP-PHYSICS (1001) [EC_DRHOOK:ruly-Inspiron-5558:1:1:1920:1920] [20230225:030359:1677323039:36338.782] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : EC_PHYS [EC_DRHOOK:ruly-Inspiron-5558:1:1:1920:1920] [20230225:030359:1677323039:36338.782] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CALLPAR [EC_DRHOOK:ruly-Inspiron-5558:1:1:1920:1920] [20230225:030359:1677323039:36338.782] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CLOUD_LAYER [EC_DRHOOK:ruly-Inspiron-5558:1:1:1920:1920] [20230225:030359:1677323039:36338.782] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CLOUDSC [EC_DRHOOK:ruly-Inspiron-5558:1:1:1920:1920] [20230225:030400:1677323040:36339.784] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1734] DrHook backtrace done for signal#15, nsigs = 1 [EC_DRHOOK:ruly-Inspiron-5558:1:1:1920:1920] [20230225:030400:1677323040:36339.784] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1785] Calling previous signal handler at 0x1ce8bb0 for signal#15, nsigs = 1 forrtl: error (78): process killed (SIGTERM) Image PC Routine Line Source oifs_43r3_model.e 0000000001CE916B Unknown Unknown Unknown oifs_43r3_model.e 000000000081FFF1 Unknown Unknown Unknown oifs_43r3_model.e 0000000001DC9090 Unknown Unknown Unknown oifs_43r3_model.e 0000000001038431 cloudsc_ 727 cloudsc.F90 oifs_43r3_model.e 0000000001014D57 cloud_layer_ 131 cloud_layer.F90 oifs_43r3_model.e 0000000000F1A790 callpar_ 1223 callpar.F90 oifs_43r3_model.e 0000000000E4940F ec_phys_ 670 ec_phys.F90 oifs_43r3_model.e 0000000000E3610C ec_phys_drv_ 599 ec_phys_drv.F90 oifs_43r3_model.e 0000000000BFA3D3 gp_model_ 613 gp_model.F90 oifs_43r3_model.e 00000000012E9482 gp_model_heap_ 74 gp_model_heap.F90 oifs_43r3_model.e 0000000000BBC1CE scan2m_ 535 scan2m.F90 oifs_43r3_model.e 000000000057A664 stepo_ 327 stepo.F90 oifs_43r3_model.e 000000000055DEEC cnt4_ 1133 cnt4.F90 oifs_43r3_model.e 00000000005471C9 cnt3_ 267 cnt3.F90 oifs_43r3_model.e 0000000000546412 cnt2_ 88 cnt2.F90 oifs_43r3_model.e 0000000000545E28 cnt1_ 92 cnt1.F90 oifs_43r3_model.e 000000000040708D cnt0_ 146 cnt0.F90 oifs_43r3_model.e 000000000040220F MAIN__ 96 master.F90 oifs_43r3_model.e 00000000004021A2 Unknown Unknown Unknown oifs_43r3_model.e 0000000001DCA390 Unknown Unknown Unknown oifs_43r3_model.e 000000000040206E Unknown Unknown Unknown (argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu (argv1) start_date: 1999050100 (argv2) exptid: hq0f (argv3) unique_member_id: 0404 (argv4) batchid: 968 (argv5) wuid: 12185048 (argv6) fclen: 123 (argv7) app_name: oifs_43r3_ps (argv8) nthreads: 1 Working directory is: /var/lib/boinc-client/slots/1 Project directory is: /var/lib/boinc-client/projects/climateprediction.net/ app name: oifs_43r3_ps version: 1.05 Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185048 ..mkdir for temp folder for results failed Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/1/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip Unzipping the app zip file: /var/lib/boinc-client/slots/1/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip Copying the namelist files from: ../../projects/climateprediction.net/jf_225b339456f5bf3a9c69f53378c00e4f to: /var/lib/boinc-client/slots/1/oifs_43r3_ps_0404_1999050100_123_968_12185048.zip Unzipping the namelist zip file: /var/lib/boinc-client/slots/1/oifs_43r3_ps_0404_1999050100_123_968_12185048.zip ic_ancil_file: ic_ancil_12185048 ifsdata_file: ifsdata_12185048 climate_data_file: clim_data_12185048 horiz_resolution: 159 vert_resolution: 60 grid_type: l_2 upload_interval: 24 utstep: 3600 nfrres: restart dump frequency (steps) 24 Copying IC ancils from: ../../projects/climateprediction.net/jf_b1bd23a97400b970c5860cd01c824e8d to: /var/lib/boinc-client/slots/1/ic_ancil_12185048.zip Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/1/ic_ancil_12185048.zip ..mkdir for ifsdata folder failed Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_684e9d374803bdcac30fb7de32ac0032 to: /var/lib/boinc-client/slots/1/ifsdata/ifsdata_12185048.zip Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/1/ifsdata/ifsdata_12185048.zip ..mkdir for the climate data folder failed Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/1/159l_2/clim_data_12185048.zip Unzipping the climate data zip file: /var/lib/boinc-client/slots/1/159l_2/clim_data_12185048.zip Checking for progress XML file: /var/lib/boinc-client/slots/1/progress_file_12185048.xml Opened progress file ok : /var/lib/boinc-client/slots/1/progress_file_12185048.xml -- Model is restarting -- Adjusting last_iter, 1445, to previous model restart step. Creating progress file: /var/lib/boinc-client/slots/1/progress_file_12185048.xml last_cpu_time: 19790 upload_file_number: 30 last_iter: 1439 last_upload: 5184000 model_completed: 0 total_length_of_simulation: 10627200 result_base_name: oifs_43r3_ps_0404_1999050100_123_968_12185048_1_r578666857 The child process has been launched with process id: 1902 Executing the command: /var/lib/boinc-client/slots/1/oifs_43r3_model.exe [EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1 [EC_DRHOOK:ruly-Inspiron-5558:1:1:1902:1902] [20230225:185002:1677379802:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0 [EC_DRHOOK:ruly-Inspiron-5558:1:1:1902:1902] [20230225:185002:1677379802:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1 [EC_DRHOOK:ruly-Inspiron-5558:1:1:1902:1902] [20230225:185002:1677379802:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0) [EC_DRHOOK:ruly-Inspiron-5558:1:1:1902:1902] [20230225:185002:1677379802:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1 [EC_DRHOOK:ruly-Inspiron-5558:1:1:1902:1902] [20230225:185002:1677379802:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000 [EC_DRHOOK:ruly-Inspiron-5558:1:1:1902:1902] [20230225:185002:1677379802:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647) [EC_DRHOOK:ruly-Inspiron-5558:1:1:1902:1902] [20230225:185002:1677379802:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16 [EC_DRHOOK:ruly-Inspiron-5558:1:1:1902:1902] [20230225:185002:1677379802:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0 [EC_DRHOOK:ruly-Inspiron-5558:1:1:1902:1902] [20230225:185002:1677379802:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500 [EC_DRHOOK:ruly-Inspiron-5558:1:1:1902:1902] [20230225:185002:1677379802:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1 [EC_DRHOOK:ruly-Inspiron-5558:1:1:1902:1902] [20230225:185002:1677379802:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1 [EC_DRHOOK:ruly-Inspiron-5558:1:1:1902:1902] [20230225:185002:1677379802:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1 [EC_DRHOOK:ruly-Inspiron-5558:1:1:1902:1902] [20230225:185002:1677379802:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1 [EC_DRHOOK:ruly-Inspiron-5558:1:1:1902:1902] [20230225:185002:1677379802:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef> [EC_DRHOOK:ruly-Inspiron-5558:1:1:1902:1902] [20230225:185002:1677379802:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef> [EC_DRHOOK:ruly-Inspiron-5558:1:1:1902:1902] [20230225:185002:1677379802:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:ruly-Inspiron-5558:1:1:1902:1902] [20230225:185002:1677379802:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil)) [EC_DRHOOK:ruly-Inspiron-5558:1:1:1902:1902] [20230225:185002:1677379802:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:ruly-Inspiron-5558:1:1:1902:1902] [20230225:185002:1677379802:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil)) [EC_DRHOOK:ruly-Inspiron-5558:1:1:1902:1902] [20230225:185002:1677379802:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1 [EC_DRHOOK:ruly-Inspiron-5558:1:1:1902:1902] [20230225:185002:1677379802:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:ruly-Inspiron-5558:1:1:1902:1902] [20230225:185002:1677379802:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:ruly-Inspiron-5558:1:1:1902:1902] [20230225:185002:1677379802:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil)) [EC_DRHOOK:ruly-Inspiron-5558:1:1:1902:1902] [20230225:185002:1677379802:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:ruly-Inspiron-5558:1:1:1902:1902] [20230225:185002:1677379802:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:ruly-Inspiron-5558:1:1:1902:1902] [20230225:185002:1677379802:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:ruly-Inspiron-5558:1:1:1902:1902] [20230225:185002:1677379802:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil)) [EC_DRHOOK:ruly-Inspiron-5558:1:1:1902:1902] [20230225:185002:1677379802:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil)) [EC_DRHOOK:ruly-Inspiron-5558:1:1:1902:1902] [20230225:185002:1677379802:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil)) [EC_DRHOOK:ruly-Inspiron-5558:1:1:1902:1902] [20230225:185002:1677379802:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef> 02:50:42 STEP 1440 H=1440:00 +CPU= 34.216 02:51:01 STEP 1441 H=1441:00 +CPU= 19.067 02:51:24 STEP 1442 H=1442:00 +CPU= 21.675 02:51:54 STEP 1443 H=1443:00 +CPU= 30.500 02:52:14 STEP 1444 H=1444:00 +CPU= 19.760 02:52:33 STEP 1445 H=1445:00 +CPU= 19.066 02:53:09 STEP 1446 H=1446:00 +CPU= 35.550 02:53:37 STEP 1447 H=1447:00 +CPU= 27.381 02:54:00 STEP 1448 H=1448:00 +CPU= 22.783 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 02:57:06 STEP 1449 H=1449:00 +CPU= 49.037 02:57:36 STEP 1450 H=1450:00 +CPU= 28.463 02:58:05 STEP 1451 H=1451:00 +CPU= 27.516 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 02:59:20 STEP 1452 H=1452:00 +CPU= 49.815 Suspend request received from the BOINC client, suspending the child process Resuming the child process Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+001452 Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+001452 Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+001452 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 03:00:53 STEP 1453 H=1453:00 +CPU= 35.187 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 03:02:03 STEP 1454 H=1454:00 +CPU= 26.271 Suspend request received from the BOINC client, suspending the child process Resuming the child process 03:03:00 STEP 1455 H=1455:00 +CPU= 35.238 Suspend request received from the BOINC client, suspending the child process Resuming the child process 03:03:57 STEP 1456 H=1456:00 +CPU= 25.064 03:04:25 STEP 1457 H=1457:00 +CPU= 26.342 Suspend request received from the BOINC client, suspending the child process Resuming the child process 03:05:37 STEP 1458 H=1458:00 +CPU= 38.658 03:05:58 STEP 1459 H=1459:00 +CPU= 21.072 03:06:19 STEP 1460 H=1460:00 +CPU= 20.402 03:06:49 STEP 1461 H=1461:00 +CPU= 30.127 03:07:09 STEP 1462 H=1462:00 +CPU= 20.152 03:07:29 STEP 1463 H=1463:00 +CPU= 19.999 03:08:07 STEP 1464 H=1464:00 +CPU= 36.843 Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+001464 Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+001464 Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+001464 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185048/ICMGGhq0f+001440 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185048/ICMSHhq0f+001440 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185048/ICMUAhq0f+001440 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185048/ICMGGhq0f+001452 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185048/ICMSHhq0f+001452 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185048/ICMUAhq0f+001452 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0404_1999050100_123_968_12185048_1_r578666857_30.zip Uploading the intermediate file: upload_file_30.zip Suspend request received from the BOINC client, suspending the child process Resuming the child process 03:08:55 STEP 1465 H=1465:00 +CPU= 27.689 03:09:20 STEP 1466 H=1466:00 +CPU= 23.775 Suspend request received from the BOINC client, suspending the child process Resuming the child process 03:10:23 STEP 1467 H=1467:00 +CPU= 40.231 Suspend request received from the BOINC client, suspending the child process Resuming the child process 03:11:01 STEP 1468 H=1468:00 +CPU= 25.124 03:11:25 STEP 1469 H=1469:00 +CPU= 23.239 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 03:13:13 STEP 1470 H=1470:00 +CPU= 42.625 Suspend request received from the BOINC client, suspending the child process Resuming the child process 03:13:53 STEP 1471 H=1471:00 +CPU= 27.431 Suspend request received from the BOINC client, suspending the child process Resuming the child process 03:14:44 STEP 1472 H=1472:00 +CPU= 28.078 03:15:23 STEP 1473 H=1473:00 +CPU= 38.005 Suspend request received from the BOINC client, suspending the child process Resuming the child process 03:16:15 STEP 1474 H=1474:00 +CPU= 28.098 03:16:35 STEP 1475 H=1475:00 +CPU= 19.955 03:17:10 STEP 1476 H=1476:00 +CPU= 34.761 Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+001476 Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+001476 Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+001476 03:17:30 STEP 1477 H=1477:00 +CPU= 19.749 Suspend request received from the BOINC client, suspending the child process Resuming the child process 03:18:05 STEP 1478 H=1478:00 +CPU= 23.611 [EC_DRHOOK:ruly-Inspiron-5558:1:1:1902:1902] [20230226:031826:1677410306:30503.688] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1538] Received signal#15 (SIGTERM) :: 3996MB (heap), 4419MB (maxrss), 0MB (maxstack), 0 (paging), nsigs = 1 [EC_DRHOOK:ruly-Inspiron-5558:1:1:1902:1902] [20230226:031826:1677410306:30503.688] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1542] Also activating Harakiri-alarm (SIGALRM=14) to expire after 500s elapsed to prevent hangs, nsigs = 1 [EC_DRHOOK:ruly-Inspiron-5558:1:1:1902:1902] [20230226:031826:1677410306:30503.689] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1544] Harakiri signal handler 'signal_harakiri' for signal#14 (SIGALRM) installed at 0x81f3c0 (old at (nil)) [EC_DRHOOK:ruly-Inspiron-5558:1:1:1902:1902] [20230226:031826:1677410306:30503.689] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1617] Signal#15 was caused by unrecognized si_code [memaddr=0x1], nsigs = 1 [EC_DRHOOK:ruly-Inspiron-5558:1:1:1902:1902] [20230226:031826:1677410306:30503.689] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1686] Starting DrHook backtrace for signal#15, nsigs = 1 [EC_DRHOOK:ruly-Inspiron-5558:1:1:1902:1902] [20230226:031826:1677410306:30503.689] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3843] 3996 MB (maxheap), 4419 MB (maxrss), 0 MB (maxstack) [EC_DRHOOK:ruly-Inspiron-5558:1:1:1902:1902] [20230226:031826:1677410306:30503.689] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : MASTER [EC_DRHOOK:ruly-Inspiron-5558:1:1:1902:1902] [20230226:031826:1677410306:30503.689] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT0 [EC_DRHOOK:ruly-Inspiron-5558:1:1:1902:1902] [20230226:031826:1677410306:30503.689] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT1 [EC_DRHOOK:ruly-Inspiron-5558:1:1:1902:1902] [20230226:031826:1677410306:30503.689] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT2 [EC_DRHOOK:ruly-Inspiron-5558:1:1:1902:1902] [20230226:031826:1677410306:30503.689] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT3 [EC_DRHOOK:ruly-Inspiron-5558:1:1:1902:1902] [20230226:031826:1677410306:30503.689] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT4 [EC_DRHOOK:ruly-Inspiron-5558:1:1:1902:1902] [20230226:031826:1677410306:30503.689] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : STEPO [EC_DRHOOK:ruly-Inspiron-5558:1:1:1902:1902] [20230226:031826:1677410306:30503.689] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : SCAN2M [EC_DRHOOK:ruly-Inspiron-5558:1:1:1902:1902] [20230226:031826:1677410306:30503.689] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : GP_MODEL_HEAP [EC_DRHOOK:ruly-Inspiron-5558:1:1:1902:1902] [20230226:031826:1677410306:30503.689] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : GP_MODEL [EC_DRHOOK:ruly-Inspiron-5558:1:1:1902:1902] [20230226:031826:1677410306:30503.689] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : EC_PHYS_DRV [EC_DRHOOK:ruly-Inspiron-5558:1:1:1902:1902] [20230226:031826:1677410306:30503.689] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : >OMP-PHYSICS CLDPP T/S (1002) [EC_DRHOOK:ruly-Inspiron-5558:1:1:1902:1902] [20230226:031826:1677410306:30503.689] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : EC_PHYS [EC_DRHOOK:ruly-Inspiron-5558:1:1:1902:1902] [20230226:031826:1677410306:30503.689] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CALLPAR [EC_DRHOOK:ruly-Inspiron-5558:1:1:1902:1902] [20230226:031826:1677410306:30503.689] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CLOUD_LAYER [EC_DRHOOK:ruly-Inspiron-5558:1:1:1902:1902] [20230226:031826:1677410306:30503.689] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CLOUDSC [EC_DRHOOK:ruly-Inspiron-5558:1:1:1902:1902] [20230226:031826:1677410306:30503.689] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CUADJTQ [EC_DRHOOK:ruly-Inspiron-5558:1:1:1902:1902] [20230226:031827:1677410307:30504.689] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1734] DrHook backtrace done for signal#15, nsigs = 1 [EC_DRHOOK:ruly-Inspiron-5558:1:1:1902:1902] [20230226:031827:1677410307:30504.689] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1785] Calling previous signal handler at 0x1ce8bb0 for signal#15, nsigs = 1 forrtl: error (78): process killed (SIGTERM) Image PC Routine Line Source oifs_43r3_model.e 0000000001CE916B Unknown Unknown Unknown oifs_43r3_model.e 000000000081FFF1 Unknown Unknown Unknown oifs_43r3_model.e 0000000001DC9090 Unknown Unknown Unknown oifs_43r3_model.e 0000000001CBB804 Unknown Unknown Unknown oifs_43r3_model.e 0000000000FBCB79 cuadjtq_ 299 cuadjtq.F90 oifs_43r3_model.e 00000000010255EE cloudsc_ 1522 cloudsc.F90 oifs_43r3_model.e 0000000001014D57 cloud_layer_ 131 cloud_layer.F90 oifs_43r3_model.e 0000000000F10C1B callpar_ 1011 callpar.F90 oifs_43r3_model.e 0000000000E4940F ec_phys_ 670 ec_phys.F90 oifs_43r3_model.e 0000000000E3610C ec_phys_drv_ 599 ec_phys_drv.F90 oifs_43r3_model.e 0000000000BFA3D3 gp_model_ 613 gp_model.F90 oifs_43r3_model.e 00000000012E9482 gp_model_heap_ 74 gp_model_heap.F90 oifs_43r3_model.e 0000000000BBC1CE scan2m_ 535 scan2m.F90 oifs_43r3_model.e 000000000057A664 stepo_ 327 stepo.F90 oifs_43r3_model.e 000000000055DEEC cnt4_ 1133 cnt4.F90 oifs_43r3_model.e 00000000005471C9 cnt3_ 267 cnt3.F90 oifs_43r3_model.e 0000000000546412 cnt2_ 88 cnt2.F90 oifs_43r3_model.e 0000000000545E28 cnt1_ 92 cnt1.F90 oifs_43r3_model.e 000000000040708D cnt0_ 146 cnt0.F90 oifs_43r3_model.e 000000000040220F MAIN__ 96 master.F90 oifs_43r3_model.e 00000000004021A2 Unknown Unknown Unknown oifs_43r3_model.e 0000000001DCA390 Unknown Unknown Unknown oifs_43r3_model.e 000000000040206E Unknown Unknown Unknown (argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu (argv1) start_date: 1999050100 (argv2) exptid: hq0f (argv3) unique_member_id: 0404 (argv4) batchid: 968 (argv5) wuid: 12185048 (argv6) fclen: 123 (argv7) app_name: oifs_43r3_ps (argv8) nthreads: 1 Working directory is: /var/lib/boinc-client/slots/1 Project directory is: /var/lib/boinc-client/projects/climateprediction.net/ app name: oifs_43r3_ps version: 1.05 Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185048 ..mkdir for temp folder for results failed Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/1/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip Unzipping the app zip file: /var/lib/boinc-client/slots/1/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip Copying the namelist files from: ../../projects/climateprediction.net/jf_225b339456f5bf3a9c69f53378c00e4f to: /var/lib/boinc-client/slots/1/oifs_43r3_ps_0404_1999050100_123_968_12185048.zip Unzipping the namelist zip file: /var/lib/boinc-client/slots/1/oifs_43r3_ps_0404_1999050100_123_968_12185048.zip ic_ancil_file: ic_ancil_12185048 ifsdata_file: ifsdata_12185048 climate_data_file: clim_data_12185048 horiz_resolution: 159 vert_resolution: 60 grid_type: l_2 upload_interval: 24 utstep: 3600 nfrres: restart dump frequency (steps) 24 Copying IC ancils from: ../../projects/climateprediction.net/jf_b1bd23a97400b970c5860cd01c824e8d to: /var/lib/boinc-client/slots/1/ic_ancil_12185048.zip Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/1/ic_ancil_12185048.zip ..mkdir for ifsdata folder failed Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_684e9d374803bdcac30fb7de32ac0032 to: /var/lib/boinc-client/slots/1/ifsdata/ifsdata_12185048.zip Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/1/ifsdata/ifsdata_12185048.zip ..mkdir for the climate data folder failed Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/1/159l_2/clim_data_12185048.zip Unzipping the climate data zip file: /var/lib/boinc-client/slots/1/159l_2/clim_data_12185048.zip Checking for progress XML file: /var/lib/boinc-client/slots/1/progress_file_12185048.xml Opened progress file ok : /var/lib/boinc-client/slots/1/progress_file_12185048.xml -- Model is restarting -- Adjusting last_iter, 1479, to previous model restart step. Creating progress file: /var/lib/boinc-client/slots/1/progress_file_12185048.xml last_cpu_time: 20928 upload_file_number: 31 last_iter: 1463 last_upload: 5270400 model_completed: 0 total_length_of_simulation: 10627200 result_base_name: oifs_43r3_ps_0404_1999050100_123_968_12185048_1_r578666857 The child process has been launched with process id: 1897 Executing the command: /var/lib/boinc-client/slots/1/oifs_43r3_model.exe [EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1 [EC_DRHOOK:ruly-Inspiron-5558:1:1:1897:1897] [20230226:193438:1677468878:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0 [EC_DRHOOK:ruly-Inspiron-5558:1:1:1897:1897] [20230226:193438:1677468878:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1 [EC_DRHOOK:ruly-Inspiron-5558:1:1:1897:1897] [20230226:193438:1677468878:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0) [EC_DRHOOK:ruly-Inspiron-5558:1:1:1897:1897] [20230226:193438:1677468878:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1 [EC_DRHOOK:ruly-Inspiron-5558:1:1:1897:1897] [20230226:193438:1677468878:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000 [EC_DRHOOK:ruly-Inspiron-5558:1:1:1897:1897] [20230226:193438:1677468878:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647) [EC_DRHOOK:ruly-Inspiron-5558:1:1:1897:1897] [20230226:193438:1677468878:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16 [EC_DRHOOK:ruly-Inspiron-5558:1:1:1897:1897] [20230226:193438:1677468878:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0 [EC_DRHOOK:ruly-Inspiron-5558:1:1:1897:1897] [20230226:193438:1677468878:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500 [EC_DRHOOK:ruly-Inspiron-5558:1:1:1897:1897] [20230226:193438:1677468878:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1 [EC_DRHOOK:ruly-Inspiron-5558:1:1:1897:1897] [20230226:193438:1677468878:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1 [EC_DRHOOK:ruly-Inspiron-5558:1:1:1897:1897] [20230226:193438:1677468878:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1 [EC_DRHOOK:ruly-Inspiron-5558:1:1:1897:1897] [20230226:193438:1677468878:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1 [EC_DRHOOK:ruly-Inspiron-5558:1:1:1897:1897] [20230226:193438:1677468878:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef> [EC_DRHOOK:ruly-Inspiron-5558:1:1:1897:1897] [20230226:193438:1677468878:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef> [EC_DRHOOK:ruly-Inspiron-5558:1:1:1897:1897] [20230226:193438:1677468878:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:ruly-Inspiron-5558:1:1:1897:1897] [20230226:193438:1677468878:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil)) [EC_DRHOOK:ruly-Inspiron-5558:1:1:1897:1897] [20230226:193438:1677468878:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:ruly-Inspiron-5558:1:1:1897:1897] [20230226:193438:1677468878:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil)) [EC_DRHOOK:ruly-Inspiron-5558:1:1:1897:1897] [20230226:193438:1677468878:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1 [EC_DRHOOK:ruly-Inspiron-5558:1:1:1897:1897] [20230226:193438:1677468878:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:ruly-Inspiron-5558:1:1:1897:1897] [20230226:193438:1677468878:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:ruly-Inspiron-5558:1:1:1897:1897] [20230226:193438:1677468878:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil)) [EC_DRHOOK:ruly-Inspiron-5558:1:1:1897:1897] [20230226:193438:1677468878:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:ruly-Inspiron-5558:1:1:1897:1897] [20230226:193438:1677468878:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:ruly-Inspiron-5558:1:1:1897:1897] [20230226:193438:1677468878:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:ruly-Inspiron-5558:1:1:1897:1897] [20230226:193438:1677468878:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil)) [EC_DRHOOK:ruly-Inspiron-5558:1:1:1897:1897] [20230226:193438:1677468878:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil)) [EC_DRHOOK:ruly-Inspiron-5558:1:1:1897:1897] [20230226:193438:1677468878:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil)) [EC_DRHOOK:ruly-Inspiron-5558:1:1:1897:1897] [20230226:193438:1677468878:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef> Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 03:37:16 STEP 1464 H=1464:00 +CPU= 45.459 03:37:46 STEP 1465 H=1465:00 +CPU= 28.325 03:38:09 STEP 1466 H=1466:00 +CPU= 22.425 03:38:46 STEP 1467 H=1467:00 +CPU= 35.896 03:39:11 STEP 1468 H=1468:00 +CPU= 23.990 03:39:34 STEP 1469 H=1469:00 +CPU= 22.065 03:40:07 STEP 1470 H=1470:00 +CPU= 32.720 03:40:27 STEP 1471 H=1471:00 +CPU= 20.380 03:40:49 STEP 1472 H=1472:00 +CPU= 21.125 03:41:25 STEP 1473 H=1473:00 +CPU= 35.373 03:41:47 STEP 1474 H=1474:00 +CPU= 21.177 03:42:08 STEP 1475 H=1475:00 +CPU= 20.988 03:42:47 STEP 1476 H=1476:00 +CPU= 37.493 Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+001476 Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+001476 Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+001476 03:43:06 STEP 1477 H=1477:00 +CPU= 19.469 03:43:26 STEP 1478 H=1478:00 +CPU= 19.755 03:43:57 STEP 1479 H=1479:00 +CPU= 31.066 03:44:17 STEP 1480 H=1480:00 +CPU= 19.818 03:44:37 STEP 1481 H=1481:00 +CPU= 19.350 03:45:09 STEP 1482 H=1482:00 +CPU= 31.708 03:45:28 STEP 1483 H=1483:00 +CPU= 19.588 03:45:52 STEP 1484 H=1484:00 +CPU= 23.454 03:46:24 STEP 1485 H=1485:00 +CPU= 31.095 03:46:43 STEP 1486 H=1486:00 +CPU= 19.690 03:47:03 STEP 1487 H=1487:00 +CPU= 19.726 03:47:39 STEP 1488 H=1488:00 +CPU= 35.207 Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+001488 Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+001488 Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+001488 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185048/ICMGGhq0f+001464 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185048/ICMSHhq0f+001464 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185048/ICMUAhq0f+001464 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185048/ICMGGhq0f+001476 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185048/ICMSHhq0f+001476 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185048/ICMUAhq0f+001476 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0404_1999050100_123_968_12185048_1_r578666857_31.zip Uploading the intermediate file: upload_file_31.zip 03:47:58 STEP 1489 H=1489:00 +CPU= 19.152 03:48:17 STEP 1490 H=1490:00 +CPU= 19.701 03:48:47 STEP 1491 H=1491:00 +CPU= 29.876 03:49:07 STEP 1492 H=1492:00 +CPU= 19.715 03:49:26 STEP 1493 H=1493:00 +CPU= 19.149 03:49:57 STEP 1494 H=1494:00 +CPU= 30.579 Abort request received from BOINC client, ending the child process </stderr_txt> ]]> |
Latest Trickles Received | ||||||
---|---|---|---|---|---|---|
Time Sent (UTC) | Host ID | Result ID | Result Name | Timestep | CPU Time (sec) | Average (sec/TS) |
09 May 2023 16:53:32 | 1509518 | 22305870 | oifs_43r3_ps_0404_1999050100_123_968_12185048_1 | 5,097,600 | 19,011 | 0.0037 |
09 May 2023 16:52:33 | 1509518 | 22305870 | oifs_43r3_ps_0404_1999050100_123_968_12185048_1 | 4,147,200 | 11,714 | 0.0028 |
09 May 2023 16:48:25 | 1509518 | 22305870 | oifs_43r3_ps_0404_1999050100_123_968_12185048_1 | 1,728,000 | 13,171 | 0.0076 |
09 May 2023 16:48:23 | 1509518 | 22305870 | oifs_43r3_ps_0404_1999050100_123_968_12185048_1 | 864,000 | 6,617 | 0.0077 |
©2024 cpdn.org