Name | oifs_43r3_ps_0898_1996050100_123_965_12182542_0 |
Workunit | 12182542 |
Created | 22 Dec 2022, 12:10:31 UTC |
Sent | 19 Jan 2023, 16:27:29 UTC |
Report deadline | 20 Mar 2023, 16:27:29 UTC |
Received | 25 Jan 2023, 8:00:30 UTC |
Server state | Over |
Outcome | Computation error |
Client state | Compute error |
Exit status | 196 (0x000000C4) EXIT_DISK_LIMIT_EXCEEDED |
Computer ID | 1478188 |
Run time | 1 days 1 hours 1 min 48 sec |
CPU time | 1 days 0 hours 37 min 10 sec |
Validate state | Invalid |
Credit | 11,733.12 |
Device peak FLOPS | 4.20 GFLOPS |
Application version | OpenIFS 43r3 Perturbed Surface v1.05 x86_64-pc-linux-gnu |
Peak working set size | 4,415.96 MB |
Peak swap size | 4,978.08 MB |
Peak disk usage | 7,749.07 MB |
Stderr | <core_client_version>7.20.2</core_client_version> <![CDATA[ <message> Disk usage limit exceeded</message> <stderr_txt> 244 14:34:54 STEP 2245 H=2245:00 +CPU= 25.466 14:35:19 STEP 2246 H=2246:00 +CPU= 24.309 14:36:05 STEP 2247 H=2247:00 +CPU= 44.346 14:36:32 STEP 2248 H=2248:00 +CPU= 27.062 14:37:01 STEP 2249 H=2249:00 +CPU= 27.712 14:37:46 STEP 2250 H=2250:00 +CPU= 43.873 14:38:12 STEP 2251 H=2251:00 +CPU= 25.675 14:38:40 STEP 2252 H=2252:00 +CPU= 27.027 14:39:24 STEP 2253 H=2253:00 +CPU= 43.629 14:39:52 STEP 2254 H=2254:00 +CPU= 26.775 14:40:20 STEP 2255 H=2255:00 +CPU= 27.053 14:41:13 STEP 2256 H=2256:00 +CPU= 52.179 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMGGhq0f+002256 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMSHhq0f+002256 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMUAhq0f+002256 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182542/ICMGGhq0f+002232 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182542/ICMSHhq0f+002232 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182542/ICMUAhq0f+002232 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182542/ICMGGhq0f+002244 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182542/ICMSHhq0f+002244 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182542/ICMUAhq0f+002244 Zipping up the intermediate file: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0898_1996050100_123_965_12182542_0_r28250566_93.zip Uploading the intermediate file: upload_file_93.zip 14:41:39 STEP 2257 H=2257:00 +CPU= 24.476 14:42:05 STEP 2258 H=2258:00 +CPU= 25.948 14:42:46 STEP 2259 H=2259:00 +CPU= 40.297 14:43:16 STEP 2260 H=2260:00 +CPU= 29.238 14:43:43 STEP 2261 H=2261:00 +CPU= 26.488 14:44:25 STEP 2262 H=2262:00 +CPU= 40.921 14:44:52 STEP 2263 H=2263:00 +CPU= 25.651 14:45:21 STEP 2264 H=2264:00 +CPU= 28.050 14:46:05 STEP 2265 H=2265:00 +CPU= 43.646 14:46:33 STEP 2266 H=2266:00 +CPU= 26.841 14:47:01 STEP 2267 H=2267:00 +CPU= 26.882 14:47:49 STEP 2268 H=2268:00 +CPU= 47.345 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMGGhq0f+002268 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMSHhq0f+002268 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMUAhq0f+002268 14:48:17 STEP 2269 H=2269:00 +CPU= 27.034 14:48:45 STEP 2270 H=2270:00 +CPU= 27.082 14:49:25 STEP 2271 H=2271:00 +CPU= 39.465 14:49:53 STEP 2272 H=2272:00 +CPU= 27.773 14:50:22 STEP 2273 H=2273:00 +CPU= 27.592 14:51:11 STEP 2274 H=2274:00 +CPU= 47.740 14:51:39 STEP 2275 H=2275:00 +CPU= 27.549 14:52:07 STEP 2276 H=2276:00 +CPU= 27.895 14:52:52 STEP 2277 H=2277:00 +CPU= 43.890 14:53:21 STEP 2278 H=2278:00 +CPU= 28.002 14:53:51 STEP 2279 H=2279:00 +CPU= 29.614 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 16:19:53 STEP 2280 H=2280:00 +CPU= 56.144 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMGGhq0f+002280 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMSHhq0f+002280 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMUAhq0f+002280 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182542/ICMGGhq0f+002256 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182542/ICMSHhq0f+002256 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182542/ICMUAhq0f+002256 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182542/ICMGGhq0f+002268 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182542/ICMSHhq0f+002268 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182542/ICMUAhq0f+002268 Zipping up the intermediate file: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0898_1996050100_123_965_12182542_0_r28250566_94.zip Uploading the intermediate file: upload_file_94.zip 16:20:21 STEP 2281 H=2281:00 +CPU= 28.244 16:20:46 STEP 2282 H=2282:00 +CPU= 23.892 16:21:30 STEP 2283 H=2283:00 +CPU= 42.647 16:21:56 STEP 2284 H=2284:00 +CPU= 26.243 16:22:25 STEP 2285 H=2285:00 +CPU= 27.706 16:23:07 STEP 2286 H=2286:00 +CPU= 41.722 16:23:33 STEP 2287 H=2287:00 +CPU= 24.753 16:24:02 STEP 2288 H=2288:00 +CPU= 29.062 16:24:44 STEP 2289 H=2289:00 +CPU= 40.562 16:25:07 STEP 2290 H=2290:00 +CPU= 22.737 16:25:35 STEP 2291 H=2291:00 +CPU= 27.256 16:26:25 STEP 2292 H=2292:00 +CPU= 48.706 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMGGhq0f+002292 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMSHhq0f+002292 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMUAhq0f+002292 16:26:50 STEP 2293 H=2293:00 +CPU= 24.141 16:27:19 STEP 2294 H=2294:00 +CPU= 27.817 16:28:07 STEP 2295 H=2295:00 +CPU= 47.291 16:28:36 STEP 2296 H=2296:00 +CPU= 28.561 16:29:02 STEP 2297 H=2297:00 +CPU= 25.700 16:29:45 STEP 2298 H=2298:00 +CPU= 41.243 16:30:12 STEP 2299 H=2299:00 +CPU= 27.170 16:30:41 STEP 2300 H=2300:00 +CPU= 27.775 16:31:22 STEP 2301 H=2301:00 +CPU= 40.206 16:31:50 STEP 2302 H=2302:00 +CPU= 27.584 16:32:17 STEP 2303 H=2303:00 +CPU= 25.814 16:33:02 STEP 2304 H=2304:00 +CPU= 43.189 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMGGhq0f+002304 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMSHhq0f+002304 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMUAhq0f+002304 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182542/ICMGGhq0f+002280 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182542/ICMSHhq0f+002280 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182542/ICMUAhq0f+002280 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182542/ICMGGhq0f+002292 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182542/ICMSHhq0f+002292 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182542/ICMUAhq0f+002292 Zipping up the intermediate file: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0898_1996050100_123_965_12182542_0_r28250566_95.zip Uploading the intermediate file: upload_file_95.zip 16:33:25 STEP 2305 H=2305:00 +CPU= 23.326 16:33:50 STEP 2306 H=2306:00 +CPU= 23.929 16:34:27 STEP 2307 H=2307:00 +CPU= 36.739 16:34:52 STEP 2308 H=2308:00 +CPU= 24.043 16:35:15 STEP 2309 H=2309:00 +CPU= 22.633 16:35:49 STEP 2310 H=2310:00 +CPU= 34.110 16:36:10 STEP 2311 H=2311:00 +CPU= 20.183 16:36:32 STEP 2312 H=2312:00 +CPU= 21.868 16:37:11 STEP 2313 H=2313:00 +CPU= 38.175 16:37:33 STEP 2314 H=2314:00 +CPU= 21.856 16:37:54 STEP 2315 H=2315:00 +CPU= 21.423 16:38:36 STEP 2316 H=2316:00 +CPU= 40.679 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMGGhq0f+002316 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMSHhq0f+002316 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMUAhq0f+002316 16:38:59 STEP 2317 H=2317:00 +CPU= 23.332 16:39:22 STEP 2318 H=2318:00 +CPU= 22.714 16:39:58 STEP 2319 H=2319:00 +CPU= 35.022 16:40:22 STEP 2320 H=2320:00 +CPU= 23.294 16:40:44 STEP 2321 H=2321:00 +CPU= 22.613 16:41:21 STEP 2322 H=2322:00 +CPU= 35.972 16:41:44 STEP 2323 H=2323:00 +CPU= 22.208 16:42:08 STEP 2324 H=2324:00 +CPU= 23.532 16:42:46 STEP 2325 H=2325:00 +CPU= 37.508 16:43:10 STEP 2326 H=2326:00 +CPU= 22.920 16:43:33 STEP 2327 H=2327:00 +CPU= 22.637 16:44:18 STEP 2328 H=2328:00 +CPU= 43.233 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMGGhq0f+002328 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMSHhq0f+002328 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMUAhq0f+002328 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182542/ICMGGhq0f+002304 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182542/ICMSHhq0f+002304 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182542/ICMUAhq0f+002304 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182542/ICMGGhq0f+002316 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182542/ICMSHhq0f+002316 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182542/ICMUAhq0f+002316 Zipping up the intermediate file: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0898_1996050100_123_965_12182542_0_r28250566_96.zip Uploading the intermediate file: upload_file_96.zip 16:44:41 STEP 2329 H=2329:00 +CPU= 22.315 Uploading trickle at timestep: 8380800 16:45:04 STEP 2330 H=2330:00 +CPU= 22.821 16:45:39 STEP 2331 H=2331:00 +CPU= 34.659 16:46:01 STEP 2332 H=2332:00 +CPU= 22.130 16:46:23 STEP 2333 H=2333:00 +CPU= 21.779 16:46:59 STEP 2334 H=2334:00 +CPU= 35.547 16:47:21 STEP 2335 H=2335:00 +CPU= 21.744 16:47:44 STEP 2336 H=2336:00 +CPU= 22.855 16:48:20 STEP 2337 H=2337:00 +CPU= 34.765 16:48:42 STEP 2338 H=2338:00 +CPU= 22.258 16:49:04 STEP 2339 H=2339:00 +CPU= 21.477 16:49:43 STEP 2340 H=2340:00 +CPU= 39.058 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMGGhq0f+002340 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMSHhq0f+002340 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMUAhq0f+002340 16:50:06 STEP 2341 H=2341:00 +CPU= 22.103 16:50:30 STEP 2342 H=2342:00 +CPU= 23.145 16:51:07 STEP 2343 H=2343:00 +CPU= 36.343 16:51:30 STEP 2344 H=2344:00 +CPU= 22.707 16:51:53 STEP 2345 H=2345:00 +CPU= 22.599 16:52:30 STEP 2346 H=2346:00 +CPU= 36.735 16:52:55 STEP 2347 H=2347:00 +CPU= 23.837 16:53:20 STEP 2348 H=2348:00 +CPU= 23.854 16:53:59 STEP 2349 H=2349:00 +CPU= 37.914 16:54:24 STEP 2350 H=2350:00 +CPU= 23.639 16:54:47 STEP 2351 H=2351:00 +CPU= 22.317 16:55:30 STEP 2352 H=2352:00 +CPU= 42.498 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMGGhq0f+002352 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMSHhq0f+002352 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMUAhq0f+002352 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182542/ICMGGhq0f+002328 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182542/ICMSHhq0f+002328 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182542/ICMUAhq0f+002328 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182542/ICMGGhq0f+002340 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182542/ICMSHhq0f+002340 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182542/ICMUAhq0f+002340 Zipping up the intermediate file: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0898_1996050100_123_965_12182542_0_r28250566_97.zip Uploading the intermediate file: upload_file_97.zip 16:55:53 STEP 2353 H=2353:00 +CPU= 22.274 16:56:16 STEP 2354 H=2354:00 +CPU= 22.299 16:56:51 STEP 2355 H=2355:00 +CPU= 34.597 16:57:14 STEP 2356 H=2356:00 +CPU= 22.504 16:57:36 STEP 2357 H=2357:00 +CPU= 21.561 16:58:12 STEP 2358 H=2358:00 +CPU= 36.318 16:58:34 STEP 2359 H=2359:00 +CPU= 21.781 16:58:57 STEP 2360 H=2360:00 +CPU= 21.820 16:59:32 STEP 2361 H=2361:00 +CPU= 34.677 16:59:55 STEP 2362 H=2362:00 +CPU= 22.808 17:00:17 STEP 2363 H=2363:00 +CPU= 22.018 17:00:56 STEP 2364 H=2364:00 +CPU= 38.232 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMGGhq0f+002364 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMSHhq0f+002364 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMUAhq0f+002364 17:01:18 STEP 2365 H=2365:00 +CPU= 21.314 17:01:40 STEP 2366 H=2366:00 +CPU= 22.529 17:02:15 STEP 2367 H=2367:00 +CPU= 34.269 17:02:37 STEP 2368 H=2368:00 +CPU= 22.136 17:02:59 STEP 2369 H=2369:00 +CPU= 21.715 17:03:36 STEP 2370 H=2370:00 +CPU= 35.850 17:03:57 STEP 2371 H=2371:00 +CPU= 21.295 17:04:20 STEP 2372 H=2372:00 +CPU= 22.956 17:04:55 STEP 2373 H=2373:00 +CPU= 33.926 17:05:16 STEP 2374 H=2374:00 +CPU= 21.552 17:05:39 STEP 2375 H=2375:00 +CPU= 22.266 17:06:20 STEP 2376 H=2376:00 +CPU= 40.633 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMGGhq0f+002376 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMSHhq0f+002376 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMUAhq0f+002376 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182542/ICMGGhq0f+002352 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182542/ICMSHhq0f+002352 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182542/ICMUAhq0f+002352 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182542/ICMGGhq0f+002364 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182542/ICMSHhq0f+002364 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182542/ICMUAhq0f+002364 Zipping up the intermediate file: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0898_1996050100_123_965_12182542_0_r28250566_98.zip Uploading the intermediate file: upload_file_98.zip 17:06:42 STEP 2377 H=2377:00 +CPU= 21.438 17:07:05 STEP 2378 H=2378:00 +CPU= 22.691 17:07:39 STEP 2379 H=2379:00 +CPU= 34.284 17:08:02 STEP 2380 H=2380:00 +CPU= 22.541 17:08:23 STEP 2381 H=2381:00 +CPU= 20.950 17:08:58 STEP 2382 H=2382:00 +CPU= 34.845 17:09:19 STEP 2383 H=2383:00 +CPU= 20.809 17:09:42 STEP 2384 H=2384:00 +CPU= 22.523 17:10:22 STEP 2385 H=2385:00 +CPU= 38.672 17:10:46 STEP 2386 H=2386:00 +CPU= 23.481 17:11:07 STEP 2387 H=2387:00 +CPU= 21.160 17:11:50 STEP 2388 H=2388:00 +CPU= 41.527 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMGGhq0f+002388 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMSHhq0f+002388 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMUAhq0f+002388 17:12:13 STEP 2389 H=2389:00 +CPU= 22.712 17:12:35 STEP 2390 H=2390:00 +CPU= 22.194 17:13:10 STEP 2391 H=2391:00 +CPU= 34.421 17:13:34 STEP 2392 H=2392:00 +CPU= 23.235 17:13:56 STEP 2393 H=2393:00 +CPU= 22.451 17:14:33 STEP 2394 H=2394:00 +CPU= 36.062 17:14:55 STEP 2395 H=2395:00 +CPU= 21.758 17:15:18 STEP 2396 H=2396:00 +CPU= 22.721 17:15:54 STEP 2397 H=2397:00 +CPU= 34.817 17:16:16 STEP 2398 H=2398:00 +CPU= 22.382 17:16:39 STEP 2399 H=2399:00 +CPU= 21.890 17:17:23 STEP 2400 H=2400:00 +CPU= 43.221 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMGGhq0f+002400 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMSHhq0f+002400 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMUAhq0f+002400 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182542/ICMGGhq0f+002376 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182542/ICMSHhq0f+002376 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182542/ICMUAhq0f+002376 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182542/ICMGGhq0f+002388 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182542/ICMSHhq0f+002388 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182542/ICMUAhq0f+002388 Zipping up the intermediate file: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0898_1996050100_123_965_12182542_0_r28250566_99.zip Uploading the intermediate file: upload_file_99.zip 17:17:46 STEP 2401 H=2401:00 +CPU= 22.031 17:18:10 STEP 2402 H=2402:00 +CPU= 23.538 17:18:45 STEP 2403 H=2403:00 +CPU= 35.105 17:19:09 STEP 2404 H=2404:00 +CPU= 23.118 17:19:32 STEP 2405 H=2405:00 +CPU= 22.382 17:20:11 STEP 2406 H=2406:00 +CPU= 37.761 17:20:34 STEP 2407 H=2407:00 +CPU= 22.436 17:20:59 STEP 2408 H=2408:00 +CPU= 24.287 17:21:38 STEP 2409 H=2409:00 +CPU= 37.606 17:22:01 STEP 2410 H=2410:00 +CPU= 22.135 17:22:24 STEP 2411 H=2411:00 +CPU= 22.945 17:23:06 STEP 2412 H=2412:00 +CPU= 40.744 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMGGhq0f+002412 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMSHhq0f+002412 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMUAhq0f+002412 17:23:28 STEP 2413 H=2413:00 +CPU= 21.841 17:23:51 STEP 2414 H=2414:00 +CPU= 22.251 17:24:26 STEP 2415 H=2415:00 +CPU= 34.424 17:24:49 STEP 2416 H=2416:00 +CPU= 22.659 17:25:10 STEP 2417 H=2417:00 +CPU= 21.417 17:25:47 STEP 2418 H=2418:00 +CPU= 36.253 17:26:09 STEP 2419 H=2419:00 +CPU= 21.419 17:26:32 STEP 2420 H=2420:00 +CPU= 23.166 17:27:08 STEP 2421 H=2421:00 +CPU= 34.641 17:27:31 STEP 2422 H=2422:00 +CPU= 23.406 17:27:54 STEP 2423 H=2423:00 +CPU= 22.433 17:28:36 STEP 2424 H=2424:00 +CPU= 41.564 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMGGhq0f+002424 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMSHhq0f+002424 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMUAhq0f+002424 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182542/ICMGGhq0f+002400 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182542/ICMSHhq0f+002400 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182542/ICMUAhq0f+002400 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182542/ICMGGhq0f+002412 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182542/ICMSHhq0f+002412 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182542/ICMUAhq0f+002412 Zipping up the intermediate file: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0898_1996050100_123_965_12182542_0_r28250566_100.zip Uploading the intermediate file: upload_file_100.zip 17:28:58 STEP 2425 H=2425:00 +CPU= 21.347 17:29:20 STEP 2426 H=2426:00 +CPU= 22.477 17:29:56 STEP 2427 H=2427:00 +CPU= 34.843 17:30:18 STEP 2428 H=2428:00 +CPU= 22.055 17:30:39 STEP 2429 H=2429:00 +CPU= 21.208 17:31:16 STEP 2430 H=2430:00 +CPU= 35.802 17:31:38 STEP 2431 H=2431:00 +CPU= 21.582 17:32:00 STEP 2432 H=2432:00 +CPU= 21.975 17:32:35 STEP 2433 H=2433:00 +CPU= 34.394 17:32:57 STEP 2434 H=2434:00 +CPU= 21.492 17:33:20 STEP 2435 H=2435:00 +CPU= 23.048 17:34:00 STEP 2436 H=2436:00 +CPU= 39.167 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMGGhq0f+002436 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMSHhq0f+002436 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMUAhq0f+002436 17:34:22 STEP 2437 H=2437:00 +CPU= 22.042 17:34:51 STEP 2438 H=2438:00 +CPU= 28.280 17:35:35 STEP 2439 H=2439:00 +CPU= 43.000 17:36:03 STEP 2440 H=2440:00 +CPU= 27.792 17:36:31 STEP 2441 H=2441:00 +CPU= 27.317 17:37:15 STEP 2442 H=2442:00 +CPU= 42.900 17:37:42 STEP 2443 H=2443:00 +CPU= 26.834 17:38:10 STEP 2444 H=2444:00 +CPU= 27.053 17:38:56 STEP 2445 H=2445:00 +CPU= 44.681 17:39:22 STEP 2446 H=2446:00 +CPU= 25.782 17:39:50 STEP 2447 H=2447:00 +CPU= 27.763 17:40:42 STEP 2448 H=2448:00 +CPU= 50.716 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMGGhq0f+002448 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMSHhq0f+002448 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMUAhq0f+002448 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182542/ICMGGhq0f+002424 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182542/ICMSHhq0f+002424 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182542/ICMUAhq0f+002424 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182542/ICMGGhq0f+002436 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182542/ICMSHhq0f+002436 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182542/ICMUAhq0f+002436 Zipping up the intermediate file: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0898_1996050100_123_965_12182542_0_r28250566_101.zip Uploading the intermediate file: upload_file_101.zip 17:41:08 STEP 2449 H=2449:00 +CPU= 25.151 17:41:35 STEP 2450 H=2450:00 +CPU= 26.597 17:42:19 STEP 2451 H=2451:00 +CPU= 42.861 17:42:46 STEP 2452 H=2452:00 +CPU= 26.318 17:43:17 STEP 2453 H=2453:00 +CPU= 30.480 17:44:04 STEP 2454 H=2454:00 +CPU= 45.627 17:44:34 STEP 2455 H=2455:00 +CPU= 28.781 17:45:06 STEP 2456 H=2456:00 +CPU= 31.715 17:45:51 STEP 2457 H=2457:00 +CPU= 44.201 17:46:18 STEP 2458 H=2458:00 +CPU= 26.341 17:46:44 STEP 2459 H=2459:00 +CPU= 25.609 17:47:35 STEP 2460 H=2460:00 +CPU= 49.681 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMGGhq0f+002460 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMSHhq0f+002460 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMUAhq0f+002460 17:48:01 STEP 2461 H=2461:00 +CPU= 25.961 17:48:29 STEP 2462 H=2462:00 +CPU= 26.789 17:49:14 STEP 2463 H=2463:00 +CPU= 44.917 17:49:44 STEP 2464 H=2464:00 +CPU= 29.173 17:50:12 STEP 2465 H=2465:00 +CPU= 27.663 17:50:58 STEP 2466 H=2466:00 +CPU= 44.388 17:51:26 STEP 2467 H=2467:00 +CPU= 27.343 17:51:52 STEP 2468 H=2468:00 +CPU= 26.255 17:52:41 STEP 2469 H=2469:00 +CPU= 47.307 17:53:10 STEP 2470 H=2470:00 +CPU= 28.974 17:53:37 STEP 2471 H=2471:00 +CPU= 26.072 17:54:29 STEP 2472 H=2472:00 +CPU= 51.078 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMGGhq0f+002472 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMSHhq0f+002472 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMUAhq0f+002472 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182542/ICMGGhq0f+002448 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182542/ICMSHhq0f+002448 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182542/ICMUAhq0f+002448 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182542/ICMGGhq0f+002460 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182542/ICMSHhq0f+002460 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182542/ICMUAhq0f+002460 Zipping up the intermediate file: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0898_1996050100_123_965_12182542_0_r28250566_102.zip Uploading the intermediate file: upload_file_102.zip 17:54:55 STEP 2473 H=2473:00 +CPU= 25.266 17:55:21 STEP 2474 H=2474:00 +CPU= 26.264 17:56:05 STEP 2475 H=2475:00 +CPU= 42.561 17:56:34 STEP 2476 H=2476:00 +CPU= 28.260 17:57:01 STEP 2477 H=2477:00 +CPU= 27.205 17:57:47 STEP 2478 H=2478:00 +CPU= 45.005 17:58:13 STEP 2479 H=2479:00 +CPU= 25.055 17:58:40 STEP 2480 H=2480:00 +CPU= 26.703 17:59:22 STEP 2481 H=2481:00 +CPU= 41.246 17:59:52 STEP 2482 H=2482:00 +CPU= 28.956 18:00:18 STEP 2483 H=2483:00 +CPU= 25.502 18:01:04 STEP 2484 H=2484:00 +CPU= 45.216 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMGGhq0f+002484 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMSHhq0f+002484 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMUAhq0f+002484 18:01:30 STEP 2485 H=2485:00 +CPU= 25.795 18:01:58 STEP 2486 H=2486:00 +CPU= 26.993 18:02:43 STEP 2487 H=2487:00 +CPU= 44.440 18:03:12 STEP 2488 H=2488:00 +CPU= 27.833 18:03:39 STEP 2489 H=2489:00 +CPU= 26.843 18:04:24 STEP 2490 H=2490:00 +CPU= 43.792 18:04:52 STEP 2491 H=2491:00 +CPU= 27.060 18:05:19 STEP 2492 H=2492:00 +CPU= 26.894 18:06:01 STEP 2493 H=2493:00 +CPU= 41.606 18:06:30 STEP 2494 H=2494:00 +CPU= 28.387 18:06:59 STEP 2495 H=2495:00 +CPU= 28.015 18:07:51 STEP 2496 H=2496:00 +CPU= 50.655 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMGGhq0f+002496 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMSHhq0f+002496 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMUAhq0f+002496 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182542/ICMGGhq0f+002472 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182542/ICMSHhq0f+002472 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182542/ICMUAhq0f+002472 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182542/ICMGGhq0f+002484 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182542/ICMSHhq0f+002484 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182542/ICMUAhq0f+002484 Zipping up the intermediate file: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0898_1996050100_123_965_12182542_0_r28250566_103.zip Uploading the intermediate file: upload_file_103.zip 18:08:18 STEP 2497 H=2497:00 +CPU= 26.734 18:08:46 STEP 2498 H=2498:00 +CPU= 27.643 18:09:32 STEP 2499 H=2499:00 +CPU= 44.649 18:10:00 STEP 2500 H=2500:00 +CPU= 27.398 18:10:29 STEP 2501 H=2501:00 +CPU= 28.272 18:11:13 STEP 2502 H=2502:00 +CPU= 43.792 Suspend request received from the BOINC client, suspending the child process [EC_DRHOOK:victor-Precision-7920-Tower:1:1:19431:19431] [20230124:183539:1674585339:37610.117] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1538] Received signal#15 (SIGTERM) :: 4749MB (heap), 4411MB (maxrss), 0MB (maxstack), 0 (paging), nsigs = 1 [EC_DRHOOK:victor-Precision-7920-Tower:1:1:19431:19431] [20230124:183539:1674585339:37610.220] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1542] Also activating Harakiri-alarm (SIGALRM=14) to expire after 500s elapsed to prevent hangs, nsigs = 1 [EC_DRHOOK:victor-Precision-7920-Tower:1:1:19431:19431] [20230124:183539:1674585339:37610.220] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1544] Harakiri signal handler 'signal_harakiri' for signal#14 (SIGALRM) installed at 0x81f3c0 (old at (nil)) [EC_DRHOOK:victor-Precision-7920-Tower:1:1:19431:19431] [20230124:183539:1674585339:37610.220] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1617] Signal#15 was caused by unrecognized si_code [memaddr=0x1], nsigs = 1 [EC_DRHOOK:victor-Precision-7920-Tower:1:1:19431:19431] [20230124:183539:1674585339:37610.220] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1686] Starting DrHook backtrace for signal#15, nsigs = 1 [EC_DRHOOK:victor-Precision-7920-Tower:1:1:19431:19431] [20230124:183539:1674585339:37610.220] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3843] 4749 MB (maxheap), 4411 MB (maxrss), 0 MB (maxstack) [EC_DRHOOK:victor-Precision-7920-Tower:1:1:19431:19431] [20230124:183539:1674585339:37610.220] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : MASTER [EC_DRHOOK:victor-Precision-7920-Tower:1:1:19431:19431] [20230124:183539:1674585339:37610.250] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT0 [EC_DRHOOK:victor-Precision-7920-Tower:1:1:19431:19431] [20230124:183539:1674585339:37610.250] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT1 [EC_DRHOOK:victor-Precision-7920-Tower:1:1:19431:19431] [20230124:183539:1674585339:37610.250] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT2 [EC_DRHOOK:victor-Precision-7920-Tower:1:1:19431:19431] [20230124:183539:1674585339:37610.251] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT3 [EC_DRHOOK:victor-Precision-7920-Tower:1:1:19431:19431] [20230124:183539:1674585339:37610.251] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT4 [EC_DRHOOK:victor-Precision-7920-Tower:1:1:19431:19431] [20230124:183539:1674585339:37610.251] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : STEPO [EC_DRHOOK:victor-Precision-7920-Tower:1:1:19431:19431] [20230124:183539:1674585339:37610.251] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : SCAN2M [EC_DRHOOK:victor-Precision-7920-Tower:1:1:19431:19431] [20230124:183539:1674585339:37610.251] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : GP_MODEL_HEAP [EC_DRHOOK:victor-Precision-7920-Tower:1:1:19431:19431] [20230124:183539:1674585339:37610.251] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : GP_MODEL [EC_DRHOOK:victor-Precision-7920-Tower:1:1:19431:19431] [20230124:183539:1674585339:37610.251] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CALL_SL_HEAP [EC_DRHOOK:victor-Precision-7920-Tower:1:1:19431:19431] [20230124:183539:1674585339:37610.251] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CALL_SL [EC_DRHOOK:victor-Precision-7920-Tower:1:1:19431:19431] [20230124:183539:1674585339:37610.251] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : >OMP-CALL_SL 2 (1005) [EC_DRHOOK:victor-Precision-7920-Tower:1:1:19431:19431] [20230124:183539:1674585339:37610.251] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : LAPINEB [EC_DRHOOK:victor-Precision-7920-Tower:1:1:19431:19431] [20230124:183539:1674585339:37610.251] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : LARCINB [EC_DRHOOK:victor-Precision-7920-Tower:1:1:19431:19431] [20230124:183539:1674585339:37610.251] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : LAITLI [EC_DRHOOK:victor-Precision-7920-Tower:1:1:19431:19431] [20230124:183540:1674585340:37611.285] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1734] DrHook backtrace done for signal#15, nsigs = 1 [EC_DRHOOK:victor-Precision-7920-Tower:1:1:19431:19431] [20230124:183540:1674585340:37611.285] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1785] Calling previous signal handler at 0x1ce8bb0 for signal#15, nsigs = 1 forrtl: error (78): process killed (SIGTERM) Image PC Routine Line Source oifs_43r3_model.e 0000000001CE916B Unknown Unknown Unknown oifs_43r3_model.e 000000000081FFF1 Unknown Unknown Unknown oifs_43r3_model.e 0000000001DC9090 Unknown Unknown Unknown oifs_43r3_model.e 0000000000C1A930 laitli_ 151 laitli.F90 oifs_43r3_model.e 0000000000DC65C0 larcinb_ 860 larcinb.F90 oifs_43r3_model.e 0000000000D7EBA8 lapineb_ 451 lapineb.F90 oifs_43r3_model.e 0000000000D7B935 call_sl_ 363 call_sl.F90 oifs_43r3_model.e 0000000000D784FD call_sl_heap_ 94 call_sl_heap.F90 oifs_43r3_model.e 0000000000BF9958 gp_model_ 533 gp_model.F90 oifs_43r3_model.e 00000000012E9482 gp_model_heap_ 74 gp_model_heap.F90 oifs_43r3_model.e 0000000000BBC1CE scan2m_ 535 scan2m.F90 oifs_43r3_model.e 000000000057A664 stepo_ 327 stepo.F90 oifs_43r3_model.e 000000000055DEEC cnt4_ 1133 cnt4.F90 oifs_43r3_model.e 00000000005471C9 cnt3_ 267 cnt3.F90 oifs_43r3_model.e 0000000000546412 cnt2_ 88 cnt2.F90 oifs_43r3_model.e 0000000000545E28 cnt1_ 92 cnt1.F90 oifs_43r3_model.e 000000000040708D cnt0_ 146 cnt0.F90 oifs_43r3_model.e 000000000040220F MAIN__ 96 master.F90 oifs_43r3_model.e 00000000004021A2 Unknown Unknown Unknown oifs_43r3_model.e 0000000001DCA390 Unknown Unknown Unknown oifs_43r3_model.e 000000000040206E Unknown Unknown Unknown (argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu (argv1) start_date: 1996050100 (argv2) exptid: hq0f (argv3) unique_member_id: 0898 (argv4) batchid: 965 (argv5) wuid: 12182542 (argv6) fclen: 123 (argv7) app_name: oifs_43r3_ps (argv8) nthreads: 1 Working directory is: /data/boinc/boinc-client/slots/23 Project directory is: /data/boinc/boinc-client/projects/climateprediction.net/ app name: oifs_43r3_ps version: 1.05 Location of temp folder: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182542 ..mkdir for temp folder for results failed Copying: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /data/boinc/boinc-client/slots/23/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip Unzipping the app zip file: /data/boinc/boinc-client/slots/23/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip Copying the namelist files from: ../../projects/climateprediction.net/jf_cf5aa86019b3ab2298975036fca385e4 to: /data/boinc/boinc-client/slots/23/oifs_43r3_ps_0898_1996050100_123_965_12182542.zip Unzipping the namelist zip file: /data/boinc/boinc-client/slots/23/oifs_43r3_ps_0898_1996050100_123_965_12182542.zip ic_ancil_file: ic_ancil_12182542 ifsdata_file: ifsdata_12182542 climate_data_file: clim_data_12182542 horiz_resolution: 159 vert_resolution: 60 grid_type: l_2 upload_interval: 24 utstep: 3600 nfrres: restart dump frequency (steps) 24 Copying IC ancils from: ../../projects/climateprediction.net/jf_aea8575242bc70c86843eef8aa8a35ab to: /data/boinc/boinc-client/slots/23/ic_ancil_12182542.zip Unzipping the IC ancils zip file: /data/boinc/boinc-client/slots/23/ic_ancil_12182542.zip ..mkdir for ifsdata folder failed Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_0e800e9cd34dc568ee3f15fb977f3e1f to: /data/boinc/boinc-client/slots/23/ifsdata/ifsdata_12182542.zip Unzipping the ifsdata_zip file: /data/boinc/boinc-client/slots/23/ifsdata/ifsdata_12182542.zip ..mkdir for the climate data folder failed Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /data/boinc/boinc-client/slots/23/159l_2/clim_data_12182542.zip Unzipping the climate data zip file: /data/boinc/boinc-client/slots/23/159l_2/clim_data_12182542.zip Checking for progress XML file: /data/boinc/boinc-client/slots/23/progress_file_12182542.xml Opened progress file ok : /data/boinc/boinc-client/slots/23/progress_file_12182542.xml -- Model is restarting -- Adjusting last_iter, 2503, to previous model restart step. Creating progress file: /data/boinc/boinc-client/slots/23/progress_file_12182542.xml last_cpu_time: 87420 upload_file_number: 104 last_iter: 2495 last_upload: 8985600 model_completed: 0 total_length_of_simulation: 10627200 result_base_name: oifs_43r3_ps_0898_1996050100_123_965_12182542_0_r28250566 The child process has been launched with process id: 5762 Executing the command: /data/boinc/boinc-client/slots/23/oifs_43r3_model.exe [EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1 [EC_DRHOOK:victor-Precision-7920-Tower:1:1:5762:5762] [20230125:063936:1674628776:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0 [EC_DRHOOK:victor-Precision-7920-Tower:1:1:5762:5762] [20230125:063936:1674628776:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1 [EC_DRHOOK:victor-Precision-7920-Tower:1:1:5762:5762] [20230125:063936:1674628776:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0) [EC_DRHOOK:victor-Precision-7920-Tower:1:1:5762:5762] [20230125:063936:1674628776:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1 [EC_DRHOOK:victor-Precision-7920-Tower:1:1:5762:5762] [20230125:063936:1674628776:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000 [EC_DRHOOK:victor-Precision-7920-Tower:1:1:5762:5762] [20230125:063936:1674628776:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647) [EC_DRHOOK:victor-Precision-7920-Tower:1:1:5762:5762] [20230125:063936:1674628776:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16 [EC_DRHOOK:victor-Precision-7920-Tower:1:1:5762:5762] [20230125:063936:1674628776:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0 [EC_DRHOOK:victor-Precision-7920-Tower:1:1:5762:5762] [20230125:063936:1674628776:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500 [EC_DRHOOK:victor-Precision-7920-Tower:1:1:5762:5762] [20230125:063936:1674628776:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1 [EC_DRHOOK:victor-Precision-7920-Tower:1:1:5762:5762] [20230125:063936:1674628776:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1 [EC_DRHOOK:victor-Precision-7920-Tower:1:1:5762:5762] [20230125:063936:1674628776:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1 [EC_DRHOOK:victor-Precision-7920-Tower:1:1:5762:5762] [20230125:063936:1674628776:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1 [EC_DRHOOK:victor-Precision-7920-Tower:1:1:5762:5762] [20230125:063936:1674628776:0.001] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef> [EC_DRHOOK:victor-Precision-7920-Tower:1:1:5762:5762] [20230125:063936:1674628776:0.001] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef> [EC_DRHOOK:victor-Precision-7920-Tower:1:1:5762:5762] [20230125:063936:1674628776:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:victor-Precision-7920-Tower:1:1:5762:5762] [20230125:063936:1674628776:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil)) [EC_DRHOOK:victor-Precision-7920-Tower:1:1:5762:5762] [20230125:063936:1674628776:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:victor-Precision-7920-Tower:1:1:5762:5762] [20230125:063936:1674628776:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil)) [EC_DRHOOK:victor-Precision-7920-Tower:1:1:5762:5762] [20230125:063936:1674628776:0.001] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1 [EC_DRHOOK:victor-Precision-7920-Tower:1:1:5762:5762] [20230125:063936:1674628776:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:victor-Precision-7920-Tower:1:1:5762:5762] [20230125:063936:1674628776:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:victor-Precision-7920-Tower:1:1:5762:5762] [20230125:063936:1674628776:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil)) [EC_DRHOOK:victor-Precision-7920-Tower:1:1:5762:5762] [20230125:063936:1674628776:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:victor-Precision-7920-Tower:1:1:5762:5762] [20230125:063936:1674628776:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:victor-Precision-7920-Tower:1:1:5762:5762] [20230125:063936:1674628776:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:victor-Precision-7920-Tower:1:1:5762:5762] [20230125:063936:1674628776:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil)) [EC_DRHOOK:victor-Precision-7920-Tower:1:1:5762:5762] [20230125:063936:1674628776:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil)) [EC_DRHOOK:victor-Precision-7920-Tower:1:1:5762:5762] [20230125:063936:1674628776:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil)) [EC_DRHOOK:victor-Precision-7920-Tower:1:1:5762:5762] [20230125:063936:1674628776:0.001] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef> 06:40:56 STEP 2496 H=2496:00 +CPU= 32.562 06:41:14 STEP 2497 H=2497:00 +CPU= 17.905 06:41:33 STEP 2498 H=2498:00 +CPU= 18.400 06:42:01 STEP 2499 H=2499:00 +CPU= 28.270 06:42:20 STEP 2500 H=2500:00 +CPU= 18.540 06:42:38 STEP 2501 H=2501:00 +CPU= 18.845 06:43:09 STEP 2502 H=2502:00 +CPU= 30.232 06:43:27 STEP 2503 H=2503:00 +CPU= 18.051 06:43:46 STEP 2504 H=2504:00 +CPU= 18.733 Suspend request received from the BOINC client, suspending the child process [EC_DRHOOK:victor-Precision-7920-Tower:1:1:5762:5762] [20230125:064435:1674629075:298.949] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1538] Received signal#15 (SIGTERM) :: 4746MB (heap), 4403MB (maxrss), 0MB (maxstack), 0 (paging), nsigs = 1 [EC_DRHOOK:victor-Precision-7920-Tower:1:1:5762:5762] [20230125:064435:1674629075:298.957] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1542] Also activating Harakiri-alarm (SIGALRM=14) to expire after 500s elapsed to prevent hangs, nsigs = 1 double free or corruption (top) [EC_DRHOOK:victor-Precision-7920-Tower:1:1:5762:5762] [20230125:064435:1674629075:298.957] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1544] Harakiri signal handler 'signal_harakiri' for signal#14 (SIGALRM) installed at 0x81f3c0 (old at (nil)) [EC_DRHOOK:victor-Precision-7920-Tower:1:1:5762:5762] [20230125:064435:1674629075:298.957] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1617] Signal#15 was caused by unrecognized si_code [memaddr=0x1], nsigs = 1 [EC_DRHOOK:victor-Precision-7920-Tower:1:1:5762:5762] [20230125:064435:1674629075:298.957] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1686] Starting DrHook backtrace for signal#15, nsigs = 1 [EC_DRHOOK:victor-Precision-7920-Tower:1:1:5762:5762] [20230125:064435:1674629075:298.957] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3843] 4746 MB (maxheap), 4403 MB (maxrss), 0 MB (maxstack) [EC_DRHOOK:victor-Precision-7920-Tower:1:1:5762:5762] [20230125:064435:1674629075:298.957] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : MASTER [EC_DRHOOK:victor-Precision-7920-Tower:1:1:5762:5762] [20230125:064435:1674629075:298.957] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT0 [EC_DRHOOK:victor-Precision-7920-Tower:1:1:5762:5762] [20230125:064435:1674629075:298.957] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT1 [EC_DRHOOK:victor-Precision-7920-Tower:1:1:5762:5762] [20230125:064435:1674629075:298.957] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT2 [EC_DRHOOK:victor-Precision-7920-Tower:1:1:5762:5762] [20230125:064435:1674629075:298.957] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT3 [EC_DRHOOK:victor-Precision-7920-Tower:1:1:5762:5762] [20230125:064435:1674629075:298.957] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT4 [EC_DRHOOK:victor-Precision-7920-Tower:1:1:5762:5762] [20230125:064435:1674629075:298.957] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : STEPO [EC_DRHOOK:victor-Precision-7920-Tower:1:1:5762:5762] [20230125:064435:1674629075:298.957] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : SCAN2M [EC_DRHOOK:victor-Precision-7920-Tower:1:1:5762:5762] [20230125:064435:1674629075:298.957] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : GP_MODEL_HEAP [EC_DRHOOK:victor-Precision-7920-Tower:1:1:5762:5762] [20230125:064435:1674629075:298.957] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : GP_MODEL [EC_DRHOOK:victor-Precision-7920-Tower:1:1:5762:5762] [20230125:064435:1674629075:298.957] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CALL_SL_HEAP [EC_DRHOOK:victor-Precision-7920-Tower:1:1:5762:5762] [20230125:064435:1674629075:298.957] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CALL_SL [EC_DRHOOK:victor-Precision-7920-Tower:1:1:5762:5762] [20230125:064435:1674629075:298.957] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : >OMP-CALL_SL 1 (1004) [EC_DRHOOK:victor-Precision-7920-Tower:1:1:5762:5762] [20230125:064435:1674629075:298.957] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : LAPINEA [EC_DRHOOK:victor-Precision-7920-Tower:1:1:5762:5762] [20230125:064435:1674629075:298.957] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : LARCINA [EC_DRHOOK:victor-Precision-7920-Tower:1:1:5762:5762] [20230125:064435:1674629075:298.957] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : LASCAW [EC_DRHOOK:victor-Precision-7920-Tower:1:1:5762:5762] [20230125:064436:1674629076:299.958] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1734] DrHook backtrace done for signal#15, nsigs = 1 [EC_DRHOOK:victor-Precision-7920-Tower:1:1:5762:5762] [20230125:064436:1674629076:299.958] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1785] Calling previous signal handler at 0x1ce8bb0 for signal#15, nsigs = 1 forrtl: error (78): process killed (SIGTERM) Image PC Routine Line Source oifs_43r3_model.e 0000000001CE916B Unknown Unknown Unknown oifs_43r3_model.e 000000000081FFF1 Unknown Unknown Unknown oifs_43r3_model.e 0000000001DC9090 Unknown Unknown Unknown oifs_43r3_model.e 0000000000DF2A7D lascaw_ 634 lascaw.F90 oifs_43r3_model.e 0000000000DED525 larcina_ 319 larcina.F90 oifs_43r3_model.e 0000000000DE60CD lapinea_ 325 lapinea.F90 oifs_43r3_model.e 0000000000D7A4EC call_sl_ 300 call_sl.F90 oifs_43r3_model.e 0000000000D784FD call_sl_heap_ 94 call_sl_heap.F90 oifs_43r3_model.e 0000000000BF9958 gp_model_ 533 gp_model.F90 oifs_43r3_model.e 00000000012E9482 gp_model_heap_ 74 gp_model_heap.F90 oifs_43r3_model.e 0000000000BBC1CE scan2m_ 535 scan2m.F90 oifs_43r3_model.e 000000000057A664 stepo_ 327 stepo.F90 oifs_43r3_model.e 000000000055DEEC cnt4_ 1133 cnt4.F90 oifs_43r3_model.e 00000000005471C9 cnt3_ 267 cnt3.F90 oifs_43r3_model.e 0000000000546412 cnt2_ 88 cnt2.F90 oifs_43r3_model.e 0000000000545E28 cnt1_ 92 cnt1.F90 oifs_43r3_model.e 000000000040708D cnt0_ 146 cnt0.F90 oifs_43r3_model.e 000000000040220F MAIN__ 96 master.F90 oifs_43r3_model.e 00000000004021A2 Unknown Unknown Unknown oifs_43r3_model.e 0000000001DCA390 Unknown Unknown Unknown oifs_43r3_model.e 000000000040206E Unknown Unknown Unknown (argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu (argv1) start_date: 1996050100 (argv2) exptid: hq0f (argv3) unique_member_id: 0898 (argv4) batchid: 965 (argv5) wuid: 12182542 (argv6) fclen: 123 (argv7) app_name: oifs_43r3_ps (argv8) nthreads: 1 Working directory is: /data/boinc/boinc-client/slots/23 Project directory is: /data/boinc/boinc-client/projects/climateprediction.net/ app name: oifs_43r3_ps version: 1.05 Location of temp folder: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182542 ..mkdir for temp folder for results failed Copying: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /data/boinc/boinc-client/slots/23/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip Unzipping the app zip file: /data/boinc/boinc-client/slots/23/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip Copying the namelist files from: ../../projects/climateprediction.net/jf_cf5aa86019b3ab2298975036fca385e4 to: /data/boinc/boinc-client/slots/23/oifs_43r3_ps_0898_1996050100_123_965_12182542.zip Unzipping the namelist zip file: /data/boinc/boinc-client/slots/23/oifs_43r3_ps_0898_1996050100_123_965_12182542.zip ic_ancil_file: ic_ancil_12182542 ifsdata_file: ifsdata_12182542 climate_data_file: clim_data_12182542 horiz_resolution: 159 vert_resolution: 60 grid_type: l_2 upload_interval: 24 utstep: 3600 nfrres: restart dump frequency (steps) 24 Copying IC ancils from: ../../projects/climateprediction.net/jf_aea8575242bc70c86843eef8aa8a35ab to: /data/boinc/boinc-client/slots/23/ic_ancil_12182542.zip Unzipping the IC ancils zip file: /data/boinc/boinc-client/slots/23/ic_ancil_12182542.zip ..mkdir for ifsdata folder failed Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_0e800e9cd34dc568ee3f15fb977f3e1f to: /data/boinc/boinc-client/slots/23/ifsdata/ifsdata_12182542.zip Unzipping the ifsdata_zip file: /data/boinc/boinc-client/slots/23/ifsdata/ifsdata_12182542.zip ..mkdir for the climate data folder failed Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /data/boinc/boinc-client/slots/23/159l_2/clim_data_12182542.zip Unzipping the climate data zip file: /data/boinc/boinc-client/slots/23/159l_2/clim_data_12182542.zip Checking for progress XML file: /data/boinc/boinc-client/slots/23/progress_file_12182542.xml Opened progress file ok : /data/boinc/boinc-client/slots/23/progress_file_12182542.xml -- Model is restarting -- Adjusting last_iter, 2505, to previous model restart step. Creating progress file: /data/boinc/boinc-client/slots/23/progress_file_12182542.xml last_cpu_time: 87636 upload_file_number: 104 last_iter: 2495 last_upload: 8985600 model_completed: 0 total_length_of_simulation: 10627200 result_base_name: oifs_43r3_ps_0898_1996050100_123_965_12182542_0_r28250566 The child process has been launched with process id: 21138 Executing the command: /data/boinc/boinc-client/slots/23/oifs_43r3_model.exe [EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1 [EC_DRHOOK:victor-Precision-7920-Tower:1:1:21138:21138] [20230125:070514:1674630314:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0 [EC_DRHOOK:victor-Precision-7920-Tower:1:1:21138:21138] [20230125:070514:1674630314:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1 [EC_DRHOOK:victor-Precision-7920-Tower:1:1:21138:21138] [20230125:070514:1674630314:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0) [EC_DRHOOK:victor-Precision-7920-Tower:1:1:21138:21138] [20230125:070514:1674630314:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1 [EC_DRHOOK:victor-Precision-7920-Tower:1:1:21138:21138] [20230125:070514:1674630314:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000 [EC_DRHOOK:victor-Precision-7920-Tower:1:1:21138:21138] [20230125:070514:1674630314:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647) [EC_DRHOOK:victor-Precision-7920-Tower:1:1:21138:21138] [20230125:070514:1674630314:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16 [EC_DRHOOK:victor-Precision-7920-Tower:1:1:21138:21138] [20230125:070514:1674630314:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0 [EC_DRHOOK:victor-Precision-7920-Tower:1:1:21138:21138] [20230125:070514:1674630314:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500 [EC_DRHOOK:victor-Precision-7920-Tower:1:1:21138:21138] [20230125:070514:1674630314:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1 [EC_DRHOOK:victor-Precision-7920-Tower:1:1:21138:21138] [20230125:070514:1674630314:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1 [EC_DRHOOK:victor-Precision-7920-Tower:1:1:21138:21138] [20230125:070514:1674630314:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1 [EC_DRHOOK:victor-Precision-7920-Tower:1:1:21138:21138] [20230125:070514:1674630314:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1 [EC_DRHOOK:victor-Precision-7920-Tower:1:1:21138:21138] [20230125:070514:1674630314:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef> [EC_DRHOOK:victor-Precision-7920-Tower:1:1:21138:21138] [20230125:070514:1674630314:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef> [EC_DRHOOK:victor-Precision-7920-Tower:1:1:21138:21138] [20230125:070514:1674630314:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:victor-Precision-7920-Tower:1:1:21138:21138] [20230125:070514:1674630314:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil)) [EC_DRHOOK:victor-Precision-7920-Tower:1:1:21138:21138] [20230125:070514:1674630314:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:victor-Precision-7920-Tower:1:1:21138:21138] [20230125:070514:1674630314:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil)) [EC_DRHOOK:victor-Precision-7920-Tower:1:1:21138:21138] [20230125:070514:1674630314:0.001] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1 [EC_DRHOOK:victor-Precision-7920-Tower:1:1:21138:21138] [20230125:070514:1674630314:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:victor-Precision-7920-Tower:1:1:21138:21138] [20230125:070514:1674630314:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:victor-Precision-7920-Tower:1:1:21138:21138] [20230125:070514:1674630314:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil)) [EC_DRHOOK:victor-Precision-7920-Tower:1:1:21138:21138] [20230125:070514:1674630314:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:victor-Precision-7920-Tower:1:1:21138:21138] [20230125:070514:1674630314:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:victor-Precision-7920-Tower:1:1:21138:21138] [20230125:070514:1674630314:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:victor-Precision-7920-Tower:1:1:21138:21138] [20230125:070514:1674630314:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil)) [EC_DRHOOK:victor-Precision-7920-Tower:1:1:21138:21138] [20230125:070514:1674630314:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil)) [EC_DRHOOK:victor-Precision-7920-Tower:1:1:21138:21138] [20230125:070514:1674630314:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil)) [EC_DRHOOK:victor-Precision-7920-Tower:1:1:21138:21138] [20230125:070514:1674630314:0.001] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef> 07:06:54 STEP 2496 H=2496:00 +CPU= 54.093 07:07:25 STEP 2497 H=2497:00 +CPU= 30.696 Suspend request received from the BOINC client, suspending the child process Resuming the child process 07:10:27 STEP 2498 H=2498:00 +CPU= 29.788 07:11:15 STEP 2499 H=2499:00 +CPU= 47.803 07:11:46 STEP 2500 H=2500:00 +CPU= 30.517 07:12:16 STEP 2501 H=2501:00 +CPU= 29.886 07:13:05 STEP 2502 H=2502:00 +CPU= 49.184 07:13:35 STEP 2503 H=2503:00 +CPU= 29.378 07:14:06 STEP 2504 H=2504:00 +CPU= 30.544 07:14:55 STEP 2505 H=2505:00 +CPU= 48.822 07:15:27 STEP 2506 H=2506:00 +CPU= 32.164 07:15:58 STEP 2507 H=2507:00 +CPU= 30.625 07:16:52 STEP 2508 H=2508:00 +CPU= 53.553 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMGGhq0f+002508 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMSHhq0f+002508 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMUAhq0f+002508 07:17:22 STEP 2509 H=2509:00 +CPU= 29.864 07:17:53 STEP 2510 H=2510:00 +CPU= 30.667 07:18:42 STEP 2511 H=2511:00 +CPU= 49.101 07:19:14 STEP 2512 H=2512:00 +CPU= 31.225 07:19:44 STEP 2513 H=2513:00 +CPU= 29.602 07:20:34 STEP 2514 H=2514:00 +CPU= 50.282 Suspend request received from the BOINC client, suspending the child process Resuming the child process 07:26:41 STEP 2515 H=2515:00 +CPU= 29.789 07:27:12 STEP 2516 H=2516:00 +CPU= 29.956 07:28:00 STEP 2517 H=2517:00 +CPU= 47.613 07:28:31 STEP 2518 H=2518:00 +CPU= 31.095 07:29:02 STEP 2519 H=2519:00 +CPU= 31.165 07:30:02 STEP 2520 H=2520:00 +CPU= 59.147 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMGGhq0f+002520 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMSHhq0f+002520 Moving to projects directory: /data/boinc/boinc-client/slots/23/ICMUAhq0f+002520 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182542/ICMGGhq0f+002496 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182542/ICMSHhq0f+002496 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182542/ICMUAhq0f+002496 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182542/ICMGGhq0f+002508 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182542/ICMSHhq0f+002508 Adding to the zip: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12182542/ICMUAhq0f+002508 Zipping up the intermediate file: /data/boinc/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0898_1996050100_123_965_12182542_0_r28250566_104.zip Uploading the intermediate file: upload_file_104.zip 07:30:33 STEP 2521 H=2521:00 +CPU= 31.170 Abort request received from BOINC client, ending the child process ..Failed, model did not complete successfully </stderr_txt> ]]> |
Latest Trickles Received | ||||||
---|---|---|---|---|---|---|
Time Sent (UTC) | Host ID | Result ID | Result Name | Timestep | CPU Time (sec) | Average (sec/TS) |
09 May 2023 16:39:38 | 1478188 | 22267646 | oifs_43r3_ps_0898_1996050100_123_965_12182542_0 | 8,380,800 | 82,297 | 0.0098 |
09 May 2023 16:38:56 | 1478188 | 22267646 | oifs_43r3_ps_0898_1996050100_123_965_12182542_0 | 7,171,200 | 70,276 | 0.0098 |
09 May 2023 16:37:46 | 1478188 | 22267646 | oifs_43r3_ps_0898_1996050100_123_965_12182542_0 | 6,307,200 | 62,939 | 0.0100 |
09 May 2023 16:36:40 | 1478188 | 22267646 | oifs_43r3_ps_0898_1996050100_123_965_12182542_0 | 5,443,200 | 55,116 | 0.0101 |
09 May 2023 16:36:29 | 1478188 | 22267646 | oifs_43r3_ps_0898_1996050100_123_965_12182542_0 | 4,579,200 | 46,612 | 0.0102 |
09 May 2023 16:31:47 | 1478188 | 22267646 | oifs_43r3_ps_0898_1996050100_123_965_12182542_0 | 3,715,200 | 37,588 | 0.0101 |
09 May 2023 16:31:35 | 1478188 | 22267646 | oifs_43r3_ps_0898_1996050100_123_965_12182542_0 | 2,851,200 | 29,224 | 0.0102 |
09 May 2023 16:31:15 | 1478188 | 22267646 | oifs_43r3_ps_0898_1996050100_123_965_12182542_0 | 1,987,200 | 20,890 | 0.0105 |
©2024 cpdn.org