climateprediction.net home page
Task 22277797

Task 22277797

Name oifs_43r3_ps_0844_1982050100_123_951_12168488_1
Workunit 12168488
Created 23 Dec 2022, 13:10:14 UTC
Sent 15 Jan 2023, 9:56:45 UTC
Report deadline 14 Feb 2023, 9:56:45 UTC
Received 15 Jan 2023, 22:31:05 UTC
Server state Over
Outcome Success
Client state Done
Exit status 0 (0x00000000)
Computer ID 1537625
Run time 11 hours 13 min 50 sec
CPU time 11 hours 10 min 18 sec
Validate state Valid
Credit 14,873.04
Device peak FLOPS 1.00 GFLOPS
Application version OpenIFS 43r3 Perturbed Surface v1.05
x86_64-pc-linux-gnu
Peak working set size 4,798.24 MB
Peak swap size 4,974.26 MB
Peak disk usage 3,010.42 MB
Stderr
<core_client_version>7.18.1</core_client_version>
<![CDATA[
<stderr_txt>
8/ICMSHhq0f+002268
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMUAhq0f+002268
Zipping up the intermediate file: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_0844_1982050100_123_951_12168488_1_r121377581_94.zip
Uploading the intermediate file: upload_file_94.zip
  06:15:11 STEP 2281 H=2281:00 +CPU= 10.491
  06:15:22 STEP 2282 H=2282:00 +CPU= 11.073
  06:15:39 STEP 2283 H=2283:00 +CPU= 17.194
  06:15:50 STEP 2284 H=2284:00 +CPU= 10.892
  06:16:01 STEP 2285 H=2285:00 +CPU= 10.505
  06:16:18 STEP 2286 H=2286:00 +CPU= 17.444
  06:16:29 STEP 2287 H=2287:00 +CPU= 10.535
  06:16:40 STEP 2288 H=2288:00 +CPU= 10.994
  06:16:57 STEP 2289 H=2289:00 +CPU= 17.044
  06:17:08 STEP 2290 H=2290:00 +CPU= 10.893
  06:17:19 STEP 2291 H=2291:00 +CPU= 10.758
  06:17:38 STEP 2292 H=2292:00 +CPU= 18.910
Moving to projects directory: /home/gpuser/slots/0/ICMGGhq0f+002292
Moving to projects directory: /home/gpuser/slots/0/ICMSHhq0f+002292
Moving to projects directory: /home/gpuser/slots/0/ICMUAhq0f+002292
  06:17:48 STEP 2293 H=2293:00 +CPU= 10.578
  06:17:59 STEP 2294 H=2294:00 +CPU= 10.818
  06:18:16 STEP 2295 H=2295:00 +CPU= 17.184
  06:18:27 STEP 2296 H=2296:00 +CPU= 10.645
  06:18:37 STEP 2297 H=2297:00 +CPU= 10.500
  06:18:55 STEP 2298 H=2298:00 +CPU= 17.309
  06:19:05 STEP 2299 H=2299:00 +CPU= 10.472
  06:19:16 STEP 2300 H=2300:00 +CPU= 10.576
  06:19:33 STEP 2301 H=2301:00 +CPU= 17.452
  06:19:44 STEP 2302 H=2302:00 +CPU= 10.952
  06:19:55 STEP 2303 H=2303:00 +CPU= 10.740
  06:20:15 STEP 2304 H=2304:00 +CPU= 19.985
Moving to projects directory: /home/gpuser/slots/0/ICMGGhq0f+002304
Moving to projects directory: /home/gpuser/slots/0/ICMSHhq0f+002304
Moving to projects directory: /home/gpuser/slots/0/ICMUAhq0f+002304
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMGGhq0f+002280
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMSHhq0f+002280
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMUAhq0f+002280
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMGGhq0f+002292
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMSHhq0f+002292
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMUAhq0f+002292
Zipping up the intermediate file: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_0844_1982050100_123_951_12168488_1_r121377581_95.zip
Uploading the intermediate file: upload_file_95.zip
  06:20:25 STEP 2305 H=2305:00 +CPU= 10.570
  06:20:36 STEP 2306 H=2306:00 +CPU= 10.757
  06:20:53 STEP 2307 H=2307:00 +CPU= 17.038
  06:21:04 STEP 2308 H=2308:00 +CPU= 11.082
  06:21:15 STEP 2309 H=2309:00 +CPU= 10.478
  06:21:32 STEP 2310 H=2310:00 +CPU= 17.470
  06:21:43 STEP 2311 H=2311:00 +CPU= 10.459
  06:21:54 STEP 2312 H=2312:00 +CPU= 11.101
  06:22:11 STEP 2313 H=2313:00 +CPU= 17.137
  06:22:22 STEP 2314 H=2314:00 +CPU= 10.907
  06:22:33 STEP 2315 H=2315:00 +CPU= 10.591
  06:22:51 STEP 2316 H=2316:00 +CPU= 18.833
Moving to projects directory: /home/gpuser/slots/0/ICMGGhq0f+002316
Moving to projects directory: /home/gpuser/slots/0/ICMSHhq0f+002316
Moving to projects directory: /home/gpuser/slots/0/ICMUAhq0f+002316
  06:23:02 STEP 2317 H=2317:00 +CPU= 10.574
  06:23:13 STEP 2318 H=2318:00 +CPU= 10.835
  06:23:30 STEP 2319 H=2319:00 +CPU= 16.986
  06:23:41 STEP 2320 H=2320:00 +CPU= 10.919
  06:23:51 STEP 2321 H=2321:00 +CPU= 10.443
  06:24:08 STEP 2322 H=2322:00 +CPU= 17.244
  06:24:19 STEP 2323 H=2323:00 +CPU= 10.371
  06:24:30 STEP 2324 H=2324:00 +CPU= 10.803
  06:24:47 STEP 2325 H=2325:00 +CPU= 17.130
  06:24:58 STEP 2326 H=2326:00 +CPU= 10.963
  06:25:09 STEP 2327 H=2327:00 +CPU= 11.157
  06:25:29 STEP 2328 H=2328:00 +CPU= 20.126
Moving to projects directory: /home/gpuser/slots/0/ICMGGhq0f+002328
Moving to projects directory: /home/gpuser/slots/0/ICMSHhq0f+002328
Moving to projects directory: /home/gpuser/slots/0/ICMUAhq0f+002328
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMGGhq0f+002304
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMSHhq0f+002304
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMUAhq0f+002304
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMGGhq0f+002316
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMSHhq0f+002316
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMUAhq0f+002316
Zipping up the intermediate file: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_0844_1982050100_123_951_12168488_1_r121377581_96.zip
Uploading the intermediate file: upload_file_96.zip
  06:25:40 STEP 2329 H=2329:00 +CPU= 10.716
  06:25:51 STEP 2330 H=2330:00 +CPU= 10.973
  06:26:08 STEP 2331 H=2331:00 +CPU= 17.276
  06:26:19 STEP 2332 H=2332:00 +CPU= 10.669
  06:26:29 STEP 2333 H=2333:00 +CPU= 10.573
  06:26:47 STEP 2334 H=2334:00 +CPU= 17.411
  06:26:57 STEP 2335 H=2335:00 +CPU= 10.413
  06:27:08 STEP 2336 H=2336:00 +CPU= 10.680
  06:27:25 STEP 2337 H=2337:00 +CPU= 17.154
  06:27:36 STEP 2338 H=2338:00 +CPU= 10.983
  06:27:47 STEP 2339 H=2339:00 +CPU= 10.645
  06:28:05 STEP 2340 H=2340:00 +CPU= 18.820
Moving to projects directory: /home/gpuser/slots/0/ICMGGhq0f+002340
Moving to projects directory: /home/gpuser/slots/0/ICMSHhq0f+002340
Moving to projects directory: /home/gpuser/slots/0/ICMUAhq0f+002340
  06:28:16 STEP 2341 H=2341:00 +CPU= 10.635
  06:28:27 STEP 2342 H=2342:00 +CPU= 10.969
  06:28:44 STEP 2343 H=2343:00 +CPU= 17.270
  06:28:56 STEP 2344 H=2344:00 +CPU= 11.081
  06:29:06 STEP 2345 H=2345:00 +CPU= 10.462
  06:29:23 STEP 2346 H=2346:00 +CPU= 17.352
  06:29:34 STEP 2347 H=2347:00 +CPU= 10.366
  06:29:45 STEP 2348 H=2348:00 +CPU= 10.842
  06:30:02 STEP 2349 H=2349:00 +CPU= 17.064
  06:30:13 STEP 2350 H=2350:00 +CPU= 10.918
  06:30:24 STEP 2351 H=2351:00 +CPU= 11.011
  06:30:44 STEP 2352 H=2352:00 +CPU= 19.945
Moving to projects directory: /home/gpuser/slots/0/ICMGGhq0f+002352
Moving to projects directory: /home/gpuser/slots/0/ICMSHhq0f+002352
Moving to projects directory: /home/gpuser/slots/0/ICMUAhq0f+002352
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMGGhq0f+002328
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMSHhq0f+002328
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMUAhq0f+002328
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMGGhq0f+002340
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMSHhq0f+002340
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMUAhq0f+002340
Zipping up the intermediate file: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_0844_1982050100_123_951_12168488_1_r121377581_97.zip
Uploading the intermediate file: upload_file_97.zip
  06:30:54 STEP 2353 H=2353:00 +CPU= 10.619
  06:31:05 STEP 2354 H=2354:00 +CPU= 10.941
  06:31:22 STEP 2355 H=2355:00 +CPU= 17.121
  06:31:33 STEP 2356 H=2356:00 +CPU= 10.915
  06:31:44 STEP 2357 H=2357:00 +CPU= 10.414
  06:32:01 STEP 2358 H=2358:00 +CPU= 17.410
  06:32:12 STEP 2359 H=2359:00 +CPU= 10.596
  06:32:22 STEP 2360 H=2360:00 +CPU= 10.835
  06:32:40 STEP 2361 H=2361:00 +CPU= 17.110
  06:32:50 STEP 2362 H=2362:00 +CPU= 10.662
  06:33:01 STEP 2363 H=2363:00 +CPU= 11.052
  06:33:21 STEP 2364 H=2364:00 +CPU= 19.354
Moving to projects directory: /home/gpuser/slots/0/ICMGGhq0f+002364
Moving to projects directory: /home/gpuser/slots/0/ICMSHhq0f+002364
Moving to projects directory: /home/gpuser/slots/0/ICMUAhq0f+002364
  06:33:31 STEP 2365 H=2365:00 +CPU= 10.817
  06:33:43 STEP 2366 H=2366:00 +CPU= 11.229
  06:34:00 STEP 2367 H=2367:00 +CPU= 17.338
  06:34:11 STEP 2368 H=2368:00 +CPU= 11.040
  06:34:22 STEP 2369 H=2369:00 +CPU= 10.859
  06:34:40 STEP 2370 H=2370:00 +CPU= 18.190
  06:34:51 STEP 2371 H=2371:00 +CPU= 11.034
  06:35:03 STEP 2372 H=2372:00 +CPU= 11.267
  06:35:20 STEP 2373 H=2373:00 +CPU= 17.504
  06:35:31 STEP 2374 H=2374:00 +CPU= 11.052
  06:35:43 STEP 2375 H=2375:00 +CPU= 11.515
  06:36:03 STEP 2376 H=2376:00 +CPU= 20.341
Moving to projects directory: /home/gpuser/slots/0/ICMGGhq0f+002376
Moving to projects directory: /home/gpuser/slots/0/ICMSHhq0f+002376
Moving to projects directory: /home/gpuser/slots/0/ICMUAhq0f+002376
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMGGhq0f+002352
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMSHhq0f+002352
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMUAhq0f+002352
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMGGhq0f+002364
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMSHhq0f+002364
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMUAhq0f+002364
Zipping up the intermediate file: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_0844_1982050100_123_951_12168488_1_r121377581_98.zip
Uploading the intermediate file: upload_file_98.zip
  06:36:14 STEP 2377 H=2377:00 +CPU= 10.804
  06:36:25 STEP 2378 H=2378:00 +CPU= 11.064
  06:36:42 STEP 2379 H=2379:00 +CPU= 17.456
  06:36:53 STEP 2380 H=2380:00 +CPU= 10.997
  06:37:04 STEP 2381 H=2381:00 +CPU= 10.681
  06:37:22 STEP 2382 H=2382:00 +CPU= 17.631
  06:37:32 STEP 2383 H=2383:00 +CPU= 10.728
  06:37:44 STEP 2384 H=2384:00 +CPU= 11.155
  06:38:01 STEP 2385 H=2385:00 +CPU= 17.483
  06:38:10 STEP 2386 H=2386:00 +CPU=  8.721
  06:38:18 STEP 2387 H=2387:00 +CPU=  8.332
  06:38:33 STEP 2388 H=2388:00 +CPU= 14.996
Moving to projects directory: /home/gpuser/slots/0/ICMGGhq0f+002388
Moving to projects directory: /home/gpuser/slots/0/ICMSHhq0f+002388
Moving to projects directory: /home/gpuser/slots/0/ICMUAhq0f+002388
  06:38:42 STEP 2389 H=2389:00 +CPU=  8.518
  06:38:50 STEP 2390 H=2390:00 +CPU=  8.525
  06:39:04 STEP 2391 H=2391:00 +CPU= 13.415
  06:39:12 STEP 2392 H=2392:00 +CPU=  8.688
  06:39:21 STEP 2393 H=2393:00 +CPU=  8.303
  06:39:34 STEP 2394 H=2394:00 +CPU= 13.664
  06:39:43 STEP 2395 H=2395:00 +CPU=  8.386
  06:39:51 STEP 2396 H=2396:00 +CPU=  8.477
  06:40:05 STEP 2397 H=2397:00 +CPU= 13.480
  06:40:13 STEP 2398 H=2398:00 +CPU=  8.573
  06:40:22 STEP 2399 H=2399:00 +CPU=  8.465
  06:40:37 STEP 2400 H=2400:00 +CPU= 15.708
Moving to projects directory: /home/gpuser/slots/0/ICMGGhq0f+002400
Moving to projects directory: /home/gpuser/slots/0/ICMSHhq0f+002400
Moving to projects directory: /home/gpuser/slots/0/ICMUAhq0f+002400
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMGGhq0f+002376
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMSHhq0f+002376
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMUAhq0f+002376
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMGGhq0f+002388
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMSHhq0f+002388
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMUAhq0f+002388
Zipping up the intermediate file: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_0844_1982050100_123_951_12168488_1_r121377581_99.zip
Uploading the intermediate file: upload_file_99.zip
  06:40:46 STEP 2401 H=2401:00 +CPU=  8.312
  06:40:54 STEP 2402 H=2402:00 +CPU=  8.385
Uploading trickle at timestep: 8640000
  06:41:07 STEP 2403 H=2403:00 +CPU= 13.304
  06:41:16 STEP 2404 H=2404:00 +CPU=  8.542
  06:41:24 STEP 2405 H=2405:00 +CPU=  8.176
  06:41:38 STEP 2406 H=2406:00 +CPU= 13.555
  06:41:46 STEP 2407 H=2407:00 +CPU=  8.352
  06:41:55 STEP 2408 H=2408:00 +CPU=  8.757
  06:42:08 STEP 2409 H=2409:00 +CPU= 13.522
  06:42:17 STEP 2410 H=2410:00 +CPU=  8.804
  06:42:26 STEP 2411 H=2411:00 +CPU=  8.455
  06:42:41 STEP 2412 H=2412:00 +CPU= 15.123
Moving to projects directory: /home/gpuser/slots/0/ICMGGhq0f+002412
Moving to projects directory: /home/gpuser/slots/0/ICMSHhq0f+002412
Moving to projects directory: /home/gpuser/slots/0/ICMUAhq0f+002412
  06:42:49 STEP 2413 H=2413:00 +CPU=  8.557
  06:42:58 STEP 2414 H=2414:00 +CPU=  8.569
  06:43:11 STEP 2415 H=2415:00 +CPU= 13.661
  06:43:20 STEP 2416 H=2416:00 +CPU=  8.832
  06:43:29 STEP 2417 H=2417:00 +CPU=  8.460
  06:43:43 STEP 2418 H=2418:00 +CPU= 14.109
  06:43:52 STEP 2419 H=2419:00 +CPU=  9.523
  06:44:02 STEP 2420 H=2420:00 +CPU=  9.839
  06:44:16 STEP 2421 H=2421:00 +CPU= 13.798
  06:44:25 STEP 2422 H=2422:00 +CPU=  8.718
  06:44:34 STEP 2423 H=2423:00 +CPU=  8.784
  06:44:50 STEP 2424 H=2424:00 +CPU= 16.358
Moving to projects directory: /home/gpuser/slots/0/ICMGGhq0f+002424
Moving to projects directory: /home/gpuser/slots/0/ICMSHhq0f+002424
Moving to projects directory: /home/gpuser/slots/0/ICMUAhq0f+002424
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMGGhq0f+002400
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMSHhq0f+002400
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMUAhq0f+002400
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMGGhq0f+002412
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMSHhq0f+002412
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMUAhq0f+002412
Zipping up the intermediate file: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_0844_1982050100_123_951_12168488_1_r121377581_100.zip
Uploading the intermediate file: upload_file_100.zip
  06:44:58 STEP 2425 H=2425:00 +CPU=  8.505
  06:45:07 STEP 2426 H=2426:00 +CPU=  8.766
  06:45:21 STEP 2427 H=2427:00 +CPU= 13.639
  06:45:30 STEP 2428 H=2428:00 +CPU=  8.696
  06:45:38 STEP 2429 H=2429:00 +CPU=  8.387
  06:45:52 STEP 2430 H=2430:00 +CPU= 14.051
  06:46:00 STEP 2431 H=2431:00 +CPU=  8.537
  06:46:09 STEP 2432 H=2432:00 +CPU=  8.750
  06:46:23 STEP 2433 H=2433:00 +CPU= 13.928
  06:46:32 STEP 2434 H=2434:00 +CPU=  8.927
  06:46:41 STEP 2435 H=2435:00 +CPU=  8.541
  06:46:56 STEP 2436 H=2436:00 +CPU= 15.527
Moving to projects directory: /home/gpuser/slots/0/ICMGGhq0f+002436
Moving to projects directory: /home/gpuser/slots/0/ICMSHhq0f+002436
Moving to projects directory: /home/gpuser/slots/0/ICMUAhq0f+002436
  06:47:06 STEP 2437 H=2437:00 +CPU=  9.409
  06:47:15 STEP 2438 H=2438:00 +CPU=  9.014
  06:47:28 STEP 2439 H=2439:00 +CPU= 13.870
  06:47:37 STEP 2440 H=2440:00 +CPU=  8.928
  06:47:46 STEP 2441 H=2441:00 +CPU=  8.711
  06:48:00 STEP 2442 H=2442:00 +CPU= 14.227
  06:48:09 STEP 2443 H=2443:00 +CPU=  8.671
  06:48:18 STEP 2444 H=2444:00 +CPU=  8.799
  06:48:32 STEP 2445 H=2445:00 +CPU= 13.768
  06:48:40 STEP 2446 H=2446:00 +CPU=  8.859
  06:48:49 STEP 2447 H=2447:00 +CPU=  8.778
  06:49:06 STEP 2448 H=2448:00 +CPU= 17.218
  06:49:15 STEP 2449 H=2449:00 +CPU=  8.596
Moving to projects directory: /home/gpuser/slots/0/ICMGGhq0f+002448
Moving to projects directory: /home/gpuser/slots/0/ICMSHhq0f+002448
Moving to projects directory: /home/gpuser/slots/0/ICMUAhq0f+002448
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMGGhq0f+002424
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMSHhq0f+002424
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMUAhq0f+002424
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMGGhq0f+002436
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMSHhq0f+002436
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMUAhq0f+002436
Zipping up the intermediate file: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_0844_1982050100_123_951_12168488_1_r121377581_101.zip
Uploading the intermediate file: upload_file_101.zip
  06:49:24 STEP 2450 H=2450:00 +CPU=  8.765
  06:49:38 STEP 2451 H=2451:00 +CPU= 13.865
  06:49:47 STEP 2452 H=2452:00 +CPU=  8.976
  06:49:55 STEP 2453 H=2453:00 +CPU=  8.669
  06:50:09 STEP 2454 H=2454:00 +CPU= 14.204
  06:50:18 STEP 2455 H=2455:00 +CPU=  8.603
  06:50:27 STEP 2456 H=2456:00 +CPU=  8.786
  06:50:42 STEP 2457 H=2457:00 +CPU= 14.733
  06:50:50 STEP 2458 H=2458:00 +CPU=  8.814
  06:50:59 STEP 2459 H=2459:00 +CPU=  8.577
  06:51:14 STEP 2460 H=2460:00 +CPU= 15.413
Moving to projects directory: /home/gpuser/slots/0/ICMGGhq0f+002460
Moving to projects directory: /home/gpuser/slots/0/ICMSHhq0f+002460
Moving to projects directory: /home/gpuser/slots/0/ICMUAhq0f+002460
  06:51:23 STEP 2461 H=2461:00 +CPU=  8.558
  06:51:32 STEP 2462 H=2462:00 +CPU=  8.877
  06:51:46 STEP 2463 H=2463:00 +CPU= 13.984
  06:51:55 STEP 2464 H=2464:00 +CPU=  8.839
  06:52:03 STEP 2465 H=2465:00 +CPU=  8.607
  06:52:18 STEP 2466 H=2466:00 +CPU= 15.089
  06:52:27 STEP 2467 H=2467:00 +CPU=  8.683
  06:52:36 STEP 2468 H=2468:00 +CPU=  8.774
  06:52:50 STEP 2469 H=2469:00 +CPU= 13.908
  06:52:59 STEP 2470 H=2470:00 +CPU=  8.774
  06:53:07 STEP 2471 H=2471:00 +CPU=  8.794
  06:53:24 STEP 2472 H=2472:00 +CPU= 16.366
Moving to projects directory: /home/gpuser/slots/0/ICMGGhq0f+002472
Moving to projects directory: /home/gpuser/slots/0/ICMSHhq0f+002472
Moving to projects directory: /home/gpuser/slots/0/ICMUAhq0f+002472
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMGGhq0f+002448
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMSHhq0f+002448
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMUAhq0f+002448
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMGGhq0f+002460
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMSHhq0f+002460
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMUAhq0f+002460
Zipping up the intermediate file: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_0844_1982050100_123_951_12168488_1_r121377581_102.zip
Uploading the intermediate file: upload_file_102.zip
  06:53:32 STEP 2473 H=2473:00 +CPU=  8.619
  06:53:41 STEP 2474 H=2474:00 +CPU=  9.145
  06:53:56 STEP 2475 H=2475:00 +CPU= 14.047
  06:54:04 STEP 2476 H=2476:00 +CPU=  8.812
  06:54:13 STEP 2477 H=2477:00 +CPU=  8.558
  06:54:27 STEP 2478 H=2478:00 +CPU= 14.204
  06:54:36 STEP 2479 H=2479:00 +CPU=  8.645
  06:54:45 STEP 2480 H=2480:00 +CPU=  8.904
  06:54:59 STEP 2481 H=2481:00 +CPU= 14.865
  06:55:08 STEP 2482 H=2482:00 +CPU=  8.909
  06:55:17 STEP 2483 H=2483:00 +CPU=  8.622
  06:55:32 STEP 2484 H=2484:00 +CPU= 15.443
Moving to projects directory: /home/gpuser/slots/0/ICMGGhq0f+002484
Moving to projects directory: /home/gpuser/slots/0/ICMSHhq0f+002484
Moving to projects directory: /home/gpuser/slots/0/ICMUAhq0f+002484
  06:55:41 STEP 2485 H=2485:00 +CPU=  8.660
  06:55:51 STEP 2486 H=2486:00 +CPU=  9.455
  06:56:05 STEP 2487 H=2487:00 +CPU= 14.203
  06:56:14 STEP 2488 H=2488:00 +CPU=  8.933
  06:56:22 STEP 2489 H=2489:00 +CPU=  8.683
  06:56:37 STEP 2490 H=2490:00 +CPU= 14.261
  06:56:45 STEP 2491 H=2491:00 +CPU=  8.640
  06:56:54 STEP 2492 H=2492:00 +CPU=  8.883
  06:57:08 STEP 2493 H=2493:00 +CPU= 13.960
  06:57:17 STEP 2494 H=2494:00 +CPU=  8.821
  06:57:26 STEP 2495 H=2495:00 +CPU=  9.080
  06:57:43 STEP 2496 H=2496:00 +CPU= 16.757
Moving to projects directory: /home/gpuser/slots/0/ICMGGhq0f+002496
Moving to projects directory: /home/gpuser/slots/0/ICMSHhq0f+002496
Moving to projects directory: /home/gpuser/slots/0/ICMUAhq0f+002496
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMGGhq0f+002472
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMSHhq0f+002472
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMUAhq0f+002472
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMGGhq0f+002484
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMSHhq0f+002484
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMUAhq0f+002484
Zipping up the intermediate file: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_0844_1982050100_123_951_12168488_1_r121377581_103.zip
Uploading the intermediate file: upload_file_103.zip
  06:57:51 STEP 2497 H=2497:00 +CPU=  8.608
  06:58:00 STEP 2498 H=2498:00 +CPU=  8.809
  06:58:15 STEP 2499 H=2499:00 +CPU= 14.367
  06:58:23 STEP 2500 H=2500:00 +CPU=  8.808
  06:58:32 STEP 2501 H=2501:00 +CPU=  8.601
  06:58:47 STEP 2502 H=2502:00 +CPU= 14.961
  06:58:56 STEP 2503 H=2503:00 +CPU=  8.558
  06:59:04 STEP 2504 H=2504:00 +CPU=  8.889
  06:59:18 STEP 2505 H=2505:00 +CPU= 14.070
  06:59:27 STEP 2506 H=2506:00 +CPU=  8.938
  06:59:37 STEP 2507 H=2507:00 +CPU=  9.081
  06:59:52 STEP 2508 H=2508:00 +CPU= 15.860
Moving to projects directory: /home/gpuser/slots/0/ICMGGhq0f+002508
Moving to projects directory: /home/gpuser/slots/0/ICMSHhq0f+002508
Moving to projects directory: /home/gpuser/slots/0/ICMUAhq0f+002508
  07:00:01 STEP 2509 H=2509:00 +CPU=  8.716
  07:00:10 STEP 2510 H=2510:00 +CPU=  9.000
  07:00:24 STEP 2511 H=2511:00 +CPU= 13.916
  07:00:33 STEP 2512 H=2512:00 +CPU=  9.281
  07:00:42 STEP 2513 H=2513:00 +CPU=  8.767
  07:00:56 STEP 2514 H=2514:00 +CPU= 14.151
  07:01:05 STEP 2515 H=2515:00 +CPU=  8.494
  07:01:13 STEP 2516 H=2516:00 +CPU=  8.694
  07:01:27 STEP 2517 H=2517:00 +CPU= 13.738
  07:01:36 STEP 2518 H=2518:00 +CPU=  8.743
  07:01:45 STEP 2519 H=2519:00 +CPU=  8.741
  07:02:01 STEP 2520 H=2520:00 +CPU= 16.364
  07:02:09 STEP 2521 H=2521:00 +CPU=  8.504
Moving to projects directory: /home/gpuser/slots/0/ICMGGhq0f+002520
Moving to projects directory: /home/gpuser/slots/0/ICMSHhq0f+002520
Moving to projects directory: /home/gpuser/slots/0/ICMUAhq0f+002520
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMGGhq0f+002496
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMSHhq0f+002496
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMUAhq0f+002496
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMGGhq0f+002508
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMSHhq0f+002508
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMUAhq0f+002508
Zipping up the intermediate file: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_0844_1982050100_123_951_12168488_1_r121377581_104.zip
Uploading the intermediate file: upload_file_104.zip
  07:02:18 STEP 2522 H=2522:00 +CPU=  8.710
  07:02:33 STEP 2523 H=2523:00 +CPU= 14.506
  07:02:43 STEP 2524 H=2524:00 +CPU= 10.640
  07:02:54 STEP 2525 H=2525:00 +CPU= 10.983
  07:03:12 STEP 2526 H=2526:00 +CPU= 17.930
  07:03:23 STEP 2527 H=2527:00 +CPU= 10.866
  07:03:35 STEP 2528 H=2528:00 +CPU= 11.932
  07:03:53 STEP 2529 H=2529:00 +CPU= 17.907
  07:04:04 STEP 2530 H=2530:00 +CPU= 11.301
  07:04:15 STEP 2531 H=2531:00 +CPU= 10.968
  07:04:35 STEP 2532 H=2532:00 +CPU= 19.816
Moving to projects directory: /home/gpuser/slots/0/ICMGGhq0f+002532
Moving to projects directory: /home/gpuser/slots/0/ICMSHhq0f+002532
Moving to projects directory: /home/gpuser/slots/0/ICMUAhq0f+002532
  07:04:46 STEP 2533 H=2533:00 +CPU= 11.135
  07:04:57 STEP 2534 H=2534:00 +CPU= 11.176
  07:05:16 STEP 2535 H=2535:00 +CPU= 18.774
  07:05:28 STEP 2536 H=2536:00 +CPU= 11.501
  07:05:39 STEP 2537 H=2537:00 +CPU= 11.159
  07:05:57 STEP 2538 H=2538:00 +CPU= 18.057
  07:06:08 STEP 2539 H=2539:00 +CPU= 11.153
  07:06:20 STEP 2540 H=2540:00 +CPU= 11.479
  07:06:37 STEP 2541 H=2541:00 +CPU= 17.951
  07:06:49 STEP 2542 H=2542:00 +CPU= 11.443
  07:07:00 STEP 2543 H=2543:00 +CPU= 11.049
  07:07:22 STEP 2544 H=2544:00 +CPU= 21.812
Moving to projects directory: /home/gpuser/slots/0/ICMGGhq0f+002544
Moving to projects directory: /home/gpuser/slots/0/ICMSHhq0f+002544
Moving to projects directory: /home/gpuser/slots/0/ICMUAhq0f+002544
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMGGhq0f+002520
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMSHhq0f+002520
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMUAhq0f+002520
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMGGhq0f+002532
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMSHhq0f+002532
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMUAhq0f+002532
Zipping up the intermediate file: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_0844_1982050100_123_951_12168488_1_r121377581_105.zip
Uploading the intermediate file: upload_file_105.zip
  07:07:32 STEP 2545 H=2545:00 +CPU= 10.696
  07:07:44 STEP 2546 H=2546:00 +CPU= 11.084
  07:08:01 STEP 2547 H=2547:00 +CPU= 17.429
  07:08:12 STEP 2548 H=2548:00 +CPU= 11.185
  07:08:23 STEP 2549 H=2549:00 +CPU= 10.723
  07:08:41 STEP 2550 H=2550:00 +CPU= 17.685
  07:08:52 STEP 2551 H=2551:00 +CPU= 11.702
  07:09:03 STEP 2552 H=2552:00 +CPU= 11.050
  07:09:21 STEP 2553 H=2553:00 +CPU= 17.387
  07:09:32 STEP 2554 H=2554:00 +CPU= 11.102
  07:09:43 STEP 2555 H=2555:00 +CPU= 10.685
  07:10:02 STEP 2556 H=2556:00 +CPU= 19.190
Moving to projects directory: /home/gpuser/slots/0/ICMGGhq0f+002556
Moving to projects directory: /home/gpuser/slots/0/ICMSHhq0f+002556
Moving to projects directory: /home/gpuser/slots/0/ICMUAhq0f+002556
  07:10:13 STEP 2557 H=2557:00 +CPU= 10.792
  07:10:22 STEP 2558 H=2558:00 +CPU=  9.633
  07:10:36 STEP 2559 H=2559:00 +CPU= 13.866
  07:10:45 STEP 2560 H=2560:00 +CPU=  9.328
  07:10:54 STEP 2561 H=2561:00 +CPU=  8.973
  07:11:08 STEP 2562 H=2562:00 +CPU= 14.155
  07:11:17 STEP 2563 H=2563:00 +CPU=  8.569
  07:11:26 STEP 2564 H=2564:00 +CPU=  8.866
  07:11:40 STEP 2565 H=2565:00 +CPU= 13.971
  07:11:49 STEP 2566 H=2566:00 +CPU=  8.894
  07:11:58 STEP 2567 H=2567:00 +CPU=  8.885
  07:12:14 STEP 2568 H=2568:00 +CPU= 16.308
Moving to projects directory: /home/gpuser/slots/0/ICMGGhq0f+002568
Moving to projects directory: /home/gpuser/slots/0/ICMSHhq0f+002568
Moving to projects directory: /home/gpuser/slots/0/ICMUAhq0f+002568
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMGGhq0f+002544
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMSHhq0f+002544
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMUAhq0f+002544
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMGGhq0f+002556
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMSHhq0f+002556
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMUAhq0f+002556
Zipping up the intermediate file: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_0844_1982050100_123_951_12168488_1_r121377581_106.zip
  07:12:23 STEP 2569 H=2569:00 +CPU=  8.539
Uploading the intermediate file: upload_file_106.zip
  07:12:32 STEP 2570 H=2570:00 +CPU=  9.327
  07:12:46 STEP 2571 H=2571:00 +CPU= 13.925
  07:12:55 STEP 2572 H=2572:00 +CPU=  8.802
  07:13:03 STEP 2573 H=2573:00 +CPU=  8.536
  07:13:17 STEP 2574 H=2574:00 +CPU= 14.141
  07:13:26 STEP 2575 H=2575:00 +CPU=  8.601
  07:13:35 STEP 2576 H=2576:00 +CPU=  9.544
  07:13:49 STEP 2577 H=2577:00 +CPU= 13.937
  07:13:58 STEP 2578 H=2578:00 +CPU=  8.986
  07:14:07 STEP 2579 H=2579:00 +CPU=  8.576
  07:14:22 STEP 2580 H=2580:00 +CPU= 15.423
Moving to projects directory: /home/gpuser/slots/0/ICMGGhq0f+002580
Moving to projects directory: /home/gpuser/slots/0/ICMSHhq0f+002580
Moving to projects directory: /home/gpuser/slots/0/ICMUAhq0f+002580
  07:14:31 STEP 2581 H=2581:00 +CPU=  8.739
  07:14:40 STEP 2582 H=2582:00 +CPU=  8.917
  07:14:54 STEP 2583 H=2583:00 +CPU= 13.811
  07:15:03 STEP 2584 H=2584:00 +CPU=  9.565
  07:15:12 STEP 2585 H=2585:00 +CPU=  8.672
  07:15:26 STEP 2586 H=2586:00 +CPU= 14.115
  07:15:35 STEP 2587 H=2587:00 +CPU=  8.668
  07:15:44 STEP 2588 H=2588:00 +CPU=  8.770
  07:15:57 STEP 2589 H=2589:00 +CPU= 13.819
  07:16:06 STEP 2590 H=2590:00 +CPU=  8.917
  07:16:15 STEP 2591 H=2591:00 +CPU=  8.774
  07:16:32 STEP 2592 H=2592:00 +CPU= 16.837
Moving to projects directory: /home/gpuser/slots/0/ICMGGhq0f+002592
Moving to projects directory: /home/gpuser/slots/0/ICMSHhq0f+002592
Moving to projects directory: /home/gpuser/slots/0/ICMUAhq0f+002592
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMGGhq0f+002568
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMSHhq0f+002568
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMUAhq0f+002568
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMGGhq0f+002580
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMSHhq0f+002580
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMUAhq0f+002580
Zipping up the intermediate file: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_0844_1982050100_123_951_12168488_1_r121377581_107.zip
Uploading the intermediate file: upload_file_107.zip
  07:16:41 STEP 2593 H=2593:00 +CPU=  9.078
  07:16:50 STEP 2594 H=2594:00 +CPU=  8.695
  07:17:03 STEP 2595 H=2595:00 +CPU= 13.659
  07:17:12 STEP 2596 H=2596:00 +CPU=  8.718
  07:17:21 STEP 2597 H=2597:00 +CPU=  8.415
  07:17:34 STEP 2598 H=2598:00 +CPU= 13.866
  07:17:43 STEP 2599 H=2599:00 +CPU=  8.466
  07:17:52 STEP 2600 H=2600:00 +CPU=  8.710
  07:18:05 STEP 2601 H=2601:00 +CPU= 13.845
  07:18:14 STEP 2602 H=2602:00 +CPU=  8.861
  07:18:23 STEP 2603 H=2603:00 +CPU=  8.566
  07:18:38 STEP 2604 H=2604:00 +CPU= 15.387
Moving to projects directory: /home/gpuser/slots/0/ICMGGhq0f+002604
Moving to projects directory: /home/gpuser/slots/0/ICMSHhq0f+002604
Moving to projects directory: /home/gpuser/slots/0/ICMUAhq0f+002604
  07:18:47 STEP 2605 H=2605:00 +CPU=  8.665
  07:18:56 STEP 2606 H=2606:00 +CPU=  9.333
  07:19:15 STEP 2607 H=2607:00 +CPU= 18.834
  07:19:26 STEP 2608 H=2608:00 +CPU= 10.963
  07:19:37 STEP 2609 H=2609:00 +CPU= 10.864
  07:19:55 STEP 2610 H=2610:00 +CPU= 17.916
  07:20:05 STEP 2611 H=2611:00 +CPU= 10.682
  07:20:17 STEP 2612 H=2612:00 +CPU= 11.195
  07:20:34 STEP 2613 H=2613:00 +CPU= 17.191
  07:20:45 STEP 2614 H=2614:00 +CPU= 11.237
  07:20:56 STEP 2615 H=2615:00 +CPU= 11.010
  07:21:17 STEP 2616 H=2616:00 +CPU= 20.494
Moving to projects directory: /home/gpuser/slots/0/ICMGGhq0f+002616
Moving to projects directory: /home/gpuser/slots/0/ICMSHhq0f+002616
Moving to projects directory: /home/gpuser/slots/0/ICMUAhq0f+002616
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMGGhq0f+002592
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMSHhq0f+002592
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMUAhq0f+002592
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMGGhq0f+002604
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMSHhq0f+002604
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMUAhq0f+002604
Zipping up the intermediate file: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_0844_1982050100_123_951_12168488_1_r121377581_108.zip
Uploading the intermediate file: upload_file_108.zip
  07:21:27 STEP 2617 H=2617:00 +CPU= 10.731
  07:21:39 STEP 2618 H=2618:00 +CPU= 11.231
  07:21:58 STEP 2619 H=2619:00 +CPU= 19.332
  07:22:09 STEP 2620 H=2620:00 +CPU= 11.463
  07:22:20 STEP 2621 H=2621:00 +CPU= 10.987
  07:22:39 STEP 2622 H=2622:00 +CPU= 18.210
  07:22:50 STEP 2623 H=2623:00 +CPU= 11.149
  07:23:01 STEP 2624 H=2624:00 +CPU= 11.266
  07:23:19 STEP 2625 H=2625:00 +CPU= 17.744
  07:23:30 STEP 2626 H=2626:00 +CPU= 11.293
  07:23:41 STEP 2627 H=2627:00 +CPU= 10.811
  07:24:01 STEP 2628 H=2628:00 +CPU= 20.314
Moving to projects directory: /home/gpuser/slots/0/ICMGGhq0f+002628
Moving to projects directory: /home/gpuser/slots/0/ICMSHhq0f+002628
Moving to projects directory: /home/gpuser/slots/0/ICMUAhq0f+002628
  07:24:12 STEP 2629 H=2629:00 +CPU= 10.881
  07:24:23 STEP 2630 H=2630:00 +CPU= 11.286
  07:24:41 STEP 2631 H=2631:00 +CPU= 17.512
  07:24:52 STEP 2632 H=2632:00 +CPU= 11.290
  07:25:03 STEP 2633 H=2633:00 +CPU= 10.763
  07:25:21 STEP 2634 H=2634:00 +CPU= 17.855
  07:25:33 STEP 2635 H=2635:00 +CPU= 11.849
  07:25:44 STEP 2636 H=2636:00 +CPU= 11.244
  07:26:02 STEP 2637 H=2637:00 +CPU= 17.684
  07:26:13 STEP 2638 H=2638:00 +CPU= 11.327
  07:26:24 STEP 2639 H=2639:00 +CPU= 11.350
  07:26:45 STEP 2640 H=2640:00 +CPU= 20.631
Moving to projects directory: /home/gpuser/slots/0/ICMGGhq0f+002640
Moving to projects directory: /home/gpuser/slots/0/ICMSHhq0f+002640
Moving to projects directory: /home/gpuser/slots/0/ICMUAhq0f+002640
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMGGhq0f+002616
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMSHhq0f+002616
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMUAhq0f+002616
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMGGhq0f+002628
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMSHhq0f+002628
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMUAhq0f+002628
Zipping up the intermediate file: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_0844_1982050100_123_951_12168488_1_r121377581_109.zip
Uploading the intermediate file: upload_file_109.zip
  07:26:56 STEP 2641 H=2641:00 +CPU= 10.821
  07:27:07 STEP 2642 H=2642:00 +CPU= 11.051
Uploading trickle at timestep: 9504000
  07:27:25 STEP 2643 H=2643:00 +CPU= 18.053
  07:27:36 STEP 2644 H=2644:00 +CPU= 11.663
  07:27:47 STEP 2645 H=2645:00 +CPU= 10.706
  07:28:05 STEP 2646 H=2646:00 +CPU= 17.825
  07:28:16 STEP 2647 H=2647:00 +CPU= 11.005
  07:28:27 STEP 2648 H=2648:00 +CPU= 11.307
  07:28:45 STEP 2649 H=2649:00 +CPU= 17.663
  07:28:56 STEP 2650 H=2650:00 +CPU= 11.286
  07:29:07 STEP 2651 H=2651:00 +CPU= 11.198
  07:29:27 STEP 2652 H=2652:00 +CPU= 19.492
Moving to projects directory: /home/gpuser/slots/0/ICMGGhq0f+002652
Moving to projects directory: /home/gpuser/slots/0/ICMSHhq0f+002652
Moving to projects directory: /home/gpuser/slots/0/ICMUAhq0f+002652
  07:29:38 STEP 2653 H=2653:00 +CPU= 10.984
  07:29:49 STEP 2654 H=2654:00 +CPU= 11.233
  07:30:07 STEP 2655 H=2655:00 +CPU= 17.810
  07:30:18 STEP 2656 H=2656:00 +CPU= 11.423
  07:30:29 STEP 2657 H=2657:00 +CPU= 10.805
  07:30:47 STEP 2658 H=2658:00 +CPU= 17.770
  07:30:58 STEP 2659 H=2659:00 +CPU= 10.732
  07:31:09 STEP 2660 H=2660:00 +CPU= 10.958
  07:31:26 STEP 2661 H=2661:00 +CPU= 17.626
  07:31:38 STEP 2662 H=2662:00 +CPU= 12.051
  07:31:50 STEP 2663 H=2663:00 +CPU= 11.470
  07:32:10 STEP 2664 H=2664:00 +CPU= 20.577
Moving to projects directory: /home/gpuser/slots/0/ICMGGhq0f+002664
Moving to projects directory: /home/gpuser/slots/0/ICMSHhq0f+002664
Moving to projects directory: /home/gpuser/slots/0/ICMUAhq0f+002664
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMGGhq0f+002640
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMSHhq0f+002640
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMUAhq0f+002640
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMGGhq0f+002652
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMSHhq0f+002652
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMUAhq0f+002652
Zipping up the intermediate file: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_0844_1982050100_123_951_12168488_1_r121377581_110.zip
Uploading the intermediate file: upload_file_110.zip
  07:32:21 STEP 2665 H=2665:00 +CPU= 10.842
  07:32:32 STEP 2666 H=2666:00 +CPU= 11.165
  07:32:50 STEP 2667 H=2667:00 +CPU= 17.855
  07:33:01 STEP 2668 H=2668:00 +CPU= 11.233
  07:33:12 STEP 2669 H=2669:00 +CPU= 10.742
  07:33:30 STEP 2670 H=2670:00 +CPU= 17.883
  07:33:41 STEP 2671 H=2671:00 +CPU= 10.918
  07:33:52 STEP 2672 H=2672:00 +CPU= 11.036
  07:34:10 STEP 2673 H=2673:00 +CPU= 18.174
  07:34:22 STEP 2674 H=2674:00 +CPU= 11.584
  07:34:33 STEP 2675 H=2675:00 +CPU= 10.816
  07:34:52 STEP 2676 H=2676:00 +CPU= 19.239
Moving to projects directory: /home/gpuser/slots/0/ICMGGhq0f+002676
Moving to projects directory: /home/gpuser/slots/0/ICMSHhq0f+002676
Moving to projects directory: /home/gpuser/slots/0/ICMUAhq0f+002676
  07:35:03 STEP 2677 H=2677:00 +CPU= 10.831
  07:35:14 STEP 2678 H=2678:00 +CPU= 10.888
  07:35:31 STEP 2679 H=2679:00 +CPU= 17.064
  07:35:42 STEP 2680 H=2680:00 +CPU= 11.008
  07:35:52 STEP 2681 H=2681:00 +CPU= 10.558
  07:36:10 STEP 2682 H=2682:00 +CPU= 17.724
  07:36:20 STEP 2683 H=2683:00 +CPU= 10.484
  07:36:32 STEP 2684 H=2684:00 +CPU= 11.114
  07:36:49 STEP 2685 H=2685:00 +CPU= 17.948
  07:37:01 STEP 2686 H=2686:00 +CPU= 11.350
  07:37:12 STEP 2687 H=2687:00 +CPU= 11.142
  07:37:32 STEP 2688 H=2688:00 +CPU= 20.177
Moving to projects directory: /home/gpuser/slots/0/ICMGGhq0f+002688
Moving to projects directory: /home/gpuser/slots/0/ICMSHhq0f+002688
Moving to projects directory: /home/gpuser/slots/0/ICMUAhq0f+002688
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMGGhq0f+002664
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMSHhq0f+002664
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMUAhq0f+002664
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMGGhq0f+002676
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMSHhq0f+002676
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMUAhq0f+002676
Zipping up the intermediate file: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_0844_1982050100_123_951_12168488_1_r121377581_111.zip
Uploading the intermediate file: upload_file_111.zip
  07:37:43 STEP 2689 H=2689:00 +CPU= 10.865
  07:37:54 STEP 2690 H=2690:00 +CPU= 10.919
  07:38:11 STEP 2691 H=2691:00 +CPU= 17.160
  07:38:22 STEP 2692 H=2692:00 +CPU= 10.918
  07:38:33 STEP 2693 H=2693:00 +CPU= 10.689
  07:38:50 STEP 2694 H=2694:00 +CPU= 17.718
  07:39:01 STEP 2695 H=2695:00 +CPU= 10.773
  07:39:12 STEP 2696 H=2696:00 +CPU= 11.091
  07:39:30 STEP 2697 H=2697:00 +CPU= 17.447
  07:39:42 STEP 2698 H=2698:00 +CPU= 11.976
  07:39:53 STEP 2699 H=2699:00 +CPU= 11.233
  07:40:12 STEP 2700 H=2700:00 +CPU= 19.098
Moving to projects directory: /home/gpuser/slots/0/ICMGGhq0f+002700
Moving to projects directory: /home/gpuser/slots/0/ICMSHhq0f+002700
Moving to projects directory: /home/gpuser/slots/0/ICMUAhq0f+002700
  07:40:23 STEP 2701 H=2701:00 +CPU= 10.774
  07:40:34 STEP 2702 H=2702:00 +CPU= 11.056
  07:40:51 STEP 2703 H=2703:00 +CPU= 17.432
  07:41:02 STEP 2704 H=2704:00 +CPU= 11.127
  07:41:13 STEP 2705 H=2705:00 +CPU= 10.827
  07:41:31 STEP 2706 H=2706:00 +CPU= 17.589
  07:41:42 STEP 2707 H=2707:00 +CPU= 10.821
  07:41:53 STEP 2708 H=2708:00 +CPU= 11.032
  07:42:10 STEP 2709 H=2709:00 +CPU= 17.475
  07:42:21 STEP 2710 H=2710:00 +CPU= 11.113
  07:42:32 STEP 2711 H=2711:00 +CPU= 11.007
  07:42:53 STEP 2712 H=2712:00 +CPU= 20.279
Moving to projects directory: /home/gpuser/slots/0/ICMGGhq0f+002712
Moving to projects directory: /home/gpuser/slots/0/ICMSHhq0f+002712
Moving to projects directory: /home/gpuser/slots/0/ICMUAhq0f+002712
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMGGhq0f+002688
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMSHhq0f+002688
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMUAhq0f+002688
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMGGhq0f+002700
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMSHhq0f+002700
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMUAhq0f+002700
Zipping up the intermediate file: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_0844_1982050100_123_951_12168488_1_r121377581_112.zip
Uploading the intermediate file: upload_file_112.zip
  07:43:04 STEP 2713 H=2713:00 +CPU= 11.586
  07:43:15 STEP 2714 H=2714:00 +CPU= 11.185
  07:43:33 STEP 2715 H=2715:00 +CPU= 17.226
  07:43:44 STEP 2716 H=2716:00 +CPU= 11.009
  07:43:54 STEP 2717 H=2717:00 +CPU= 10.876
  07:44:12 STEP 2718 H=2718:00 +CPU= 17.805
  07:44:23 STEP 2719 H=2719:00 +CPU= 10.852
  07:44:34 STEP 2720 H=2720:00 +CPU= 11.046
  07:44:48 STEP 2721 H=2721:00 +CPU= 14.146
  07:44:57 STEP 2722 H=2722:00 +CPU=  8.849
  07:45:06 STEP 2723 H=2723:00 +CPU=  8.578
  07:45:21 STEP 2724 H=2724:00 +CPU= 15.317
  07:45:30 STEP 2725 H=2725:00 +CPU=  8.564
Moving to projects directory: /home/gpuser/slots/0/ICMGGhq0f+002724
Moving to projects directory: /home/gpuser/slots/0/ICMSHhq0f+002724
Moving to projects directory: /home/gpuser/slots/0/ICMUAhq0f+002724
  07:45:38 STEP 2726 H=2726:00 +CPU=  8.799
  07:45:52 STEP 2727 H=2727:00 +CPU= 13.674
  07:46:01 STEP 2728 H=2728:00 +CPU=  8.815
  07:46:09 STEP 2729 H=2729:00 +CPU=  8.454
  07:46:23 STEP 2730 H=2730:00 +CPU= 13.932
  07:46:32 STEP 2731 H=2731:00 +CPU=  8.492
  07:46:41 STEP 2732 H=2732:00 +CPU=  9.130
  07:46:59 STEP 2733 H=2733:00 +CPU= 18.092
  07:47:11 STEP 2734 H=2734:00 +CPU= 11.480
  07:47:21 STEP 2735 H=2735:00 +CPU= 10.974
  07:47:42 STEP 2736 H=2736:00 +CPU= 20.296
Moving to projects directory: /home/gpuser/slots/0/ICMGGhq0f+002736
Moving to projects directory: /home/gpuser/slots/0/ICMSHhq0f+002736
Moving to projects directory: /home/gpuser/slots/0/ICMUAhq0f+002736
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMGGhq0f+002712
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMSHhq0f+002712
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMUAhq0f+002712
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMGGhq0f+002724
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMSHhq0f+002724
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMUAhq0f+002724
Zipping up the intermediate file: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_0844_1982050100_123_951_12168488_1_r121377581_113.zip
Uploading the intermediate file: upload_file_113.zip
  07:47:53 STEP 2737 H=2737:00 +CPU= 10.778
  07:48:03 STEP 2738 H=2738:00 +CPU= 10.204
  07:48:17 STEP 2739 H=2739:00 +CPU= 13.872
  07:48:25 STEP 2740 H=2740:00 +CPU=  8.731
  07:48:34 STEP 2741 H=2741:00 +CPU=  8.547
  07:48:52 STEP 2742 H=2742:00 +CPU= 17.724
  07:49:02 STEP 2743 H=2743:00 +CPU= 10.737
  07:49:14 STEP 2744 H=2744:00 +CPU= 11.115
  07:49:31 STEP 2745 H=2745:00 +CPU= 17.688
  07:49:42 STEP 2746 H=2746:00 +CPU= 11.183
  07:49:53 STEP 2747 H=2747:00 +CPU= 10.753
  07:50:13 STEP 2748 H=2748:00 +CPU= 19.620
Moving to projects directory: /home/gpuser/slots/0/ICMGGhq0f+002748
Moving to projects directory: /home/gpuser/slots/0/ICMSHhq0f+002748
Moving to projects directory: /home/gpuser/slots/0/ICMUAhq0f+002748
  07:50:24 STEP 2749 H=2749:00 +CPU= 11.001
  07:50:35 STEP 2750 H=2750:00 +CPU= 11.033
  07:50:52 STEP 2751 H=2751:00 +CPU= 17.376
  07:51:03 STEP 2752 H=2752:00 +CPU= 11.004
  07:51:14 STEP 2753 H=2753:00 +CPU= 10.708
  07:51:31 STEP 2754 H=2754:00 +CPU= 17.517
  07:51:42 STEP 2755 H=2755:00 +CPU= 10.817
  07:51:53 STEP 2756 H=2756:00 +CPU= 11.135
  07:52:12 STEP 2757 H=2757:00 +CPU= 18.501
  07:52:23 STEP 2758 H=2758:00 +CPU= 11.448
  07:52:34 STEP 2759 H=2759:00 +CPU= 11.149
  07:52:55 STEP 2760 H=2760:00 +CPU= 20.167
Moving to projects directory: /home/gpuser/slots/0/ICMGGhq0f+002760
Moving to projects directory: /home/gpuser/slots/0/ICMSHhq0f+002760
Moving to projects directory: /home/gpuser/slots/0/ICMUAhq0f+002760
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMGGhq0f+002736
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMSHhq0f+002736
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMUAhq0f+002736
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMGGhq0f+002748
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMSHhq0f+002748
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMUAhq0f+002748
Zipping up the intermediate file: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_0844_1982050100_123_951_12168488_1_r121377581_114.zip
Uploading the intermediate file: upload_file_114.zip
  07:53:05 STEP 2761 H=2761:00 +CPU= 10.761
  07:53:16 STEP 2762 H=2762:00 +CPU= 11.094
  07:53:34 STEP 2763 H=2763:00 +CPU= 17.409
  07:53:45 STEP 2764 H=2764:00 +CPU= 11.023
  07:53:56 STEP 2765 H=2765:00 +CPU= 10.702
  07:54:13 STEP 2766 H=2766:00 +CPU= 17.759
  07:54:24 STEP 2767 H=2767:00 +CPU= 10.768
  07:54:35 STEP 2768 H=2768:00 +CPU= 11.287
  07:54:53 STEP 2769 H=2769:00 +CPU= 17.509
  07:55:04 STEP 2770 H=2770:00 +CPU= 11.412
  07:55:16 STEP 2771 H=2771:00 +CPU= 11.945
  07:55:36 STEP 2772 H=2772:00 +CPU= 19.435
Moving to projects directory: /home/gpuser/slots/0/ICMGGhq0f+002772
Moving to projects directory: /home/gpuser/slots/0/ICMSHhq0f+002772
Moving to projects directory: /home/gpuser/slots/0/ICMUAhq0f+002772
  07:55:47 STEP 2773 H=2773:00 +CPU= 10.973
  07:55:58 STEP 2774 H=2774:00 +CPU= 11.051
  07:56:15 STEP 2775 H=2775:00 +CPU= 17.304
  07:56:26 STEP 2776 H=2776:00 +CPU= 11.142
  07:56:37 STEP 2777 H=2777:00 +CPU= 10.834
  07:56:55 STEP 2778 H=2778:00 +CPU= 17.543
  07:57:05 STEP 2779 H=2779:00 +CPU= 10.801
  07:57:17 STEP 2780 H=2780:00 +CPU= 11.279
  07:57:34 STEP 2781 H=2781:00 +CPU= 17.643
  07:57:45 STEP 2782 H=2782:00 +CPU= 11.086
  07:57:58 STEP 2783 H=2783:00 +CPU= 12.416
  07:58:19 STEP 2784 H=2784:00 +CPU= 20.717
Moving to projects directory: /home/gpuser/slots/0/ICMGGhq0f+002784
Moving to projects directory: /home/gpuser/slots/0/ICMSHhq0f+002784
Moving to projects directory: /home/gpuser/slots/0/ICMUAhq0f+002784
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMGGhq0f+002760
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMSHhq0f+002760
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMUAhq0f+002760
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMGGhq0f+002772
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMSHhq0f+002772
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMUAhq0f+002772
Zipping up the intermediate file: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_0844_1982050100_123_951_12168488_1_r121377581_115.zip
Uploading the intermediate file: upload_file_115.zip
  07:58:29 STEP 2785 H=2785:00 +CPU= 10.908
  07:58:41 STEP 2786 H=2786:00 +CPU= 11.095
  07:58:58 STEP 2787 H=2787:00 +CPU= 17.467
  07:59:09 STEP 2788 H=2788:00 +CPU= 11.010
  07:59:20 STEP 2789 H=2789:00 +CPU= 10.849
  07:59:38 STEP 2790 H=2790:00 +CPU= 17.870
  07:59:49 STEP 2791 H=2791:00 +CPU= 10.821
  08:00:00 STEP 2792 H=2792:00 +CPU= 11.580
  08:00:18 STEP 2793 H=2793:00 +CPU= 17.924
  08:00:29 STEP 2794 H=2794:00 +CPU= 11.112
  08:00:40 STEP 2795 H=2795:00 +CPU= 10.865
  08:01:00 STEP 2796 H=2796:00 +CPU= 19.525
Moving to projects directory: /home/gpuser/slots/0/ICMGGhq0f+002796
Moving to projects directory: /home/gpuser/slots/0/ICMSHhq0f+002796
Moving to projects directory: /home/gpuser/slots/0/ICMUAhq0f+002796
  08:01:10 STEP 2797 H=2797:00 +CPU= 10.722
  08:01:21 STEP 2798 H=2798:00 +CPU= 11.122
  08:01:39 STEP 2799 H=2799:00 +CPU= 17.263
  08:01:50 STEP 2800 H=2800:00 +CPU= 11.185
  08:02:01 STEP 2801 H=2801:00 +CPU= 10.815
  08:02:19 STEP 2802 H=2802:00 +CPU= 17.847
  08:02:29 STEP 2803 H=2803:00 +CPU= 10.893
  08:02:41 STEP 2804 H=2804:00 +CPU= 11.254
  08:02:59 STEP 2805 H=2805:00 +CPU= 18.335
  08:03:11 STEP 2806 H=2806:00 +CPU= 11.837
  08:03:22 STEP 2807 H=2807:00 +CPU= 11.114
  08:03:43 STEP 2808 H=2808:00 +CPU= 20.710
Moving to projects directory: /home/gpuser/slots/0/ICMGGhq0f+002808
Moving to projects directory: /home/gpuser/slots/0/ICMSHhq0f+002808
Moving to projects directory: /home/gpuser/slots/0/ICMUAhq0f+002808
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMGGhq0f+002784
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMSHhq0f+002784
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMUAhq0f+002784
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMGGhq0f+002796
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMSHhq0f+002796
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMUAhq0f+002796
Zipping up the intermediate file: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_0844_1982050100_123_951_12168488_1_r121377581_116.zip
Uploading the intermediate file: upload_file_116.zip
  08:03:54 STEP 2809 H=2809:00 +CPU= 10.987
  08:04:05 STEP 2810 H=2810:00 +CPU= 11.150
  08:04:22 STEP 2811 H=2811:00 +CPU= 17.630
  08:04:34 STEP 2812 H=2812:00 +CPU= 11.131
  08:04:44 STEP 2813 H=2813:00 +CPU= 10.784
  08:05:02 STEP 2814 H=2814:00 +CPU= 17.965
  08:05:13 STEP 2815 H=2815:00 +CPU= 10.940
  08:05:25 STEP 2816 H=2816:00 +CPU= 11.255
  08:05:42 STEP 2817 H=2817:00 +CPU= 17.471
  08:05:53 STEP 2818 H=2818:00 +CPU= 11.264
  08:06:04 STEP 2819 H=2819:00 +CPU= 10.776
  08:06:23 STEP 2820 H=2820:00 +CPU= 19.389
Moving to projects directory: /home/gpuser/slots/0/ICMGGhq0f+002820
Moving to projects directory: /home/gpuser/slots/0/ICMSHhq0f+002820
Moving to projects directory: /home/gpuser/slots/0/ICMUAhq0f+002820
  08:06:34 STEP 2821 H=2821:00 +CPU= 10.841
  08:06:46 STEP 2822 H=2822:00 +CPU= 11.245
  08:07:03 STEP 2823 H=2823:00 +CPU= 17.682
  08:07:14 STEP 2824 H=2824:00 +CPU= 11.201
  08:07:25 STEP 2825 H=2825:00 +CPU= 10.993
  08:07:44 STEP 2826 H=2826:00 +CPU= 18.506
  08:07:55 STEP 2827 H=2827:00 +CPU= 11.543
  08:08:06 STEP 2828 H=2828:00 +CPU= 10.938
  08:08:24 STEP 2829 H=2829:00 +CPU= 17.651
  08:08:35 STEP 2830 H=2830:00 +CPU= 11.296
  08:08:46 STEP 2831 H=2831:00 +CPU= 11.129
  08:09:07 STEP 2832 H=2832:00 +CPU= 20.771
Moving to projects directory: /home/gpuser/slots/0/ICMGGhq0f+002832
Moving to projects directory: /home/gpuser/slots/0/ICMSHhq0f+002832
Moving to projects directory: /home/gpuser/slots/0/ICMUAhq0f+002832
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMGGhq0f+002808
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMSHhq0f+002808
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMUAhq0f+002808
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMGGhq0f+002820
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMSHhq0f+002820
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMUAhq0f+002820
Zipping up the intermediate file: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_0844_1982050100_123_951_12168488_1_r121377581_117.zip
Uploading the intermediate file: upload_file_117.zip
  08:09:18 STEP 2833 H=2833:00 +CPU= 11.095
  08:09:30 STEP 2834 H=2834:00 +CPU= 11.290
  08:09:48 STEP 2835 H=2835:00 +CPU= 17.932
  08:09:59 STEP 2836 H=2836:00 +CPU= 11.514
  08:10:10 STEP 2837 H=2837:00 +CPU= 10.907
  08:10:29 STEP 2838 H=2838:00 +CPU= 19.217
  08:10:40 STEP 2839 H=2839:00 +CPU= 10.804
  08:10:51 STEP 2840 H=2840:00 +CPU= 11.216
  08:11:09 STEP 2841 H=2841:00 +CPU= 17.844
  08:11:20 STEP 2842 H=2842:00 +CPU= 11.243
  08:11:31 STEP 2843 H=2843:00 +CPU= 10.855
  08:11:51 STEP 2844 H=2844:00 +CPU= 19.697
Moving to projects directory: /home/gpuser/slots/0/ICMGGhq0f+002844
Moving to projects directory: /home/gpuser/slots/0/ICMSHhq0f+002844
Moving to projects directory: /home/gpuser/slots/0/ICMUAhq0f+002844
  08:12:02 STEP 2845 H=2845:00 +CPU= 11.019
  08:12:13 STEP 2846 H=2846:00 +CPU= 11.176
  08:12:31 STEP 2847 H=2847:00 +CPU= 17.636
  08:12:42 STEP 2848 H=2848:00 +CPU= 11.206
  08:12:53 STEP 2849 H=2849:00 +CPU= 10.936
  08:13:10 STEP 2850 H=2850:00 +CPU= 17.399
  08:13:19 STEP 2851 H=2851:00 +CPU=  8.552
  08:13:28 STEP 2852 H=2852:00 +CPU=  8.751
  08:13:41 STEP 2853 H=2853:00 +CPU= 13.838
  08:13:50 STEP 2854 H=2854:00 +CPU=  8.766
  08:14:00 STEP 2855 H=2855:00 +CPU=  9.527
  08:14:16 STEP 2856 H=2856:00 +CPU= 16.530
Moving to projects directory: /home/gpuser/slots/0/ICMGGhq0f+002856
Moving to projects directory: /home/gpuser/slots/0/ICMSHhq0f+002856
Moving to projects directory: /home/gpuser/slots/0/ICMUAhq0f+002856
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMGGhq0f+002832
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMSHhq0f+002832
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMUAhq0f+002832
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMGGhq0f+002844
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMSHhq0f+002844
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMUAhq0f+002844
Zipping up the intermediate file: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_0844_1982050100_123_951_12168488_1_r121377581_118.zip
Uploading the intermediate file: upload_file_118.zip
  08:14:25 STEP 2857 H=2857:00 +CPU=  8.734
  08:14:34 STEP 2858 H=2858:00 +CPU=  8.937
  08:14:48 STEP 2859 H=2859:00 +CPU= 14.373
  08:14:58 STEP 2860 H=2860:00 +CPU=  9.504
  08:15:07 STEP 2861 H=2861:00 +CPU=  9.051
  08:15:21 STEP 2862 H=2862:00 +CPU= 14.070
  08:15:29 STEP 2863 H=2863:00 +CPU=  8.563
  08:15:39 STEP 2864 H=2864:00 +CPU=  9.062
  08:15:53 STEP 2865 H=2865:00 +CPU= 14.433
  08:16:02 STEP 2866 H=2866:00 +CPU=  9.095
  08:16:11 STEP 2867 H=2867:00 +CPU=  8.691
  08:16:26 STEP 2868 H=2868:00 +CPU= 15.661
Moving to projects directory: /home/gpuser/slots/0/ICMGGhq0f+002868
Moving to projects directory: /home/gpuser/slots/0/ICMSHhq0f+002868
Moving to projects directory: /home/gpuser/slots/0/ICMUAhq0f+002868
  08:16:35 STEP 2869 H=2869:00 +CPU=  8.708
  08:16:44 STEP 2870 H=2870:00 +CPU=  8.787
  08:16:58 STEP 2871 H=2871:00 +CPU= 14.072
  08:17:08 STEP 2872 H=2872:00 +CPU=  9.699
  08:17:16 STEP 2873 H=2873:00 +CPU=  8.709
  08:17:31 STEP 2874 H=2874:00 +CPU= 14.494
  08:17:40 STEP 2875 H=2875:00 +CPU=  8.721
  08:17:48 STEP 2876 H=2876:00 +CPU=  8.879
  08:18:03 STEP 2877 H=2877:00 +CPU= 14.336
  08:18:12 STEP 2878 H=2878:00 +CPU=  9.071
  08:18:21 STEP 2879 H=2879:00 +CPU=  8.956
  08:18:38 STEP 2880 H=2880:00 +CPU= 16.711
Moving to projects directory: /home/gpuser/slots/0/ICMGGhq0f+002880
Moving to projects directory: /home/gpuser/slots/0/ICMSHhq0f+002880
Moving to projects directory: /home/gpuser/slots/0/ICMUAhq0f+002880
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMGGhq0f+002856
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMSHhq0f+002856
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMUAhq0f+002856
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMGGhq0f+002868
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMSHhq0f+002868
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMUAhq0f+002868
Zipping up the intermediate file: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_0844_1982050100_123_951_12168488_1_r121377581_119.zip
Uploading the intermediate file: upload_file_119.zip
  08:18:46 STEP 2881 H=2881:00 +CPU=  8.693
  08:18:55 STEP 2882 H=2882:00 +CPU=  8.830
Uploading trickle at timestep: 10368000
  08:19:09 STEP 2883 H=2883:00 +CPU= 14.162
  08:19:18 STEP 2884 H=2884:00 +CPU=  9.169
  08:19:27 STEP 2885 H=2885:00 +CPU=  8.554
  08:19:41 STEP 2886 H=2886:00 +CPU= 14.493
  08:19:50 STEP 2887 H=2887:00 +CPU=  8.871
  08:19:59 STEP 2888 H=2888:00 +CPU=  9.053
  08:20:14 STEP 2889 H=2889:00 +CPU= 14.632
  08:20:23 STEP 2890 H=2890:00 +CPU=  8.959
  08:20:32 STEP 2891 H=2891:00 +CPU=  8.620
  08:20:47 STEP 2892 H=2892:00 +CPU= 15.248
Moving to projects directory: /home/gpuser/slots/0/ICMGGhq0f+002892
Moving to projects directory: /home/gpuser/slots/0/ICMSHhq0f+002892
Moving to projects directory: /home/gpuser/slots/0/ICMUAhq0f+002892
  08:20:55 STEP 2893 H=2893:00 +CPU=  8.514
  08:21:04 STEP 2894 H=2894:00 +CPU=  8.736
  08:21:18 STEP 2895 H=2895:00 +CPU= 13.707
  08:21:27 STEP 2896 H=2896:00 +CPU=  8.754
  08:21:35 STEP 2897 H=2897:00 +CPU=  8.434
  08:21:49 STEP 2898 H=2898:00 +CPU= 13.941
  08:21:57 STEP 2899 H=2899:00 +CPU=  8.456
  08:22:06 STEP 2900 H=2900:00 +CPU=  8.670
  08:22:20 STEP 2901 H=2901:00 +CPU= 13.800
  08:22:29 STEP 2902 H=2902:00 +CPU=  8.810
  08:22:37 STEP 2903 H=2903:00 +CPU=  8.777
  08:22:54 STEP 2904 H=2904:00 +CPU= 16.105
Moving to projects directory: /home/gpuser/slots/0/ICMGGhq0f+002904
Moving to projects directory: /home/gpuser/slots/0/ICMSHhq0f+002904
Moving to projects directory: /home/gpuser/slots/0/ICMUAhq0f+002904
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMGGhq0f+002880
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMSHhq0f+002880
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMUAhq0f+002880
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMGGhq0f+002892
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMSHhq0f+002892
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMUAhq0f+002892
Zipping up the intermediate file: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_0844_1982050100_123_951_12168488_1_r121377581_120.zip
Uploading the intermediate file: upload_file_120.zip
  08:23:02 STEP 2905 H=2905:00 +CPU=  8.531
  08:23:11 STEP 2906 H=2906:00 +CPU=  8.680
  08:23:25 STEP 2907 H=2907:00 +CPU= 13.978
  08:23:36 STEP 2908 H=2908:00 +CPU= 10.946
  08:23:46 STEP 2909 H=2909:00 +CPU= 10.769
  08:24:04 STEP 2910 H=2910:00 +CPU= 17.721
  08:24:15 STEP 2911 H=2911:00 +CPU= 10.900
  08:24:26 STEP 2912 H=2912:00 +CPU= 11.395
  08:24:44 STEP 2913 H=2913:00 +CPU= 17.799
  08:24:57 STEP 2914 H=2914:00 +CPU= 12.710
  08:25:08 STEP 2915 H=2915:00 +CPU= 11.014
  08:25:28 STEP 2916 H=2916:00 +CPU= 19.753
Moving to projects directory: /home/gpuser/slots/0/ICMGGhq0f+002916
Moving to projects directory: /home/gpuser/slots/0/ICMSHhq0f+002916
Moving to projects directory: /home/gpuser/slots/0/ICMUAhq0f+002916
  08:25:39 STEP 2917 H=2917:00 +CPU= 10.915
  08:25:50 STEP 2918 H=2918:00 +CPU= 11.350
  08:26:08 STEP 2919 H=2919:00 +CPU= 17.779
  08:26:20 STEP 2920 H=2920:00 +CPU= 12.030
  08:26:31 STEP 2921 H=2921:00 +CPU= 10.669
  08:26:49 STEP 2922 H=2922:00 +CPU= 18.266
  08:27:00 STEP 2923 H=2923:00 +CPU= 10.915
  08:27:11 STEP 2924 H=2924:00 +CPU= 10.992
  08:27:28 STEP 2925 H=2925:00 +CPU= 17.556
  08:27:39 STEP 2926 H=2926:00 +CPU= 11.117
  08:27:51 STEP 2927 H=2927:00 +CPU= 11.724
  08:28:11 STEP 2928 H=2928:00 +CPU= 20.298
Moving to projects directory: /home/gpuser/slots/0/ICMGGhq0f+002928
Moving to projects directory: /home/gpuser/slots/0/ICMSHhq0f+002928
Moving to projects directory: /home/gpuser/slots/0/ICMUAhq0f+002928
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMGGhq0f+002904
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMSHhq0f+002904
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMUAhq0f+002904
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMGGhq0f+002916
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMSHhq0f+002916
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMUAhq0f+002916
Zipping up the intermediate file: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_0844_1982050100_123_951_12168488_1_r121377581_121.zip
  08:28:22 STEP 2929 H=2929:00 +CPU= 10.770
Uploading the intermediate file: upload_file_121.zip
  08:28:33 STEP 2930 H=2930:00 +CPU= 11.028
  08:28:50 STEP 2931 H=2931:00 +CPU= 17.170
  08:29:01 STEP 2932 H=2932:00 +CPU= 10.936
  08:29:12 STEP 2933 H=2933:00 +CPU= 10.722
  08:29:30 STEP 2934 H=2934:00 +CPU= 17.629
  08:29:40 STEP 2935 H=2935:00 +CPU= 10.695
  08:29:52 STEP 2936 H=2936:00 +CPU= 11.216
  08:30:09 STEP 2937 H=2937:00 +CPU= 17.272
  08:30:20 STEP 2938 H=2938:00 +CPU= 11.199
  08:30:32 STEP 2939 H=2939:00 +CPU= 12.086
  08:30:52 STEP 2940 H=2940:00 +CPU= 19.434
Moving to projects directory: /home/gpuser/slots/0/ICMGGhq0f+002940
Moving to projects directory: /home/gpuser/slots/0/ICMSHhq0f+002940
Moving to projects directory: /home/gpuser/slots/0/ICMUAhq0f+002940
  08:31:02 STEP 2941 H=2941:00 +CPU= 10.844
  08:31:13 STEP 2942 H=2942:00 +CPU= 10.990
  08:31:30 STEP 2943 H=2943:00 +CPU= 17.044
  08:31:41 STEP 2944 H=2944:00 +CPU= 10.938
  08:31:52 STEP 2945 H=2945:00 +CPU= 10.662
  08:32:10 STEP 2946 H=2946:00 +CPU= 17.539
  08:32:19 STEP 2947 H=2947:00 +CPU=  8.945
  08:32:27 STEP 2948 H=2948:00 +CPU=  8.787
  08:32:41 STEP 2949 H=2949:00 +CPU= 13.839
  08:32:50 STEP 2950 H=2950:00 +CPU=  8.858
  08:32:59 STEP 2951 H=2951:00 +CPU=  8.787
  08:33:16 STEP 2952 H=2952:00 +CPU= 17.057
The child process terminated with status: 0
Moving to projects directory: /home/gpuser/slots/0/ICMGGhq0f+002952
Moving to projects directory: /home/gpuser/slots/0/ICMSHhq0f+002952
Moving to projects directory: /home/gpuser/slots/0/ICMUAhq0f+002952
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMUAhq0f+002940
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMGGhq0f+002940
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMGGhq0f+002928
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMSHhq0f+002928
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMGGhq0f+002952
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMSHhq0f+002952
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMSHhq0f+002940
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMUAhq0f+002952
Adding to the zip: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_12168488/ICMUAhq0f+002928
Zipping up the final file: /home/gpuser/projects/climateprediction.net/oifs_43r3_ps_0844_1982050100_123_951_12168488_1_r121377581_122.zip
Uploading the final file: upload_file_122.zip
Uploading trickle at timestep: 10623600
08:36:40 (150): called boinc_finish(0)

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 May 2023 16:21:24 1537625 22277797 oifs_43r3_ps_0844_1982050100_123_951_12168488_1 10,623,600 40,218 0.0038
09 May 2023 16:21:19 1537625 22277797 oifs_43r3_ps_0844_1982050100_123_951_12168488_1 10,368,000 39,340 0.0038
09 May 2023 16:21:15 1537625 22277797 oifs_43r3_ps_0844_1982050100_123_951_12168488_1 9,504,000 36,227 0.0038
09 May 2023 16:21:15 1537625 22277797 oifs_43r3_ps_0844_1982050100_123_951_12168488_1 8,640,000 33,466 0.0039
09 May 2023 16:21:11 1537625 22277797 oifs_43r3_ps_0844_1982050100_123_951_12168488_1 7,776,000 30,344 0.0039
09 May 2023 16:21:07 1537625 22277797 oifs_43r3_ps_0844_1982050100_123_951_12168488_1 6,912,000 27,174 0.0039
09 May 2023 16:21:03 1537625 22277797 oifs_43r3_ps_0844_1982050100_123_951_12168488_1 6,048,000 23,902 0.0040
09 May 2023 16:20:59 1537625 22277797 oifs_43r3_ps_0844_1982050100_123_951_12168488_1 5,184,000 20,564 0.0040
09 May 2023 16:20:55 1537625 22277797 oifs_43r3_ps_0844_1982050100_123_951_12168488_1 4,320,000 17,161 0.0040
09 May 2023 16:20:51 1537625 22277797 oifs_43r3_ps_0844_1982050100_123_951_12168488_1 3,456,000 13,762 0.0040
09 May 2023 16:20:46 1537625 22277797 oifs_43r3_ps_0844_1982050100_123_951_12168488_1 2,592,000 10,395 0.0040
09 May 2023 16:20:42 1537625 22277797 oifs_43r3_ps_0844_1982050100_123_951_12168488_1 1,728,000 7,085 0.0041
09 May 2023 16:20:42 1537625 22277797 oifs_43r3_ps_0844_1982050100_123_951_12168488_1 864,000 3,677 0.0043


©2024 climateprediction.net