climateprediction.net home page
Task 22315161

Task 22315161

Name oifs_43r3_00ae_2019110100_123_993_12213812_0
Workunit 12213812
Created 24 Feb 2023, 12:47:52 UTC
Sent 24 Feb 2023, 23:50:58 UTC
Report deadline 25 Apr 2023, 23:50:58 UTC
Received 25 Feb 2023, 19:04:16 UTC
Server state Over
Outcome Success
Client state Done
Exit status 0 (0x00000000)
Computer ID 1539146
Run time 16 hours 25 min 25 sec
CPU time 16 hours 17 min 43 sec
Validate state Valid
Credit 14,873.04
Device peak FLOPS 8.36 GFLOPS
Application version OpenIFS 43r3 v1.21
x86_64-pc-linux-gnu
Peak working set size 4,397.77 MB
Peak swap size 4,983.85 MB
Peak disk usage 1,278.77 MB
Stderr
<core_client_version>7.18.1</core_client_version>
<![CDATA[
<stderr_txt>
lib/41000/slots/8/ICMUAhnw4+002352
  08:04:09 STEP 2353 H=2353:00 +CPU= 32.589
  08:04:41 STEP 2354 H=2354:00 +CPU= 31.333
  08:05:01 STEP 2355 H=2355:00 +CPU= 20.716
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMGGhnw4+002340
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMSHhnw4+002340
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMUAhnw4+002340
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMGGhnw4+002352
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMSHhnw4+002352
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMUAhnw4+002352
Zipping up the intermediate file: /var/lib/41000/projects/climateprediction.net/oifs_43r3_00ae_2019110100_123_993_12213812_0_r335119040_97.zip
Uploading the intermediate file: upload_file_97.zip
  08:05:14 STEP 2356 H=2356:00 +CPU= 12.314
  08:05:25 STEP 2357 H=2357:00 +CPU= 11.263
  08:05:42 STEP 2358 H=2358:00 +CPU= 17.153
  08:05:52 STEP 2359 H=2359:00 +CPU=  9.870
  08:06:02 STEP 2360 H=2360:00 +CPU=  9.634
  08:06:16 STEP 2361 H=2361:00 +CPU= 14.335
  08:06:25 STEP 2362 H=2362:00 +CPU=  8.746
  08:06:33 STEP 2363 H=2363:00 +CPU=  8.426
  08:07:18 STEP 2364 H=2364:00 +CPU= 44.663
Moving to projects directory: /var/lib/41000/slots/8/ICMGGhnw4+002364
Moving to projects directory: /var/lib/41000/slots/8/ICMSHhnw4+002364
Moving to projects directory: /var/lib/41000/slots/8/ICMUAhnw4+002364
  08:07:37 STEP 2365 H=2365:00 +CPU= 19.526
  08:08:07 STEP 2366 H=2366:00 +CPU= 29.446
  08:08:46 STEP 2367 H=2367:00 +CPU= 38.841
  08:09:16 STEP 2368 H=2368:00 +CPU= 30.654
  08:09:46 STEP 2369 H=2369:00 +CPU= 29.746
  08:10:42 STEP 2370 H=2370:00 +CPU= 56.179
  08:11:07 STEP 2371 H=2371:00 +CPU= 25.151
  08:11:21 STEP 2372 H=2372:00 +CPU= 13.633
  08:11:40 STEP 2373 H=2373:00 +CPU= 18.950
  08:11:51 STEP 2374 H=2374:00 +CPU= 11.101
  08:12:02 STEP 2375 H=2375:00 +CPU= 10.492
  08:12:20 STEP 2376 H=2376:00 +CPU= 18.124
Moving to projects directory: /var/lib/41000/slots/8/ICMGGhnw4+002376
Moving to projects directory: /var/lib/41000/slots/8/ICMSHhnw4+002376
Moving to projects directory: /var/lib/41000/slots/8/ICMUAhnw4+002376
  08:12:29 STEP 2377 H=2377:00 +CPU=  8.860
  08:12:38 STEP 2378 H=2378:00 +CPU=  9.603
  08:13:24 STEP 2379 H=2379:00 +CPU= 45.405
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMGGhnw4+002364
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMSHhnw4+002364
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMUAhnw4+002364
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMGGhnw4+002376
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMSHhnw4+002376
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMUAhnw4+002376
Zipping up the intermediate file: /var/lib/41000/projects/climateprediction.net/oifs_43r3_00ae_2019110100_123_993_12213812_0_r335119040_98.zip
Uploading the intermediate file: upload_file_98.zip
  08:13:53 STEP 2380 H=2380:00 +CPU= 29.699
  08:14:21 STEP 2381 H=2381:00 +CPU= 27.204
  08:15:04 STEP 2382 H=2382:00 +CPU= 43.764
  08:15:31 STEP 2383 H=2383:00 +CPU= 26.194
  08:16:02 STEP 2384 H=2384:00 +CPU= 31.548
  08:16:40 STEP 2385 H=2385:00 +CPU= 37.747
  08:17:03 STEP 2386 H=2386:00 +CPU= 23.238
  08:17:29 STEP 2387 H=2387:00 +CPU= 25.806
  08:17:50 STEP 2388 H=2388:00 +CPU= 21.330
Moving to projects directory: /var/lib/41000/slots/8/ICMGGhnw4+002388
Moving to projects directory: /var/lib/41000/slots/8/ICMSHhnw4+002388
Moving to projects directory: /var/lib/41000/slots/8/ICMUAhnw4+002388
  08:18:00 STEP 2389 H=2389:00 +CPU= 10.058
  08:18:10 STEP 2390 H=2390:00 +CPU=  9.776
  08:18:25 STEP 2391 H=2391:00 +CPU= 14.552
  08:18:33 STEP 2392 H=2392:00 +CPU=  8.880
  08:18:42 STEP 2393 H=2393:00 +CPU=  8.533
  08:18:56 STEP 2394 H=2394:00 +CPU= 14.192
  08:19:10 STEP 2395 H=2395:00 +CPU= 13.772
  08:19:31 STEP 2396 H=2396:00 +CPU= 21.411
  08:20:14 STEP 2397 H=2397:00 +CPU= 42.822
  08:20:48 STEP 2398 H=2398:00 +CPU= 34.013
  08:21:20 STEP 2399 H=2399:00 +CPU= 31.956
  08:22:05 STEP 2400 H=2400:00 +CPU= 45.108
Moving to projects directory: /var/lib/41000/slots/8/ICMGGhnw4+002400
Moving to projects directory: /var/lib/41000/slots/8/ICMSHhnw4+002400
Moving to projects directory: /var/lib/41000/slots/8/ICMUAhnw4+002400
  08:22:36 STEP 2401 H=2401:00 +CPU= 30.644
  08:23:05 STEP 2402 H=2402:00 +CPU= 29.290
  08:23:51 STEP 2403 H=2403:00 +CPU= 45.946
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMGGhnw4+002388
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMSHhnw4+002388
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMUAhnw4+002388
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMGGhnw4+002400
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMSHhnw4+002400
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMUAhnw4+002400
Zipping up the intermediate file: /var/lib/41000/projects/climateprediction.net/oifs_43r3_00ae_2019110100_123_993_12213812_0_r335119040_99.zip
Uploading the intermediate file: upload_file_99.zip
  08:24:08 STEP 2404 H=2404:00 +CPU= 16.824
  08:24:20 STEP 2405 H=2405:00 +CPU= 11.781
  08:24:37 STEP 2406 H=2406:00 +CPU= 17.304
  08:24:47 STEP 2407 H=2407:00 +CPU=  9.796
  08:24:56 STEP 2408 H=2408:00 +CPU=  9.323
  08:25:10 STEP 2409 H=2409:00 +CPU= 13.997
  08:25:19 STEP 2410 H=2410:00 +CPU=  8.929
  08:25:29 STEP 2411 H=2411:00 +CPU=  9.542
  08:25:56 STEP 2412 H=2412:00 +CPU= 27.674
Moving to projects directory: /var/lib/41000/slots/8/ICMGGhnw4+002412
Moving to projects directory: /var/lib/41000/slots/8/ICMSHhnw4+002412
Moving to projects directory: /var/lib/41000/slots/8/ICMUAhnw4+002412
  08:26:22 STEP 2413 H=2413:00 +CPU= 25.326
  08:26:55 STEP 2414 H=2414:00 +CPU= 32.971
  08:27:38 STEP 2415 H=2415:00 +CPU= 43.213
  08:28:03 STEP 2416 H=2416:00 +CPU= 25.015
  08:28:31 STEP 2417 H=2417:00 +CPU= 28.327
  08:29:16 STEP 2418 H=2418:00 +CPU= 44.370
  08:29:40 STEP 2419 H=2419:00 +CPU= 23.994
  08:30:14 STEP 2420 H=2420:00 +CPU= 34.208
  08:30:36 STEP 2421 H=2421:00 +CPU= 22.308
  08:30:47 STEP 2422 H=2422:00 +CPU= 11.100
  08:30:57 STEP 2423 H=2423:00 +CPU= 10.098
  08:31:14 STEP 2424 H=2424:00 +CPU= 16.816
Moving to projects directory: /var/lib/41000/slots/8/ICMGGhnw4+002424
Moving to projects directory: /var/lib/41000/slots/8/ICMSHhnw4+002424
Moving to projects directory: /var/lib/41000/slots/8/ICMUAhnw4+002424
  08:31:23 STEP 2425 H=2425:00 +CPU=  8.724
  08:31:32 STEP 2426 H=2426:00 +CPU=  8.888
  08:31:46 STEP 2427 H=2427:00 +CPU= 14.671
  08:31:56 STEP 2428 H=2428:00 +CPU=  9.106
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMGGhnw4+002412
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMSHhnw4+002412
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMUAhnw4+002412
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMGGhnw4+002424
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMSHhnw4+002424
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMUAhnw4+002424
Zipping up the intermediate file: /var/lib/41000/projects/climateprediction.net/oifs_43r3_00ae_2019110100_123_993_12213812_0_r335119040_100.zip
Uploading the intermediate file: upload_file_100.zip
  08:32:04 STEP 2429 H=2429:00 +CPU=  8.405
  08:32:31 STEP 2430 H=2430:00 +CPU= 27.486
  08:32:59 STEP 2431 H=2431:00 +CPU= 27.897
  08:33:27 STEP 2432 H=2432:00 +CPU= 27.641
  08:34:11 STEP 2433 H=2433:00 +CPU= 43.697
  08:34:38 STEP 2434 H=2434:00 +CPU= 27.308
  08:35:07 STEP 2435 H=2435:00 +CPU= 28.933
  08:35:46 STEP 2436 H=2436:00 +CPU= 38.710
Moving to projects directory: /var/lib/41000/slots/8/ICMGGhnw4+002436
Moving to projects directory: /var/lib/41000/slots/8/ICMSHhnw4+002436
Moving to projects directory: /var/lib/41000/slots/8/ICMUAhnw4+002436
  08:36:15 STEP 2437 H=2437:00 +CPU= 29.459
  08:36:47 STEP 2438 H=2438:00 +CPU= 31.597
  08:37:15 STEP 2439 H=2439:00 +CPU= 28.127
  08:37:26 STEP 2440 H=2440:00 +CPU= 10.693
  08:37:35 STEP 2441 H=2441:00 +CPU=  9.713
  08:37:51 STEP 2442 H=2442:00 +CPU= 15.299
  08:37:59 STEP 2443 H=2443:00 +CPU=  8.829
  08:38:08 STEP 2444 H=2444:00 +CPU=  9.077
  08:38:23 STEP 2445 H=2445:00 +CPU= 14.162
  08:38:31 STEP 2446 H=2446:00 +CPU=  8.791
  08:38:45 STEP 2447 H=2447:00 +CPU= 13.803
  08:39:47 STEP 2448 H=2448:00 +CPU= 62.199
Moving to projects directory: /var/lib/41000/slots/8/ICMGGhnw4+002448
Moving to projects directory: /var/lib/41000/slots/8/ICMSHhnw4+002448
Moving to projects directory: /var/lib/41000/slots/8/ICMUAhnw4+002448
  08:40:18 STEP 2449 H=2449:00 +CPU= 30.811
  08:40:40 STEP 2450 H=2450:00 +CPU= 21.291
  08:41:25 STEP 2451 H=2451:00 +CPU= 45.596
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMGGhnw4+002436
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMSHhnw4+002436
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMUAhnw4+002436
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMGGhnw4+002448
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMSHhnw4+002448
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMUAhnw4+002448
Zipping up the intermediate file: /var/lib/41000/projects/climateprediction.net/oifs_43r3_00ae_2019110100_123_993_12213812_0_r335119040_101.zip
Uploading the intermediate file: upload_file_101.zip
Uploading trickle at timestep: 8823600
  08:41:52 STEP 2452 H=2452:00 +CPU= 26.527
  08:42:16 STEP 2453 H=2453:00 +CPU= 24.144
  08:43:00 STEP 2454 H=2454:00 +CPU= 43.821
  08:43:28 STEP 2455 H=2455:00 +CPU= 28.791
  08:43:43 STEP 2456 H=2456:00 +CPU= 14.824
  08:44:01 STEP 2457 H=2457:00 +CPU= 17.327
  08:44:12 STEP 2458 H=2458:00 +CPU= 10.964
  08:44:21 STEP 2459 H=2459:00 +CPU=  9.096
  08:44:36 STEP 2460 H=2460:00 +CPU= 15.573
Moving to projects directory: /var/lib/41000/slots/8/ICMGGhnw4+002460
Moving to projects directory: /var/lib/41000/slots/8/ICMSHhnw4+002460
Moving to projects directory: /var/lib/41000/slots/8/ICMUAhnw4+002460
  08:44:45 STEP 2461 H=2461:00 +CPU=  8.566
  08:45:05 STEP 2462 H=2462:00 +CPU= 20.043
  08:45:48 STEP 2463 H=2463:00 +CPU= 43.065
  08:46:17 STEP 2464 H=2464:00 +CPU= 29.216
  08:46:39 STEP 2465 H=2465:00 +CPU= 21.858
  08:47:33 STEP 2466 H=2466:00 +CPU= 54.417
  08:48:08 STEP 2467 H=2467:00 +CPU= 34.897
  08:48:40 STEP 2468 H=2468:00 +CPU= 31.964
  08:49:12 STEP 2469 H=2469:00 +CPU= 31.649
  08:49:29 STEP 2470 H=2470:00 +CPU= 17.068
  08:49:41 STEP 2471 H=2471:00 +CPU= 11.793
  08:50:01 STEP 2472 H=2472:00 +CPU= 20.594
Moving to projects directory: /var/lib/41000/slots/8/ICMGGhnw4+002472
Moving to projects directory: /var/lib/41000/slots/8/ICMSHhnw4+002472
Moving to projects directory: /var/lib/41000/slots/8/ICMUAhnw4+002472
  08:50:11 STEP 2473 H=2473:00 +CPU=  9.781
  08:50:21 STEP 2474 H=2474:00 +CPU=  9.724
  08:50:36 STEP 2475 H=2475:00 +CPU= 14.650
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMGGhnw4+002460
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMSHhnw4+002460
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMUAhnw4+002460
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMGGhnw4+002472
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMSHhnw4+002472
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMUAhnw4+002472
Zipping up the intermediate file: /var/lib/41000/projects/climateprediction.net/oifs_43r3_00ae_2019110100_123_993_12213812_0_r335119040_102.zip
Uploading the intermediate file: upload_file_102.zip
  08:50:45 STEP 2476 H=2476:00 +CPU=  8.981
  08:50:53 STEP 2477 H=2477:00 +CPU=  8.619
  08:51:18 STEP 2478 H=2478:00 +CPU= 25.185
  08:51:43 STEP 2479 H=2479:00 +CPU= 24.880
  08:52:17 STEP 2480 H=2480:00 +CPU= 33.689
  08:53:02 STEP 2481 H=2481:00 +CPU= 45.205
  08:53:26 STEP 2482 H=2482:00 +CPU= 23.589
  08:53:52 STEP 2483 H=2483:00 +CPU= 25.948
  08:54:43 STEP 2484 H=2484:00 +CPU= 50.948
Moving to projects directory: /var/lib/41000/slots/8/ICMGGhnw4+002484
Moving to projects directory: /var/lib/41000/slots/8/ICMSHhnw4+002484
Moving to projects directory: /var/lib/41000/slots/8/ICMUAhnw4+002484
  08:55:17 STEP 2485 H=2485:00 +CPU= 34.384
  08:55:40 STEP 2486 H=2486:00 +CPU= 22.823
  08:55:58 STEP 2487 H=2487:00 +CPU= 18.581
  08:56:09 STEP 2488 H=2488:00 +CPU= 10.978
  08:56:19 STEP 2489 H=2489:00 +CPU=  9.700
  08:56:34 STEP 2490 H=2490:00 +CPU= 15.372
  08:56:43 STEP 2491 H=2491:00 +CPU=  8.812
  08:56:52 STEP 2492 H=2492:00 +CPU=  9.023
  08:57:06 STEP 2493 H=2493:00 +CPU= 13.978
  08:57:16 STEP 2494 H=2494:00 +CPU=  9.278
  08:57:24 STEP 2495 H=2495:00 +CPU=  8.894
  08:58:03 STEP 2496 H=2496:00 +CPU= 38.420
Moving to projects directory: /var/lib/41000/slots/8/ICMGGhnw4+002496
Moving to projects directory: /var/lib/41000/slots/8/ICMSHhnw4+002496
Moving to projects directory: /var/lib/41000/slots/8/ICMUAhnw4+002496
  08:58:27 STEP 2497 H=2497:00 +CPU= 24.261
  08:59:03 STEP 2498 H=2498:00 +CPU= 36.044
  08:59:41 STEP 2499 H=2499:00 +CPU= 37.865
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMGGhnw4+002484
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMSHhnw4+002484
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMUAhnw4+002484
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMGGhnw4+002496
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMSHhnw4+002496
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMUAhnw4+002496
Zipping up the intermediate file: /var/lib/41000/projects/climateprediction.net/oifs_43r3_00ae_2019110100_123_993_12213812_0_r335119040_103.zip
Uploading the intermediate file: upload_file_103.zip
  09:00:08 STEP 2500 H=2500:00 +CPU= 27.355
  09:00:34 STEP 2501 H=2501:00 +CPU= 25.519
  09:01:18 STEP 2502 H=2502:00 +CPU= 44.388
  09:01:53 STEP 2503 H=2503:00 +CPU= 35.159
  09:02:08 STEP 2504 H=2504:00 +CPU= 14.121
  09:02:28 STEP 2505 H=2505:00 +CPU= 20.497
  09:02:40 STEP 2506 H=2506:00 +CPU= 12.237
  09:02:51 STEP 2507 H=2507:00 +CPU= 11.104
  09:03:09 STEP 2508 H=2508:00 +CPU= 17.771
Moving to projects directory: /var/lib/41000/slots/8/ICMGGhnw4+002508
Moving to projects directory: /var/lib/41000/slots/8/ICMSHhnw4+002508
Moving to projects directory: /var/lib/41000/slots/8/ICMUAhnw4+002508
  09:03:19 STEP 2509 H=2509:00 +CPU=  9.337
  09:03:28 STEP 2510 H=2510:00 +CPU=  9.061
  09:03:43 STEP 2511 H=2511:00 +CPU= 15.066
  09:04:03 STEP 2512 H=2512:00 +CPU= 20.355
  09:04:33 STEP 2513 H=2513:00 +CPU= 30.406
  09:05:09 STEP 2514 H=2514:00 +CPU= 35.637
  09:05:32 STEP 2515 H=2515:00 +CPU= 22.920
  09:05:58 STEP 2516 H=2516:00 +CPU= 26.407
  09:06:47 STEP 2517 H=2517:00 +CPU= 48.630
  09:07:15 STEP 2518 H=2518:00 +CPU= 27.723
  09:07:40 STEP 2519 H=2519:00 +CPU= 25.706
  09:08:16 STEP 2520 H=2520:00 +CPU= 35.442
Moving to projects directory: /var/lib/41000/slots/8/ICMGGhnw4+002520
Moving to projects directory: /var/lib/41000/slots/8/ICMSHhnw4+002520
Moving to projects directory: /var/lib/41000/slots/8/ICMUAhnw4+002520
  09:08:27 STEP 2521 H=2521:00 +CPU= 11.083
  09:08:38 STEP 2522 H=2522:00 +CPU= 10.957
  09:08:55 STEP 2523 H=2523:00 +CPU= 16.561
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMGGhnw4+002508
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMSHhnw4+002508
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMUAhnw4+002508
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMGGhnw4+002520
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMSHhnw4+002520
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMUAhnw4+002520
Zipping up the intermediate file: /var/lib/41000/projects/climateprediction.net/oifs_43r3_00ae_2019110100_123_993_12213812_0_r335119040_104.zip
Uploading the intermediate file: upload_file_104.zip
  09:09:04 STEP 2524 H=2524:00 +CPU=  9.948
  09:09:14 STEP 2525 H=2525:00 +CPU=  9.303
  09:09:29 STEP 2526 H=2526:00 +CPU= 14.932
  09:09:38 STEP 2527 H=2527:00 +CPU=  8.882
  09:09:47 STEP 2528 H=2528:00 +CPU=  8.920
  09:10:01 STEP 2529 H=2529:00 +CPU= 13.983
  09:10:09 STEP 2530 H=2530:00 +CPU=  8.861
  09:10:18 STEP 2531 H=2531:00 +CPU=  8.522
  09:10:53 STEP 2532 H=2532:00 +CPU= 34.596
Moving to projects directory: /var/lib/41000/slots/8/ICMGGhnw4+002532
Moving to projects directory: /var/lib/41000/slots/8/ICMSHhnw4+002532
Moving to projects directory: /var/lib/41000/slots/8/ICMUAhnw4+002532
  09:11:22 STEP 2533 H=2533:00 +CPU= 29.663
  09:11:52 STEP 2534 H=2534:00 +CPU= 30.103
  09:12:31 STEP 2535 H=2535:00 +CPU= 38.394
  09:12:55 STEP 2536 H=2536:00 +CPU= 24.204
  09:13:20 STEP 2537 H=2537:00 +CPU= 25.302
  09:14:02 STEP 2538 H=2538:00 +CPU= 41.928
  09:14:26 STEP 2539 H=2539:00 +CPU= 23.422
  09:14:48 STEP 2540 H=2540:00 +CPU= 22.155
  09:15:08 STEP 2541 H=2541:00 +CPU= 19.986
  09:15:21 STEP 2542 H=2542:00 +CPU= 12.969
  09:15:32 STEP 2543 H=2543:00 +CPU= 11.616
  09:15:51 STEP 2544 H=2544:00 +CPU= 18.973
Moving to projects directory: /var/lib/41000/slots/8/ICMGGhnw4+002544
Moving to projects directory: /var/lib/41000/slots/8/ICMSHhnw4+002544
Moving to projects directory: /var/lib/41000/slots/8/ICMUAhnw4+002544
  09:16:01 STEP 2545 H=2545:00 +CPU=  9.423
  09:16:10 STEP 2546 H=2546:00 +CPU=  9.387
  09:16:26 STEP 2547 H=2547:00 +CPU= 15.639
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMGGhnw4+002532
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMSHhnw4+002532
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMUAhnw4+002532
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMGGhnw4+002544
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMSHhnw4+002544
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMUAhnw4+002544
Zipping up the intermediate file: /var/lib/41000/projects/climateprediction.net/oifs_43r3_00ae_2019110100_123_993_12213812_0_r335119040_105.zip
Uploading the intermediate file: upload_file_105.zip
  09:16:35 STEP 2548 H=2548:00 +CPU=  9.014
  09:16:43 STEP 2549 H=2549:00 +CPU=  8.530
  09:17:31 STEP 2550 H=2550:00 +CPU= 48.157
  09:18:02 STEP 2551 H=2551:00 +CPU= 30.523
  09:18:27 STEP 2552 H=2552:00 +CPU= 24.777
  09:19:14 STEP 2553 H=2553:00 +CPU= 47.018
  09:19:46 STEP 2554 H=2554:00 +CPU= 32.025
  09:20:08 STEP 2555 H=2555:00 +CPU= 22.491
  09:21:01 STEP 2556 H=2556:00 +CPU= 52.891
Moving to projects directory: /var/lib/41000/slots/8/ICMGGhnw4+002556
Moving to projects directory: /var/lib/41000/slots/8/ICMSHhnw4+002556
Moving to projects directory: /var/lib/41000/slots/8/ICMUAhnw4+002556
  09:21:23 STEP 2557 H=2557:00 +CPU= 21.755
  09:21:38 STEP 2558 H=2558:00 +CPU= 14.947
  09:21:55 STEP 2559 H=2559:00 +CPU= 17.521
  09:22:06 STEP 2560 H=2560:00 +CPU= 10.448
  09:22:15 STEP 2561 H=2561:00 +CPU=  9.442
  09:22:30 STEP 2562 H=2562:00 +CPU= 15.168
  09:22:39 STEP 2563 H=2563:00 +CPU=  8.919
  09:22:49 STEP 2564 H=2564:00 +CPU=  9.594
  09:23:03 STEP 2565 H=2565:00 +CPU= 13.846
  09:23:12 STEP 2566 H=2566:00 +CPU=  8.728
  09:23:36 STEP 2567 H=2567:00 +CPU= 24.825
  09:24:31 STEP 2568 H=2568:00 +CPU= 54.248
Moving to projects directory: /var/lib/41000/slots/8/ICMGGhnw4+002568
Moving to projects directory: /var/lib/41000/slots/8/ICMSHhnw4+002568
Moving to projects directory: /var/lib/41000/slots/8/ICMUAhnw4+002568
  09:25:01 STEP 2569 H=2569:00 +CPU= 30.003
  09:25:24 STEP 2570 H=2570:00 +CPU= 23.063
  09:26:01 STEP 2571 H=2571:00 +CPU= 37.501
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMGGhnw4+002556
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMSHhnw4+002556
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMUAhnw4+002556
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMGGhnw4+002568
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMSHhnw4+002568
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMUAhnw4+002568
Zipping up the intermediate file: /var/lib/41000/projects/climateprediction.net/oifs_43r3_00ae_2019110100_123_993_12213812_0_r335119040_106.zip
Uploading the intermediate file: upload_file_106.zip
  09:26:29 STEP 2572 H=2572:00 +CPU= 27.310
  09:26:52 STEP 2573 H=2573:00 +CPU= 23.320
  09:27:27 STEP 2574 H=2574:00 +CPU= 35.017
  09:27:54 STEP 2575 H=2575:00 +CPU= 26.739
  09:28:08 STEP 2576 H=2576:00 +CPU= 14.319
  09:28:26 STEP 2577 H=2577:00 +CPU= 18.479
 SMILAG TRAJECTORY UNDERGROUND      1 TIMES.
  POINT=            5  MAX ETADOT VERTICAL VEL.=   2.278097041795184E-007
  LON  =    21.0937500000000       degrees
  LAT  =    44.2987941306940       degrees
  09:28:38 STEP 2578 H=2578:00 +CPU= 11.130
  09:28:47 STEP 2579 H=2579:00 +CPU=  9.879
  09:29:04 STEP 2580 H=2580:00 +CPU= 16.520
Moving to projects directory: /var/lib/41000/slots/8/ICMGGhnw4+002580
Moving to projects directory: /var/lib/41000/slots/8/ICMSHhnw4+002580
Moving to projects directory: /var/lib/41000/slots/8/ICMUAhnw4+002580
  09:29:13 STEP 2581 H=2581:00 +CPU=  8.707
  09:29:22 STEP 2582 H=2582:00 +CPU=  8.814
  09:29:35 STEP 2583 H=2583:00 +CPU= 13.945
 SMILAG TRAJECTORY UNDERGROUND      1 TIMES.
  POINT=            6  MAX ETADOT VERTICAL VEL.=   1.618811440087748E-007
  LON  =    142.500000000000       degrees
  LAT  =    38.6913661222016       degrees
  09:29:44 STEP 2584 H=2584:00 +CPU=  8.826
  09:29:54 STEP 2585 H=2585:00 +CPU=  9.665
  09:30:08 STEP 2586 H=2586:00 +CPU= 14.207
  09:30:17 STEP 2587 H=2587:00 +CPU=  8.384
  09:30:25 STEP 2588 H=2588:00 +CPU=  8.604
  09:30:39 STEP 2589 H=2589:00 +CPU= 13.733
  09:30:47 STEP 2590 H=2590:00 +CPU=  8.612
  09:30:56 STEP 2591 H=2591:00 +CPU=  8.610
  09:31:13 STEP 2592 H=2592:00 +CPU= 16.886
Moving to projects directory: /var/lib/41000/slots/8/ICMGGhnw4+002592
Moving to projects directory: /var/lib/41000/slots/8/ICMSHhnw4+002592
Moving to projects directory: /var/lib/41000/slots/8/ICMUAhnw4+002592
  09:31:22 STEP 2593 H=2593:00 +CPU=  8.686
  09:31:30 STEP 2594 H=2594:00 +CPU=  8.659
  09:31:44 STEP 2595 H=2595:00 +CPU= 13.738
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMGGhnw4+002580
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMSHhnw4+002580
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMUAhnw4+002580
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMGGhnw4+002592
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMSHhnw4+002592
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMUAhnw4+002592
Zipping up the intermediate file: /var/lib/41000/projects/climateprediction.net/oifs_43r3_00ae_2019110100_123_993_12213812_0_r335119040_107.zip
Uploading the intermediate file: upload_file_107.zip
  09:31:57 STEP 2596 H=2596:00 +CPU= 13.301
  09:32:24 STEP 2597 H=2597:00 +CPU= 26.852
  09:33:04 STEP 2598 H=2598:00 +CPU= 39.690
  09:33:27 STEP 2599 H=2599:00 +CPU= 23.301
  09:33:50 STEP 2600 H=2600:00 +CPU= 22.386
  09:34:34 STEP 2601 H=2601:00 +CPU= 44.473
  09:35:02 STEP 2602 H=2602:00 +CPU= 27.777
  09:35:27 STEP 2603 H=2603:00 +CPU= 25.234
  09:36:18 STEP 2604 H=2604:00 +CPU= 51.146
Moving to projects directory: /var/lib/41000/slots/8/ICMGGhnw4+002604
Moving to projects directory: /var/lib/41000/slots/8/ICMSHhnw4+002604
Moving to projects directory: /var/lib/41000/slots/8/ICMUAhnw4+002604
  09:36:34 STEP 2605 H=2605:00 +CPU= 16.077
  09:36:47 STEP 2606 H=2606:00 +CPU= 12.752
  09:37:05 STEP 2607 H=2607:00 +CPU= 17.443
  09:37:15 STEP 2608 H=2608:00 +CPU= 10.205
  09:37:24 STEP 2609 H=2609:00 +CPU=  9.521
  09:37:40 STEP 2610 H=2610:00 +CPU= 15.571
  09:37:49 STEP 2611 H=2611:00 +CPU=  8.809
  09:37:58 STEP 2612 H=2612:00 +CPU=  8.820
  09:38:11 STEP 2613 H=2613:00 +CPU= 13.831
  09:38:20 STEP 2614 H=2614:00 +CPU=  8.773
  09:38:39 STEP 2615 H=2615:00 +CPU= 19.338
  09:39:25 STEP 2616 H=2616:00 +CPU= 45.809
Moving to projects directory: /var/lib/41000/slots/8/ICMGGhnw4+002616
Moving to projects directory: /var/lib/41000/slots/8/ICMSHhnw4+002616
Moving to projects directory: /var/lib/41000/slots/8/ICMUAhnw4+002616
  09:39:40 STEP 2617 H=2617:00 +CPU= 14.575
  09:40:04 STEP 2618 H=2618:00 +CPU= 23.699
  09:40:52 STEP 2619 H=2619:00 +CPU= 48.360
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMGGhnw4+002604
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMSHhnw4+002604
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMUAhnw4+002604
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMGGhnw4+002616
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMSHhnw4+002616
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMUAhnw4+002616
Zipping up the intermediate file: /var/lib/41000/projects/climateprediction.net/oifs_43r3_00ae_2019110100_123_993_12213812_0_r335119040_108.zip
Uploading the intermediate file: upload_file_108.zip
  09:41:21 STEP 2620 H=2620:00 +CPU= 28.841
  09:41:40 STEP 2621 H=2621:00 +CPU= 19.464
  09:42:18 STEP 2622 H=2622:00 +CPU= 37.459
  09:42:48 STEP 2623 H=2623:00 +CPU= 30.363
  09:43:08 STEP 2624 H=2624:00 +CPU= 19.785
  09:43:26 STEP 2625 H=2625:00 +CPU= 18.329
  09:43:37 STEP 2626 H=2626:00 +CPU= 10.681
  09:43:47 STEP 2627 H=2627:00 +CPU= 10.228
  09:44:04 STEP 2628 H=2628:00 +CPU= 17.352
Moving to projects directory: /var/lib/41000/slots/8/ICMGGhnw4+002628
Moving to projects directory: /var/lib/41000/slots/8/ICMSHhnw4+002628
Moving to projects directory: /var/lib/41000/slots/8/ICMUAhnw4+002628
  09:44:14 STEP 2629 H=2629:00 +CPU=  9.107
  09:44:23 STEP 2630 H=2630:00 +CPU=  9.896
  09:44:37 STEP 2631 H=2631:00 +CPU= 14.022
  09:44:46 STEP 2632 H=2632:00 +CPU=  8.880
  09:45:03 STEP 2633 H=2633:00 +CPU= 16.490
  09:45:50 STEP 2634 H=2634:00 +CPU= 47.011
  09:46:12 STEP 2635 H=2635:00 +CPU= 21.797
  09:46:48 STEP 2636 H=2636:00 +CPU= 35.988
  09:47:24 STEP 2637 H=2637:00 +CPU= 36.422
  09:47:53 STEP 2638 H=2638:00 +CPU= 29.348
  09:48:20 STEP 2639 H=2639:00 +CPU= 26.900
  09:49:05 STEP 2640 H=2640:00 +CPU= 45.016
Moving to projects directory: /var/lib/41000/slots/8/ICMGGhnw4+002640
Moving to projects directory: /var/lib/41000/slots/8/ICMSHhnw4+002640
Moving to projects directory: /var/lib/41000/slots/8/ICMUAhnw4+002640
  09:49:30 STEP 2641 H=2641:00 +CPU= 24.905
  09:49:48 STEP 2642 H=2642:00 +CPU= 17.531
  09:50:06 STEP 2643 H=2643:00 +CPU= 18.662
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMGGhnw4+002628
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMSHhnw4+002628
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMUAhnw4+002628
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMGGhnw4+002640
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMSHhnw4+002640
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMUAhnw4+002640
Zipping up the intermediate file: /var/lib/41000/projects/climateprediction.net/oifs_43r3_00ae_2019110100_123_993_12213812_0_r335119040_109.zip
Uploading the intermediate file: upload_file_109.zip
  09:50:17 STEP 2644 H=2644:00 +CPU= 10.506
  09:50:26 STEP 2645 H=2645:00 +CPU=  9.362
  09:50:41 STEP 2646 H=2646:00 +CPU= 14.847
  09:50:50 STEP 2647 H=2647:00 +CPU=  8.731
  09:50:59 STEP 2648 H=2648:00 +CPU=  8.881
  09:51:13 STEP 2649 H=2649:00 +CPU= 13.966
  09:51:21 STEP 2650 H=2650:00 +CPU=  8.406
  09:51:29 STEP 2651 H=2651:00 +CPU=  8.131
  09:51:44 STEP 2652 H=2652:00 +CPU= 14.464
Moving to projects directory: /var/lib/41000/slots/8/ICMGGhnw4+002652
Moving to projects directory: /var/lib/41000/slots/8/ICMSHhnw4+002652
Moving to projects directory: /var/lib/41000/slots/8/ICMUAhnw4+002652
  09:51:52 STEP 2653 H=2653:00 +CPU=  8.132
  09:52:00 STEP 2654 H=2654:00 +CPU=  8.399
  09:52:14 STEP 2655 H=2655:00 +CPU= 13.295
  09:52:22 STEP 2656 H=2656:00 +CPU=  8.503
  09:52:30 STEP 2657 H=2657:00 +CPU=  8.172
  09:52:44 STEP 2658 H=2658:00 +CPU= 13.664
  09:52:52 STEP 2659 H=2659:00 +CPU=  8.087
  09:53:00 STEP 2660 H=2660:00 +CPU=  8.339
  09:53:15 STEP 2661 H=2661:00 +CPU= 14.129
  09:53:23 STEP 2662 H=2662:00 +CPU=  8.377
  09:53:31 STEP 2663 H=2663:00 +CPU=  8.547
  09:53:47 STEP 2664 H=2664:00 +CPU= 15.459
Moving to projects directory: /var/lib/41000/slots/8/ICMGGhnw4+002664
Moving to projects directory: /var/lib/41000/slots/8/ICMSHhnw4+002664
Moving to projects directory: /var/lib/41000/slots/8/ICMUAhnw4+002664
  09:53:55 STEP 2665 H=2665:00 +CPU=  8.205
  09:54:03 STEP 2666 H=2666:00 +CPU=  8.346
  09:54:17 STEP 2667 H=2667:00 +CPU= 13.413
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMGGhnw4+002652
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMSHhnw4+002652
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMUAhnw4+002652
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMGGhnw4+002664
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMSHhnw4+002664
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMUAhnw4+002664
Zipping up the intermediate file: /var/lib/41000/projects/climateprediction.net/oifs_43r3_00ae_2019110100_123_993_12213812_0_r335119040_110.zip
Uploading the intermediate file: upload_file_110.zip
  09:54:25 STEP 2668 H=2668:00 +CPU=  8.346
  09:54:33 STEP 2669 H=2669:00 +CPU=  8.142
  09:54:47 STEP 2670 H=2670:00 +CPU= 13.611
  09:54:55 STEP 2671 H=2671:00 +CPU=  8.069
  09:55:03 STEP 2672 H=2672:00 +CPU=  8.371
  09:55:17 STEP 2673 H=2673:00 +CPU= 13.395
  09:55:25 STEP 2674 H=2674:00 +CPU=  8.354
  09:55:33 STEP 2675 H=2675:00 +CPU=  8.156
  09:55:56 STEP 2676 H=2676:00 +CPU= 22.741
Moving to projects directory: /var/lib/41000/slots/8/ICMGGhnw4+002676
Moving to projects directory: /var/lib/41000/slots/8/ICMSHhnw4+002676
Moving to projects directory: /var/lib/41000/slots/8/ICMUAhnw4+002676
  09:56:10 STEP 2677 H=2677:00 +CPU= 14.171
  09:56:25 STEP 2678 H=2678:00 +CPU= 14.358
  09:56:47 STEP 2679 H=2679:00 +CPU= 22.509
  09:57:02 STEP 2680 H=2680:00 +CPU= 14.682
  09:57:16 STEP 2681 H=2681:00 +CPU= 13.864
  09:57:38 STEP 2682 H=2682:00 +CPU= 22.397
  09:57:52 STEP 2683 H=2683:00 +CPU= 13.916
  09:58:06 STEP 2684 H=2684:00 +CPU= 14.218
  09:58:29 STEP 2685 H=2685:00 +CPU= 22.354
  09:58:43 STEP 2686 H=2686:00 +CPU= 14.442
  09:58:57 STEP 2687 H=2687:00 +CPU= 14.412
  09:59:23 STEP 2688 H=2688:00 +CPU= 25.450
Moving to projects directory: /var/lib/41000/slots/8/ICMGGhnw4+002688
Moving to projects directory: /var/lib/41000/slots/8/ICMSHhnw4+002688
Moving to projects directory: /var/lib/41000/slots/8/ICMUAhnw4+002688
  09:59:37 STEP 2689 H=2689:00 +CPU= 14.033
  09:59:52 STEP 2690 H=2690:00 +CPU= 15.038
  10:00:14 STEP 2691 H=2691:00 +CPU= 22.422
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMGGhnw4+002676
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMSHhnw4+002676
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMUAhnw4+002676
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMGGhnw4+002688
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMSHhnw4+002688
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMUAhnw4+002688
Zipping up the intermediate file: /var/lib/41000/projects/climateprediction.net/oifs_43r3_00ae_2019110100_123_993_12213812_0_r335119040_111.zip
Uploading the intermediate file: upload_file_111.zip
  10:00:29 STEP 2692 H=2692:00 +CPU= 14.453
  10:00:43 STEP 2693 H=2693:00 +CPU= 13.965
Uploading trickle at timestep: 9687600
  10:01:05 STEP 2694 H=2694:00 +CPU= 22.643
  10:01:19 STEP 2695 H=2695:00 +CPU= 13.865
  10:01:33 STEP 2696 H=2696:00 +CPU= 14.127
  10:01:56 STEP 2697 H=2697:00 +CPU= 22.353
  10:02:10 STEP 2698 H=2698:00 +CPU= 14.098
  10:02:24 STEP 2699 H=2699:00 +CPU= 14.003
  10:02:48 STEP 2700 H=2700:00 +CPU= 24.178
Moving to projects directory: /var/lib/41000/slots/8/ICMGGhnw4+002700
Moving to projects directory: /var/lib/41000/slots/8/ICMSHhnw4+002700
Moving to projects directory: /var/lib/41000/slots/8/ICMUAhnw4+002700
  10:03:03 STEP 2701 H=2701:00 +CPU= 14.557
  10:03:17 STEP 2702 H=2702:00 +CPU= 14.374
  10:03:39 STEP 2703 H=2703:00 +CPU= 22.218
  10:03:53 STEP 2704 H=2704:00 +CPU= 14.222
  10:04:07 STEP 2705 H=2705:00 +CPU= 13.664
  10:04:30 STEP 2706 H=2706:00 +CPU= 22.466
  10:04:43 STEP 2707 H=2707:00 +CPU= 13.819
  10:04:58 STEP 2708 H=2708:00 +CPU= 14.315
  10:05:20 STEP 2709 H=2709:00 +CPU= 22.019
  10:05:34 STEP 2710 H=2710:00 +CPU= 14.029
  10:05:48 STEP 2711 H=2711:00 +CPU= 14.244
  10:06:13 STEP 2712 H=2712:00 +CPU= 25.236
Moving to projects directory: /var/lib/41000/slots/8/ICMGGhnw4+002712
Moving to projects directory: /var/lib/41000/slots/8/ICMSHhnw4+002712
Moving to projects directory: /var/lib/41000/slots/8/ICMUAhnw4+002712
  10:06:27 STEP 2713 H=2713:00 +CPU= 14.016
  10:06:42 STEP 2714 H=2714:00 +CPU= 14.945
  10:07:05 STEP 2715 H=2715:00 +CPU= 22.813
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMGGhnw4+002700
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMSHhnw4+002700
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMUAhnw4+002700
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMGGhnw4+002712
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMSHhnw4+002712
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMUAhnw4+002712
Zipping up the intermediate file: /var/lib/41000/projects/climateprediction.net/oifs_43r3_00ae_2019110100_123_993_12213812_0_r335119040_112.zip
Uploading the intermediate file: upload_file_112.zip
  10:07:19 STEP 2716 H=2716:00 +CPU= 14.116
  10:07:33 STEP 2717 H=2717:00 +CPU= 14.058
  10:07:56 STEP 2718 H=2718:00 +CPU= 22.730
  10:08:10 STEP 2719 H=2719:00 +CPU= 14.078
  10:08:25 STEP 2720 H=2720:00 +CPU= 14.734
  10:08:47 STEP 2721 H=2721:00 +CPU= 22.519
  10:09:02 STEP 2722 H=2722:00 +CPU= 14.248
  10:09:16 STEP 2723 H=2723:00 +CPU= 14.134
  10:09:40 STEP 2724 H=2724:00 +CPU= 24.340
Moving to projects directory: /var/lib/41000/slots/8/ICMGGhnw4+002724
Moving to projects directory: /var/lib/41000/slots/8/ICMSHhnw4+002724
Moving to projects directory: /var/lib/41000/slots/8/ICMUAhnw4+002724
  10:09:54 STEP 2725 H=2725:00 +CPU= 13.877
  10:10:08 STEP 2726 H=2726:00 +CPU= 14.261
  10:10:30 STEP 2727 H=2727:00 +CPU= 22.115
  10:10:45 STEP 2728 H=2728:00 +CPU= 14.641
  10:10:59 STEP 2729 H=2729:00 +CPU= 14.296
  10:11:22 STEP 2730 H=2730:00 +CPU= 22.470
  10:11:36 STEP 2731 H=2731:00 +CPU= 13.894
  10:11:50 STEP 2732 H=2732:00 +CPU= 14.507
  10:12:12 STEP 2733 H=2733:00 +CPU= 22.276
  10:12:27 STEP 2734 H=2734:00 +CPU= 15.001
  10:12:42 STEP 2735 H=2735:00 +CPU= 14.545
  10:13:07 STEP 2736 H=2736:00 +CPU= 25.231
Moving to projects directory: /var/lib/41000/slots/8/ICMGGhnw4+002736
Moving to projects directory: /var/lib/41000/slots/8/ICMSHhnw4+002736
Moving to projects directory: /var/lib/41000/slots/8/ICMUAhnw4+002736
  10:13:21 STEP 2737 H=2737:00 +CPU= 13.761
  10:13:36 STEP 2738 H=2738:00 +CPU= 14.833
  10:13:58 STEP 2739 H=2739:00 +CPU= 22.258
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMGGhnw4+002724
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMSHhnw4+002724
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMUAhnw4+002724
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMGGhnw4+002736
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMSHhnw4+002736
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMUAhnw4+002736
Zipping up the intermediate file: /var/lib/41000/projects/climateprediction.net/oifs_43r3_00ae_2019110100_123_993_12213812_0_r335119040_113.zip
Uploading the intermediate file: upload_file_113.zip
  10:14:12 STEP 2740 H=2740:00 +CPU= 14.317
  10:14:26 STEP 2741 H=2741:00 +CPU= 13.914
  10:14:49 STEP 2742 H=2742:00 +CPU= 22.819
  10:15:03 STEP 2743 H=2743:00 +CPU= 14.173
  10:15:18 STEP 2744 H=2744:00 +CPU= 14.318
  10:15:40 STEP 2745 H=2745:00 +CPU= 22.149
  10:15:54 STEP 2746 H=2746:00 +CPU= 14.478
  10:16:08 STEP 2747 H=2747:00 +CPU= 13.888
  10:16:32 STEP 2748 H=2748:00 +CPU= 23.907
Moving to projects directory: /var/lib/41000/slots/8/ICMGGhnw4+002748
Moving to projects directory: /var/lib/41000/slots/8/ICMSHhnw4+002748
Moving to projects directory: /var/lib/41000/slots/8/ICMUAhnw4+002748
  10:16:46 STEP 2749 H=2749:00 +CPU= 13.950
  10:17:00 STEP 2750 H=2750:00 +CPU= 14.287
  10:17:23 STEP 2751 H=2751:00 +CPU= 22.388
  10:17:38 STEP 2752 H=2752:00 +CPU= 15.341
  10:17:52 STEP 2753 H=2753:00 +CPU= 13.920
  10:18:14 STEP 2754 H=2754:00 +CPU= 22.254
  10:18:28 STEP 2755 H=2755:00 +CPU= 14.080
  10:18:42 STEP 2756 H=2756:00 +CPU= 14.095
  10:19:05 STEP 2757 H=2757:00 +CPU= 22.246
  10:19:19 STEP 2758 H=2758:00 +CPU= 14.291
  10:19:33 STEP 2759 H=2759:00 +CPU= 14.248
  10:19:59 STEP 2760 H=2760:00 +CPU= 25.569
Moving to projects directory: /var/lib/41000/slots/8/ICMGGhnw4+002760
Moving to projects directory: /var/lib/41000/slots/8/ICMSHhnw4+002760
Moving to projects directory: /var/lib/41000/slots/8/ICMUAhnw4+002760
  10:20:13 STEP 2761 H=2761:00 +CPU= 14.005
  10:20:27 STEP 2762 H=2762:00 +CPU= 14.146
  10:20:49 STEP 2763 H=2763:00 +CPU= 22.108
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMGGhnw4+002748
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMSHhnw4+002748
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMUAhnw4+002748
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMGGhnw4+002760
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMSHhnw4+002760
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMUAhnw4+002760
Zipping up the intermediate file: /var/lib/41000/projects/climateprediction.net/oifs_43r3_00ae_2019110100_123_993_12213812_0_r335119040_114.zip
Uploading the intermediate file: upload_file_114.zip
  10:21:03 STEP 2764 H=2764:00 +CPU= 14.098
  10:21:18 STEP 2765 H=2765:00 +CPU= 14.672
  10:21:41 STEP 2766 H=2766:00 +CPU= 22.935
  10:21:55 STEP 2767 H=2767:00 +CPU= 13.945
  10:22:09 STEP 2768 H=2768:00 +CPU= 14.183
  10:22:31 STEP 2769 H=2769:00 +CPU= 22.399
  10:22:46 STEP 2770 H=2770:00 +CPU= 14.623
  10:23:00 STEP 2771 H=2771:00 +CPU= 13.948
  10:23:24 STEP 2772 H=2772:00 +CPU= 24.171
Moving to projects directory: /var/lib/41000/slots/8/ICMGGhnw4+002772
Moving to projects directory: /var/lib/41000/slots/8/ICMSHhnw4+002772
Moving to projects directory: /var/lib/41000/slots/8/ICMUAhnw4+002772
  10:23:38 STEP 2773 H=2773:00 +CPU= 14.048
  10:23:52 STEP 2774 H=2774:00 +CPU= 14.260
  10:24:15 STEP 2775 H=2775:00 +CPU= 22.275
  10:24:29 STEP 2776 H=2776:00 +CPU= 14.415
  10:24:43 STEP 2777 H=2777:00 +CPU= 13.868
  10:25:05 STEP 2778 H=2778:00 +CPU= 22.598
  10:25:19 STEP 2779 H=2779:00 +CPU= 13.947
  10:25:34 STEP 2780 H=2780:00 +CPU= 14.205
  10:25:56 STEP 2781 H=2781:00 +CPU= 22.675
  10:26:11 STEP 2782 H=2782:00 +CPU= 14.482
  10:26:25 STEP 2783 H=2783:00 +CPU= 14.392
  10:26:50 STEP 2784 H=2784:00 +CPU= 25.282
Moving to projects directory: /var/lib/41000/slots/8/ICMGGhnw4+002784
Moving to projects directory: /var/lib/41000/slots/8/ICMSHhnw4+002784
Moving to projects directory: /var/lib/41000/slots/8/ICMUAhnw4+002784
  10:27:04 STEP 2785 H=2785:00 +CPU= 13.981
  10:27:18 STEP 2786 H=2786:00 +CPU= 14.098
  10:27:41 STEP 2787 H=2787:00 +CPU= 22.081
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMGGhnw4+002772
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMSHhnw4+002772
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMUAhnw4+002772
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMGGhnw4+002784
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMSHhnw4+002784
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMUAhnw4+002784
Zipping up the intermediate file: /var/lib/41000/projects/climateprediction.net/oifs_43r3_00ae_2019110100_123_993_12213812_0_r335119040_115.zip
Uploading the intermediate file: upload_file_115.zip
  10:27:55 STEP 2788 H=2788:00 +CPU= 14.078
  10:28:08 STEP 2789 H=2789:00 +CPU= 13.823
  10:28:31 STEP 2790 H=2790:00 +CPU= 22.528
  10:28:45 STEP 2791 H=2791:00 +CPU= 14.225
  10:29:00 STEP 2792 H=2792:00 +CPU= 14.610
  10:29:22 STEP 2793 H=2793:00 +CPU= 22.241
  10:29:37 STEP 2794 H=2794:00 +CPU= 15.206
  10:29:51 STEP 2795 H=2795:00 +CPU= 14.085
  10:30:15 STEP 2796 H=2796:00 +CPU= 24.105
Moving to projects directory: /var/lib/41000/slots/8/ICMGGhnw4+002796
Moving to projects directory: /var/lib/41000/slots/8/ICMSHhnw4+002796
Moving to projects directory: /var/lib/41000/slots/8/ICMUAhnw4+002796
  10:30:29 STEP 2797 H=2797:00 +CPU= 14.000
  10:30:42 STEP 2798 H=2798:00 +CPU= 12.722
  10:30:56 STEP 2799 H=2799:00 +CPU= 13.329
  10:31:04 STEP 2800 H=2800:00 +CPU=  8.387
  10:31:12 STEP 2801 H=2801:00 +CPU=  8.176
  10:31:26 STEP 2802 H=2802:00 +CPU= 13.657
  10:31:34 STEP 2803 H=2803:00 +CPU=  8.171
  10:31:42 STEP 2804 H=2804:00 +CPU=  8.498
  10:31:56 STEP 2805 H=2805:00 +CPU= 13.451
  10:32:04 STEP 2806 H=2806:00 +CPU=  8.419
  10:32:13 STEP 2807 H=2807:00 +CPU=  8.372
  10:32:28 STEP 2808 H=2808:00 +CPU= 15.426
Moving to projects directory: /var/lib/41000/slots/8/ICMGGhnw4+002808
Moving to projects directory: /var/lib/41000/slots/8/ICMSHhnw4+002808
Moving to projects directory: /var/lib/41000/slots/8/ICMUAhnw4+002808
  10:32:36 STEP 2809 H=2809:00 +CPU=  8.366
  10:32:46 STEP 2810 H=2810:00 +CPU=  9.172
  10:32:59 STEP 2811 H=2811:00 +CPU= 13.283
  10:33:07 STEP 2812 H=2812:00 +CPU=  8.479
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMGGhnw4+002796
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMSHhnw4+002796
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMUAhnw4+002796
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMGGhnw4+002808
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMSHhnw4+002808
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMUAhnw4+002808
Zipping up the intermediate file: /var/lib/41000/projects/climateprediction.net/oifs_43r3_00ae_2019110100_123_993_12213812_0_r335119040_116.zip
Uploading the intermediate file: upload_file_116.zip
  10:33:16 STEP 2813 H=2813:00 +CPU=  8.126
  10:33:29 STEP 2814 H=2814:00 +CPU= 13.745
  10:33:37 STEP 2815 H=2815:00 +CPU=  8.219
  10:33:46 STEP 2816 H=2816:00 +CPU=  8.415
  10:33:59 STEP 2817 H=2817:00 +CPU= 13.373
  10:34:08 STEP 2818 H=2818:00 +CPU=  8.550
  10:34:16 STEP 2819 H=2819:00 +CPU=  8.245
  10:34:31 STEP 2820 H=2820:00 +CPU= 14.536
  10:34:39 STEP 2821 H=2821:00 +CPU=  8.198
Moving to projects directory: /var/lib/41000/slots/8/ICMGGhnw4+002820
Moving to projects directory: /var/lib/41000/slots/8/ICMSHhnw4+002820
Moving to projects directory: /var/lib/41000/slots/8/ICMUAhnw4+002820
  10:34:47 STEP 2822 H=2822:00 +CPU=  8.512
  10:35:01 STEP 2823 H=2823:00 +CPU= 13.541
  10:35:09 STEP 2824 H=2824:00 +CPU=  8.407
  10:35:17 STEP 2825 H=2825:00 +CPU=  8.183
  10:35:31 STEP 2826 H=2826:00 +CPU= 13.638
  10:35:39 STEP 2827 H=2827:00 +CPU=  8.339
  10:35:48 STEP 2828 H=2828:00 +CPU=  8.456
  10:36:01 STEP 2829 H=2829:00 +CPU= 13.398
  10:36:10 STEP 2830 H=2830:00 +CPU=  8.449
  10:36:18 STEP 2831 H=2831:00 +CPU=  8.418
  10:36:34 STEP 2832 H=2832:00 +CPU= 15.360
Moving to projects directory: /var/lib/41000/slots/8/ICMGGhnw4+002832
Moving to projects directory: /var/lib/41000/slots/8/ICMSHhnw4+002832
Moving to projects directory: /var/lib/41000/slots/8/ICMUAhnw4+002832
  10:36:42 STEP 2833 H=2833:00 +CPU=  8.170
  10:36:51 STEP 2834 H=2834:00 +CPU=  9.293
  10:37:04 STEP 2835 H=2835:00 +CPU= 13.438
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMGGhnw4+002820
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMSHhnw4+002820
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMUAhnw4+002820
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMGGhnw4+002832
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMSHhnw4+002832
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMUAhnw4+002832
Zipping up the intermediate file: /var/lib/41000/projects/climateprediction.net/oifs_43r3_00ae_2019110100_123_993_12213812_0_r335119040_117.zip
Uploading the intermediate file: upload_file_117.zip
  10:37:13 STEP 2836 H=2836:00 +CPU=  8.548
  10:37:21 STEP 2837 H=2837:00 +CPU=  8.138
  10:37:35 STEP 2838 H=2838:00 +CPU= 13.703
  10:37:43 STEP 2839 H=2839:00 +CPU=  8.387
  10:37:52 STEP 2840 H=2840:00 +CPU=  8.530
  10:38:05 STEP 2841 H=2841:00 +CPU= 13.414
  10:38:14 STEP 2842 H=2842:00 +CPU=  8.473
  10:38:22 STEP 2843 H=2843:00 +CPU=  8.188
  10:38:37 STEP 2844 H=2844:00 +CPU= 14.758
Moving to projects directory: /var/lib/41000/slots/8/ICMGGhnw4+002844
Moving to projects directory: /var/lib/41000/slots/8/ICMSHhnw4+002844
Moving to projects directory: /var/lib/41000/slots/8/ICMUAhnw4+002844
  10:38:46 STEP 2845 H=2845:00 +CPU=  9.169
  10:38:54 STEP 2846 H=2846:00 +CPU=  8.558
  10:39:08 STEP 2847 H=2847:00 +CPU= 13.378
  10:39:16 STEP 2848 H=2848:00 +CPU=  8.584
  10:39:24 STEP 2849 H=2849:00 +CPU=  8.206
  10:39:38 STEP 2850 H=2850:00 +CPU= 13.846
  10:39:46 STEP 2851 H=2851:00 +CPU=  8.168
  10:39:55 STEP 2852 H=2852:00 +CPU=  8.445
  10:40:08 STEP 2853 H=2853:00 +CPU= 13.335
  10:40:17 STEP 2854 H=2854:00 +CPU=  8.399
  10:40:25 STEP 2855 H=2855:00 +CPU=  8.330
  10:40:40 STEP 2856 H=2856:00 +CPU= 15.308
  10:40:49 STEP 2857 H=2857:00 +CPU=  8.220
Moving to projects directory: /var/lib/41000/slots/8/ICMGGhnw4+002856
Moving to projects directory: /var/lib/41000/slots/8/ICMSHhnw4+002856
Moving to projects directory: /var/lib/41000/slots/8/ICMUAhnw4+002856
  10:40:57 STEP 2858 H=2858:00 +CPU=  8.357
  10:41:10 STEP 2859 H=2859:00 +CPU= 13.333
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMGGhnw4+002844
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMSHhnw4+002844
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMUAhnw4+002844
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMGGhnw4+002856
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMSHhnw4+002856
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMUAhnw4+002856
Zipping up the intermediate file: /var/lib/41000/projects/climateprediction.net/oifs_43r3_00ae_2019110100_123_993_12213812_0_r335119040_118.zip
Uploading the intermediate file: upload_file_118.zip
  10:41:19 STEP 2860 H=2860:00 +CPU=  8.388
  10:41:27 STEP 2861 H=2861:00 +CPU=  8.226
  10:41:40 STEP 2862 H=2862:00 +CPU= 13.640
  10:41:49 STEP 2863 H=2863:00 +CPU=  8.290
  10:41:57 STEP 2864 H=2864:00 +CPU=  8.513
  10:42:11 STEP 2865 H=2865:00 +CPU= 13.463
  10:42:20 STEP 2866 H=2866:00 +CPU=  8.900
  10:42:33 STEP 2867 H=2867:00 +CPU= 13.050
  10:42:57 STEP 2868 H=2868:00 +CPU= 24.280
Moving to projects directory: /var/lib/41000/slots/8/ICMGGhnw4+002868
Moving to projects directory: /var/lib/41000/slots/8/ICMSHhnw4+002868
Moving to projects directory: /var/lib/41000/slots/8/ICMUAhnw4+002868
  10:43:12 STEP 2869 H=2869:00 +CPU= 14.838
  10:43:26 STEP 2870 H=2870:00 +CPU= 14.466
  10:43:49 STEP 2871 H=2871:00 +CPU= 22.859
  10:44:08 STEP 2872 H=2872:00 +CPU= 18.860
  10:44:22 STEP 2873 H=2873:00 +CPU= 13.627
  10:44:43 STEP 2874 H=2874:00 +CPU= 21.807
  10:45:08 STEP 2875 H=2875:00 +CPU= 25.018
  10:45:28 STEP 2876 H=2876:00 +CPU= 19.247
  10:45:50 STEP 2877 H=2877:00 +CPU= 22.050
  10:46:05 STEP 2878 H=2878:00 +CPU= 15.398
  10:46:22 STEP 2879 H=2879:00 +CPU= 16.892
  10:47:02 STEP 2880 H=2880:00 +CPU= 39.750
Moving to projects directory: /var/lib/41000/slots/8/ICMGGhnw4+002880
Moving to projects directory: /var/lib/41000/slots/8/ICMSHhnw4+002880
Moving to projects directory: /var/lib/41000/slots/8/ICMUAhnw4+002880
  10:47:13 STEP 2881 H=2881:00 +CPU= 11.454
  10:47:29 STEP 2882 H=2882:00 +CPU= 16.034
  10:48:00 STEP 2883 H=2883:00 +CPU= 30.720
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMGGhnw4+002868
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMSHhnw4+002868
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMUAhnw4+002868
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMGGhnw4+002880
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMSHhnw4+002880
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMUAhnw4+002880
Zipping up the intermediate file: /var/lib/41000/projects/climateprediction.net/oifs_43r3_00ae_2019110100_123_993_12213812_0_r335119040_119.zip
Uploading the intermediate file: upload_file_119.zip
  10:48:15 STEP 2884 H=2884:00 +CPU= 15.154
  10:48:30 STEP 2885 H=2885:00 +CPU= 14.624
  10:48:58 STEP 2886 H=2886:00 +CPU= 28.657
  10:49:13 STEP 2887 H=2887:00 +CPU= 14.564
  10:49:32 STEP 2888 H=2888:00 +CPU= 19.343
  10:49:55 STEP 2889 H=2889:00 +CPU= 22.722
  10:50:15 STEP 2890 H=2890:00 +CPU= 19.580
  10:50:31 STEP 2891 H=2891:00 +CPU= 16.277
  10:51:01 STEP 2892 H=2892:00 +CPU= 29.551
Moving to projects directory: /var/lib/41000/slots/8/ICMGGhnw4+002892
Moving to projects directory: /var/lib/41000/slots/8/ICMSHhnw4+002892
Moving to projects directory: /var/lib/41000/slots/8/ICMUAhnw4+002892
  10:51:23 STEP 2893 H=2893:00 +CPU= 22.821
  10:51:43 STEP 2894 H=2894:00 +CPU= 19.691
  10:52:08 STEP 2895 H=2895:00 +CPU= 24.828
  10:52:27 STEP 2896 H=2896:00 +CPU= 18.649
  10:52:43 STEP 2897 H=2897:00 +CPU= 16.091
  10:53:17 STEP 2898 H=2898:00 +CPU= 34.286
  10:53:36 STEP 2899 H=2899:00 +CPU= 19.151
  10:53:49 STEP 2900 H=2900:00 +CPU= 12.790
  10:54:18 STEP 2901 H=2901:00 +CPU= 28.810
  10:54:33 STEP 2902 H=2902:00 +CPU= 14.987
  10:54:49 STEP 2903 H=2903:00 +CPU= 16.779
  10:55:24 STEP 2904 H=2904:00 +CPU= 34.706
Moving to projects directory: /var/lib/41000/slots/8/ICMGGhnw4+002904
Moving to projects directory: /var/lib/41000/slots/8/ICMSHhnw4+002904
Moving to projects directory: /var/lib/41000/slots/8/ICMUAhnw4+002904
  10:55:47 STEP 2905 H=2905:00 +CPU= 23.141
  10:56:04 STEP 2906 H=2906:00 +CPU= 16.599
  10:56:33 STEP 2907 H=2907:00 +CPU= 28.942
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMGGhnw4+002892
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMSHhnw4+002892
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMUAhnw4+002892
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMGGhnw4+002904
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMSHhnw4+002904
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMUAhnw4+002904
Zipping up the intermediate file: /var/lib/41000/projects/climateprediction.net/oifs_43r3_00ae_2019110100_123_993_12213812_0_r335119040_120.zip
Uploading the intermediate file: upload_file_120.zip
  10:56:48 STEP 2908 H=2908:00 +CPU= 15.214
  10:57:04 STEP 2909 H=2909:00 +CPU= 15.829
  10:57:31 STEP 2910 H=2910:00 +CPU= 27.113
  10:57:51 STEP 2911 H=2911:00 +CPU= 20.342
  10:58:07 STEP 2912 H=2912:00 +CPU= 15.376
  10:58:34 STEP 2913 H=2913:00 +CPU= 27.320
  10:58:51 STEP 2914 H=2914:00 +CPU= 17.417
  10:59:14 STEP 2915 H=2915:00 +CPU= 22.786
  10:59:50 STEP 2916 H=2916:00 +CPU= 35.828
Moving to projects directory: /var/lib/41000/slots/8/ICMGGhnw4+002916
Moving to projects directory: /var/lib/41000/slots/8/ICMSHhnw4+002916
Moving to projects directory: /var/lib/41000/slots/8/ICMUAhnw4+002916
  11:00:14 STEP 2917 H=2917:00 +CPU= 23.512
  11:00:37 STEP 2918 H=2918:00 +CPU= 23.365
  11:01:17 STEP 2919 H=2919:00 +CPU= 39.918
  11:01:41 STEP 2920 H=2920:00 +CPU= 23.845
  11:02:02 STEP 2921 H=2921:00 +CPU= 21.334
  11:02:36 STEP 2922 H=2922:00 +CPU= 33.838
  11:03:00 STEP 2923 H=2923:00 +CPU= 23.947
  11:03:14 STEP 2924 H=2924:00 +CPU= 14.406
  11:03:55 STEP 2925 H=2925:00 +CPU= 40.339
  11:04:17 STEP 2926 H=2926:00 +CPU= 22.851
  11:04:36 STEP 2927 H=2927:00 +CPU= 18.790
  11:05:22 STEP 2928 H=2928:00 +CPU= 45.918
Moving to projects directory: /var/lib/41000/slots/8/ICMGGhnw4+002928
Moving to projects directory: /var/lib/41000/slots/8/ICMSHhnw4+002928
Moving to projects directory: /var/lib/41000/slots/8/ICMUAhnw4+002928
  11:05:44 STEP 2929 H=2929:00 +CPU= 22.252
  11:06:02 STEP 2930 H=2930:00 +CPU= 17.890
  11:06:43 STEP 2931 H=2931:00 +CPU= 40.336
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMGGhnw4+002916
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMSHhnw4+002916
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMUAhnw4+002916
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMGGhnw4+002928
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMSHhnw4+002928
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMUAhnw4+002928
Zipping up the intermediate file: /var/lib/41000/projects/climateprediction.net/oifs_43r3_00ae_2019110100_123_993_12213812_0_r335119040_121.zip
Uploading the intermediate file: upload_file_121.zip
Uploading trickle at timestep: 10551600
  11:07:08 STEP 2932 H=2932:00 +CPU= 25.274
  11:07:35 STEP 2933 H=2933:00 +CPU= 27.272
  11:07:57 STEP 2934 H=2934:00 +CPU= 22.047
  11:08:16 STEP 2935 H=2935:00 +CPU= 18.845
  11:08:37 STEP 2936 H=2936:00 +CPU= 20.821
  11:08:59 STEP 2937 H=2937:00 +CPU= 22.275
  11:09:20 STEP 2938 H=2938:00 +CPU= 20.778
  11:09:39 STEP 2939 H=2939:00 +CPU= 19.065
  11:10:20 STEP 2940 H=2940:00 +CPU= 41.259
Moving to projects directory: /var/lib/41000/slots/8/ICMGGhnw4+002940
Moving to projects directory: /var/lib/41000/slots/8/ICMSHhnw4+002940
Moving to projects directory: /var/lib/41000/slots/8/ICMUAhnw4+002940
  11:10:40 STEP 2941 H=2941:00 +CPU= 19.755
  11:11:03 STEP 2942 H=2942:00 +CPU= 22.373
  11:11:38 STEP 2943 H=2943:00 +CPU= 35.670
  11:11:59 STEP 2944 H=2944:00 +CPU= 20.299
  11:12:18 STEP 2945 H=2945:00 +CPU= 19.246
  11:12:54 STEP 2946 H=2946:00 +CPU= 35.742
  11:13:14 STEP 2947 H=2947:00 +CPU= 19.982
  11:13:34 STEP 2948 H=2948:00 +CPU= 20.676
  11:14:10 STEP 2949 H=2949:00 +CPU= 35.990
  11:14:30 STEP 2950 H=2950:00 +CPU= 19.651
  11:14:52 STEP 2951 H=2951:00 +CPU= 21.848
  11:15:19 STEP 2952 H=2952:00 +CPU= 27.666
..The child process terminated with status: 0
>>> Printing last 70 lines from file: NODE.001_01

 STEPO, YSPSDT_AR1     (  1,...), SP NORMS 2D: 4.865283731608E-01 1.865286143845E-01 6.362598939121E-02
 STEPO, YSPSDT_AR1     (SUM,...), SP NORMS 2D: 5.260318722649E-01
 STEPO, YGPSDT         (  1,...), Grid Point Norms
  FIELD LEV        AVERAGE               MINIMUM               MAXIMUM       
    101  60  3.48989191816039E-02 -1.72994898449055E+00  1.89761310057317E+00
 STEPO, YGPSDT[clipped](  1,...), Grid Point Norms
  FIELD LEV        AVERAGE               MINIMUM               MAXIMUM       
    101  60  3.19925486346338E-02 -1.00000000000000E+00  1.00000000000000E+00
  MAXGPFV : MAX. VALUE =    1.18024572024494     
  MAXGPFV : MAX. VALUE =   0.748729345208885     
 NSTEP =  2952 SCAN2M_HPOS  P
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ICMGGhnw4+002952 MODE=a
 IO-STREAM CLOSED - ICMGGhnw4+002952
 MPI-TASK:   1 -    348537354 BYTES IN      8 RECORDS TRANSFERRED IN    0.0002 SECONDS******** Mbytes/s, TOTAL TIME=    0.0054( 3.7%)
   !!!!!!!!!!!!!! WAVE FIELDS INTEGRATED.  DATE IS: 20200302231500  !!!!!!!!!!!! 
   !!!!!!!!!!!!!! WAVE FIELDS INTEGRATED.  DATE IS: 20200302233000  !!!!!!!!!!!! 
   !!!!!!!!!!!!!! WAVE FIELDS INTEGRATED.  DATE IS: 20200302234500  !!!!!!!!!!!! 
   !!!!!!!!!!!!!! WAVE FIELDS INTEGRATED.  DATE IS: 20200303000000  !!!!!!!!!!!! 
 11:15:19 STEP 2952 H=2952:00 +CPU= 27.666
 END CNT3
 NSTEP =  2952 CNT0   000000000  23.099  23.099  23.133   0.999               0               0               0               0               0
IO-STREAM STATISTICS, TOTAL NO OF REC/BYTES READ           35/   804992476 WRITTEN        22380/************
 -TASK-OPENED-OPEN-RECS IN -KBYTE IN    -RECS OUT-KBYTE OUT-WALL   -WALL IN-WALL
  OU-TOT IN -TOT OUT
   1    543    0       35       786125    22380 ************    80.8     0.7    80.1     0.9   135.1
===-=== START OF TIMING STATISTICS ===-===
 
STATS FOR ALL TASKS
 NUM ROUTINE                                     CALLS  MEAN(ms)   MAX(ms)   FRAC(%)  UNBAL(%)
   0 CNT0     - COMPLETE EXECUTION                   1 ********* *********    100.00      0.00
   1 CNT4     - FORWARD INTEGRATION                  1 ********* *********    100.00      0.00
   8 SCAN2M - GRID-POINT DYNAMICS                 1561    1446.8    1446.8      7.89      0.00
   9 SPCM     - SPECTRAL COMP.                    1440     200.4     200.4      1.01      0.00
  10 SCAN2M - PHYSICS                             1441    9016.6    9016.6     45.42      0.00
  11 IOPACK   - OUTPUT P.P. RESULTS                120    1039.5    1039.5      0.44      0.00
  12 SPNORM   - SPECTRAL NORM COMP.                 61      16.4      16.4      0.00      0.00
  13 SCAN2M - RADIATION CALC.                      481    9365.1    9365.1     15.75      0.00
  17 GRIDFPOS IN CNT4                              120      38.2      38.2      0.02      0.00
  27 RESTART FILES - WRITING                        60    2078.5    2078.5      0.44      0.00
  28 RESTART FILES - READING                         1    1144.5    1144.5      0.00      0.00
  29 SU4FPOS IN CNT4                               120       0.3       0.3      0.00      0.00
  30 DYNFPOS IN CNT4                               120    1673.7    1673.7      0.70      0.00
  31 POSDDH IN STEPO                                 6       2.9       2.9      0.00      0.00
  37 CPGLAG   - SL COMPUTATIONS                   1441    2547.1    2547.1     12.83      0.00
  38 WAM      - TOTAL COST OF WAVE MODEL          1441    2126.4    2126.4     10.71      0.00
  39 SU0YOMB                                         1     112.2     112.2      0.00      0.00
  51 SCAN2M   - SL COMM. PART 1                   1441      14.7      14.7      0.07      0.00
  54 SPCM     - M TO S/S TO M TRANSP.             1440      79.9      79.9      0.40      0.00
  55 SPCIMPF  - S TO M/M TO S TRANSP.             1440      17.9      17.9      0.09      0.00
  56 SPNORM   - SPECTRAL NORM COMM.                 61       0.3       0.3      0.00      0.00
 102 LTINV_CTL   - INVERSE LEGENDRE TRANSFORM     4926     109.7     109.7      1.89      0.00
 103 LTDIR_CTL   - DIRECT LEGENDRE TRANSFORM      3000     104.2     104.2      1.09      0.00
 106 FTDIR_CTL   - DIRECT FOURIER TRANSFORM       3000      37.0      37.0      0.39      0.00
 107 FTINV_CTL   - INVERSE FOURIER TRANSFORM      4926      37.0      37.0      0.64      0.00
 140 SULEG       - COMP. OF LEGENDRE POL.            2       7.8       7.8      0.00      0.00
 152 LTINV_CTL   - M TO L TRANSPOSITION           4926      10.8      10.8      0.19      0.00
 153 LTDIR_CTL   - L TO M TRANSPOSITION           3000       9.7       9.7      0.10      0.00
 157 FTINV_CTL   - L TO G TRANSPOSITION           4926      15.1      15.1      0.26      0.00
 158 FTDIR_CTL   - G TO L TRANSPOSITION           3000      13.0      13.0      0.14      0.00
 400 GSTATS                                     287490       0.0       0.0      0.00      0.00
 401 GSTATS HOOK                                275307       0.0       0.0      0.00      0.00
TOTAL MEASURED IMBALANCE =       0.0 SECONDS,  0.0 PERCENT
TOTAL WALLCLOCK TIME    28608.2 CPU TIME   28535.3 VECTOR TIME    28535.3


===-=== END   OF TIMING STATISTICS ===-===
FORECAST DAYS PER DAY      371.5
  *** END CNT0 *** 
------------------------------------------------
Moving to projects directory: /var/lib/41000/slots/8/ICMGGhnw4+002952
Moving to projects directory: /var/lib/41000/slots/8/ICMSHhnw4+002952
Moving to projects directory: /var/lib/41000/slots/8/ICMUAhnw4+002952
Adding to the zip: /var/lib/41000/slots/8/NODE.001_01
Adding to the zip: /var/lib/41000/slots/8/ifs.stat
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMSHhnw4+001512
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMGGhnw4+000384
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMGGhnw4+001512
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMUAhnw4+002940
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMSHhnw4+002940
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMGGhnw4+002952
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMUAhnw4+001512
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMSHhnw4+002952
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMSHhnw4+000384
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMUAhnw4+000384
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMGGhnw4+002940
Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12213812/ICMUAhnw4+002952
Zipping up the final file: /var/lib/41000/projects/climateprediction.net/oifs_43r3_00ae_2019110100_123_993_12213812_0_r335119040_122.zip
Uploading the final file: upload_file_122.zip
Uploading trickle at timestep: 10623600
11:18:45 (18350): called boinc_finish(0)

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 May 2023 16:53:55 1539146 22315161 oifs_43r3_00ae_2019110100_123_993_12213812_0 10,623,600 58,663 0.0055
09 May 2023 16:53:55 1539146 22315161 oifs_43r3_00ae_2019110100_123_993_12213812_0 10,551,600 58,147 0.0055
09 May 2023 16:53:50 1539146 22315161 oifs_43r3_00ae_2019110100_123_993_12213812_0 9,687,600 54,166 0.0056
09 May 2023 16:53:46 1539146 22315161 oifs_43r3_00ae_2019110100_123_993_12213812_0 8,823,600 49,431 0.0056
09 May 2023 16:53:44 1539146 22315161 oifs_43r3_00ae_2019110100_123_993_12213812_0 7,959,600 43,923 0.0055
09 May 2023 16:53:38 1539146 22315161 oifs_43r3_00ae_2019110100_123_993_12213812_0 7,092,000 38,415 0.0054
09 May 2023 16:53:35 1539146 22315161 oifs_43r3_00ae_2019110100_123_993_12213812_0 6,228,000 33,014 0.0053
09 May 2023 16:53:29 1539146 22315161 oifs_43r3_00ae_2019110100_123_993_12213812_0 5,014,800 28,719 0.0057
09 May 2023 16:53:23 1539146 22315161 oifs_43r3_00ae_2019110100_123_993_12213812_0 3,718,800 19,763 0.0053
09 May 2023 16:53:16 1539146 22315161 oifs_43r3_00ae_2019110100_123_993_12213812_0 2,163,600 11,682 0.0054
09 May 2023 16:53:09 1539146 22315161 oifs_43r3_00ae_2019110100_123_993_12213812_0 864,000 5,099 0.0059


©2024 cpdn.org