climateprediction.net home page
Task 22315509

Task 22315509

Name oifs_43r3_00jt_2019110100_123_993_12214151_0
Workunit 12214151
Created 24 Feb 2023, 13:59:44 UTC
Sent 25 Feb 2023, 0:38:43 UTC
Report deadline 26 Apr 2023, 0:38:43 UTC
Received 26 Feb 2023, 1:42:25 UTC
Server state Over
Outcome Success
Client state Done
Exit status 0 (0x00000000)
Computer ID 1496046
Run time 23 hours 18 min 12 sec
CPU time 23 hours 15 min 28 sec
Validate state Valid
Credit 14,873.04
Device peak FLOPS 5.39 GFLOPS
Application version OpenIFS 43r3 v1.21
x86_64-pc-linux-gnu
Peak working set size 4,416.24 MB
Peak swap size 4,974.79 MB
Peak disk usage 1,845.80 MB
Stderr
<core_client_version>7.20.2</core_client_version>
<![CDATA[
<stderr_txt>
0230225:145312:1677358392:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:530FCTest:1:1:269161:269161] [20230225:145312:1677358392:0.000] [process_options@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:530FCTest:1:1:269161:269161] [20230225:145312:1677358392:0.000] [ignore_signals@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:530FCTest:1:1:269161:269161] [20230225:145312:1677358392:0.000] [restore_default_signals@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:530FCTest:1:1:269161:269161] [20230225:145312:1677358392:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x810730 (old at 0x1ce8440)
[EC_DRHOOK:530FCTest:1:1:269161:269161] [20230225:145312:1677358392:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x810730 (old at (nil))
[EC_DRHOOK:530FCTest:1:1:269161:269161] [20230225:145312:1677358392:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x810730 (old at 0x1ce8440)
[EC_DRHOOK:530FCTest:1:1:269161:269161] [20230225:145312:1677358392:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x810730 (old at (nil))
[EC_DRHOOK:530FCTest:1:1:269161:269161] [20230225:145312:1677358392:0.000] [trapfpe_treatment@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:530FCTest:1:1:269161:269161] [20230225:145312:1677358392:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x810730 (old at 0x1ce8440)
[EC_DRHOOK:530FCTest:1:1:269161:269161] [20230225:145312:1677358392:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x810730 (old at 0x1ce8440)
[EC_DRHOOK:530FCTest:1:1:269161:269161] [20230225:145312:1677358392:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x810730 (old at (nil))
[EC_DRHOOK:530FCTest:1:1:269161:269161] [20230225:145312:1677358392:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x810730 (old at 0x1ce8440)
[EC_DRHOOK:530FCTest:1:1:269161:269161] [20230225:145312:1677358392:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x810730 (old at 0x1ce8440)
[EC_DRHOOK:530FCTest:1:1:269161:269161] [20230225:145312:1677358392:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x810730 (old at 0x1ce8440)
[EC_DRHOOK:530FCTest:1:1:269161:269161] [20230225:145312:1677358392:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x810730 (old at (nil))
[EC_DRHOOK:530FCTest:1:1:269161:269161] [20230225:145312:1677358392:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x810730 (old at (nil))
[EC_DRHOOK:530FCTest:1:1:269161:269161] [20230225:145312:1677358392:0.000] [signal_drhook_init@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x810730 (old at (nil))
[EC_DRHOOK:530FCTest:1:1:269161:269161] [20230225:145312:1677358392:0.000] [catch_signals@/home/glenn/github/gc_oifs43r3/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMGGhnw4+002376
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMSHhnw4+002376
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMUAhnw4+002376
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMGGhnw4+002388
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMSHhnw4+002388
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMUAhnw4+002388
Zipping up the intermediate file: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_00jt_2019110100_123_993_12214151_0_r722857544_99.zip
Uploading the intermediate file: upload_file_99.zip
  14:53:58 STEP 2400 H=2400:00 +CPU= 43.152
Moving to projects directory: /boinc_data/boinc/slots/24/ICMGGhnw4+002400
  14:54:22 STEP 2401 H=2401:00 +CPU= 23.882
  14:54:48 STEP 2402 H=2402:00 +CPU= 25.356
  14:55:25 STEP 2403 H=2403:00 +CPU= 37.667
  14:55:50 STEP 2404 H=2404:00 +CPU= 24.038
  14:56:14 STEP 2405 H=2405:00 +CPU= 24.596
  14:56:51 STEP 2406 H=2406:00 +CPU= 37.002
  14:57:15 STEP 2407 H=2407:00 +CPU= 23.669
  14:57:41 STEP 2408 H=2408:00 +CPU= 25.352
  14:58:18 STEP 2409 H=2409:00 +CPU= 36.864
  14:58:43 STEP 2410 H=2410:00 +CPU= 25.311
  14:59:07 STEP 2411 H=2411:00 +CPU= 23.632
  14:59:48 STEP 2412 H=2412:00 +CPU= 41.469
Moving to projects directory: /boinc_data/boinc/slots/24/ICMGGhnw4+002412
Moving to projects directory: /boinc_data/boinc/slots/24/ICMSHhnw4+002412
Moving to projects directory: /boinc_data/boinc/slots/24/ICMUAhnw4+002412
  15:00:12 STEP 2413 H=2413:00 +CPU= 23.697
  15:00:36 STEP 2414 H=2414:00 +CPU= 24.308
  15:01:12 STEP 2415 H=2415:00 +CPU= 35.666
  15:01:37 STEP 2416 H=2416:00 +CPU= 24.679
  15:02:01 STEP 2417 H=2417:00 +CPU= 24.031
  15:02:39 STEP 2418 H=2418:00 +CPU= 37.390
  15:03:04 STEP 2419 H=2419:00 +CPU= 24.463
  15:03:30 STEP 2420 H=2420:00 +CPU= 25.700
  15:04:07 STEP 2421 H=2421:00 +CPU= 37.337
  15:04:33 STEP 2422 H=2422:00 +CPU= 25.197
  15:04:57 STEP 2423 H=2423:00 +CPU= 24.430
  15:05:40 STEP 2424 H=2424:00 +CPU= 42.067
Moving to projects directory: /boinc_data/boinc/slots/24/ICMGGhnw4+002424
Moving to projects directory: /boinc_data/boinc/slots/24/ICMSHhnw4+002424
Moving to projects directory: /boinc_data/boinc/slots/24/ICMUAhnw4+002424
  15:06:03 STEP 2425 H=2425:00 +CPU= 23.223
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMGGhnw4+002412
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMSHhnw4+002412
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMUAhnw4+002412
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMGGhnw4+002424
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMSHhnw4+002424
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMUAhnw4+002424
Zipping up the intermediate file: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_00jt_2019110100_123_993_12214151_0_r722857544_100.zip
Uploading the intermediate file: upload_file_100.zip
  15:06:28 STEP 2426 H=2426:00 +CPU= 24.838
  15:07:04 STEP 2427 H=2427:00 +CPU= 36.149
  15:07:29 STEP 2428 H=2428:00 +CPU= 24.049
  15:07:52 STEP 2429 H=2429:00 +CPU= 23.448
  15:08:29 STEP 2430 H=2430:00 +CPU= 36.673
  15:08:53 STEP 2431 H=2431:00 +CPU= 23.405
  15:09:16 STEP 2432 H=2432:00 +CPU= 23.185
  15:09:53 STEP 2433 H=2433:00 +CPU= 37.350
  15:10:18 STEP 2434 H=2434:00 +CPU= 24.499
  15:10:42 STEP 2435 H=2435:00 +CPU= 24.124
  15:11:22 STEP 2436 H=2436:00 +CPU= 39.759
Moving to projects directory: /boinc_data/boinc/slots/24/ICMGGhnw4+002436
Moving to projects directory: /boinc_data/boinc/slots/24/ICMSHhnw4+002436
Moving to projects directory: /boinc_data/boinc/slots/24/ICMUAhnw4+002436
  15:11:45 STEP 2437 H=2437:00 +CPU= 22.622
  15:12:09 STEP 2438 H=2438:00 +CPU= 23.743
  15:12:45 STEP 2439 H=2439:00 +CPU= 36.498
  15:13:09 STEP 2440 H=2440:00 +CPU= 24.084
  15:13:32 STEP 2441 H=2441:00 +CPU= 22.203
  15:14:12 STEP 2442 H=2442:00 +CPU= 39.586
  15:14:37 STEP 2443 H=2443:00 +CPU= 24.997
  15:15:02 STEP 2444 H=2444:00 +CPU= 24.543
  15:15:39 STEP 2445 H=2445:00 +CPU= 36.762
  15:16:03 STEP 2446 H=2446:00 +CPU= 23.930
  15:16:26 STEP 2447 H=2447:00 +CPU= 22.641
  15:17:09 STEP 2448 H=2448:00 +CPU= 42.372
Moving to projects directory: /boinc_data/boinc/slots/24/ICMGGhnw4+002448
Moving to projects directory: /boinc_data/boinc/slots/24/ICMSHhnw4+002448
Moving to projects directory: /boinc_data/boinc/slots/24/ICMUAhnw4+002448
  15:17:33 STEP 2449 H=2449:00 +CPU= 24.210
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMGGhnw4+002436
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMSHhnw4+002436
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMUAhnw4+002436
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMGGhnw4+002448
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMSHhnw4+002448
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMUAhnw4+002448
Zipping up the intermediate file: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_00jt_2019110100_123_993_12214151_0_r722857544_101.zip
Uploading the intermediate file: upload_file_101.zip
  15:17:57 STEP 2450 H=2450:00 +CPU= 24.227
  15:18:33 STEP 2451 H=2451:00 +CPU= 35.711
  15:18:57 STEP 2452 H=2452:00 +CPU= 23.168
  15:19:21 STEP 2453 H=2453:00 +CPU= 24.081
  15:19:59 STEP 2454 H=2454:00 +CPU= 37.558
  15:20:22 STEP 2455 H=2455:00 +CPU= 23.356
  15:20:46 STEP 2456 H=2456:00 +CPU= 23.972
  15:21:23 STEP 2457 H=2457:00 +CPU= 36.857
  15:21:48 STEP 2458 H=2458:00 +CPU= 24.732
  15:22:12 STEP 2459 H=2459:00 +CPU= 24.021
  15:22:54 STEP 2460 H=2460:00 +CPU= 41.460
Moving to projects directory: /boinc_data/boinc/slots/24/ICMGGhnw4+002460
Moving to projects directory: /boinc_data/boinc/slots/24/ICMSHhnw4+002460
Moving to projects directory: /boinc_data/boinc/slots/24/ICMUAhnw4+002460
  15:23:19 STEP 2461 H=2461:00 +CPU= 24.387
  15:23:44 STEP 2462 H=2462:00 +CPU= 24.443
  15:24:21 STEP 2463 H=2463:00 +CPU= 36.797
  15:24:46 STEP 2464 H=2464:00 +CPU= 25.188
  15:25:11 STEP 2465 H=2465:00 +CPU= 24.639
  15:25:48 STEP 2466 H=2466:00 +CPU= 37.241
  15:26:12 STEP 2467 H=2467:00 +CPU= 24.054
  15:26:38 STEP 2468 H=2468:00 +CPU= 25.311
  15:27:15 STEP 2469 H=2469:00 +CPU= 36.945
  15:27:40 STEP 2470 H=2470:00 +CPU= 24.617
  15:28:04 STEP 2471 H=2471:00 +CPU= 24.216
  15:28:47 STEP 2472 H=2472:00 +CPU= 42.951
Moving to projects directory: /boinc_data/boinc/slots/24/ICMGGhnw4+002472
Moving to projects directory: /boinc_data/boinc/slots/24/ICMSHhnw4+002472
Moving to projects directory: /boinc_data/boinc/slots/24/ICMUAhnw4+002472
  15:29:11 STEP 2473 H=2473:00 +CPU= 24.103
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMGGhnw4+002460
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMSHhnw4+002460
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMUAhnw4+002460
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMGGhnw4+002472
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMSHhnw4+002472
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMUAhnw4+002472
Zipping up the intermediate file: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_00jt_2019110100_123_993_12214151_0_r722857544_102.zip
Uploading the intermediate file: upload_file_102.zip
  15:29:36 STEP 2474 H=2474:00 +CPU= 24.237
  15:30:13 STEP 2475 H=2475:00 +CPU= 37.212
  15:30:39 STEP 2476 H=2476:00 +CPU= 25.140
  15:31:02 STEP 2477 H=2477:00 +CPU= 23.797
  15:31:40 STEP 2478 H=2478:00 +CPU= 37.160
  15:32:04 STEP 2479 H=2479:00 +CPU= 24.300
  15:32:29 STEP 2480 H=2480:00 +CPU= 24.650
  15:33:07 STEP 2481 H=2481:00 +CPU= 38.060
  15:33:33 STEP 2482 H=2482:00 +CPU= 25.099
  15:33:57 STEP 2483 H=2483:00 +CPU= 24.542
  15:34:38 STEP 2484 H=2484:00 +CPU= 40.296
Moving to projects directory: /boinc_data/boinc/slots/24/ICMGGhnw4+002484
Moving to projects directory: /boinc_data/boinc/slots/24/ICMSHhnw4+002484
Moving to projects directory: /boinc_data/boinc/slots/24/ICMUAhnw4+002484
  15:35:02 STEP 2485 H=2485:00 +CPU= 24.148
  15:35:27 STEP 2486 H=2486:00 +CPU= 24.393
  15:36:04 STEP 2487 H=2487:00 +CPU= 37.422
  15:36:29 STEP 2488 H=2488:00 +CPU= 24.658
  15:36:53 STEP 2489 H=2489:00 +CPU= 23.364
  15:37:30 STEP 2490 H=2490:00 +CPU= 37.515
  15:37:55 STEP 2491 H=2491:00 +CPU= 24.730
  15:38:19 STEP 2492 H=2492:00 +CPU= 24.240
  15:38:56 STEP 2493 H=2493:00 +CPU= 36.843
  15:39:21 STEP 2494 H=2494:00 +CPU= 24.933
  15:39:46 STEP 2495 H=2495:00 +CPU= 24.911
  15:40:30 STEP 2496 H=2496:00 +CPU= 43.273
Moving to projects directory: /boinc_data/boinc/slots/24/ICMGGhnw4+002496
Moving to projects directory: /boinc_data/boinc/slots/24/ICMSHhnw4+002496
Moving to projects directory: /boinc_data/boinc/slots/24/ICMUAhnw4+002496
  15:40:55 STEP 2497 H=2497:00 +CPU= 24.159
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMGGhnw4+002484
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMSHhnw4+002484
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMUAhnw4+002484
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMGGhnw4+002496
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMSHhnw4+002496
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMUAhnw4+002496
Zipping up the intermediate file: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_00jt_2019110100_123_993_12214151_0_r722857544_103.zip
Uploading the intermediate file: upload_file_103.zip
  15:41:19 STEP 2498 H=2498:00 +CPU= 24.649
  15:41:56 STEP 2499 H=2499:00 +CPU= 36.437
  15:42:21 STEP 2500 H=2500:00 +CPU= 25.166
  15:42:46 STEP 2501 H=2501:00 +CPU= 24.583
  15:43:25 STEP 2502 H=2502:00 +CPU= 38.555
  15:43:50 STEP 2503 H=2503:00 +CPU= 24.606
  15:44:15 STEP 2504 H=2504:00 +CPU= 25.233
  15:44:52 STEP 2505 H=2505:00 +CPU= 36.262
  15:45:16 STEP 2506 H=2506:00 +CPU= 24.051
  15:45:40 STEP 2507 H=2507:00 +CPU= 24.011
  15:46:20 STEP 2508 H=2508:00 +CPU= 40.496
Moving to projects directory: /boinc_data/boinc/slots/24/ICMGGhnw4+002508
Moving to projects directory: /boinc_data/boinc/slots/24/ICMSHhnw4+002508
Moving to projects directory: /boinc_data/boinc/slots/24/ICMUAhnw4+002508
  15:46:45 STEP 2509 H=2509:00 +CPU= 24.051
  15:47:09 STEP 2510 H=2510:00 +CPU= 24.775
  15:47:46 STEP 2511 H=2511:00 +CPU= 36.549
  15:48:11 STEP 2512 H=2512:00 +CPU= 25.051
  15:48:35 STEP 2513 H=2513:00 +CPU= 23.359
  15:49:11 STEP 2514 H=2514:00 +CPU= 36.372
  15:49:34 STEP 2515 H=2515:00 +CPU= 22.592
  15:49:58 STEP 2516 H=2516:00 +CPU= 23.548
  15:50:33 STEP 2517 H=2517:00 +CPU= 35.176
  15:50:58 STEP 2518 H=2518:00 +CPU= 24.611
  15:51:23 STEP 2519 H=2519:00 +CPU= 24.585
  15:52:06 STEP 2520 H=2520:00 +CPU= 43.033
Moving to projects directory: /boinc_data/boinc/slots/24/ICMGGhnw4+002520
Moving to projects directory: /boinc_data/boinc/slots/24/ICMSHhnw4+002520
Moving to projects directory: /boinc_data/boinc/slots/24/ICMUAhnw4+002520
  15:52:29 STEP 2521 H=2521:00 +CPU= 22.839
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMGGhnw4+002508
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMSHhnw4+002508
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMUAhnw4+002508
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMGGhnw4+002520
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMSHhnw4+002520
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMUAhnw4+002520
Zipping up the intermediate file: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_00jt_2019110100_123_993_12214151_0_r722857544_104.zip
Uploading the intermediate file: upload_file_104.zip
  15:52:54 STEP 2522 H=2522:00 +CPU= 24.454
  15:53:31 STEP 2523 H=2523:00 +CPU= 36.423
  15:53:55 STEP 2524 H=2524:00 +CPU= 23.913
  15:54:19 STEP 2525 H=2525:00 +CPU= 24.573
  15:54:59 STEP 2526 H=2526:00 +CPU= 39.046
  15:55:24 STEP 2527 H=2527:00 +CPU= 25.025
  15:55:50 STEP 2528 H=2528:00 +CPU= 25.323
  15:56:28 STEP 2529 H=2529:00 +CPU= 37.914
  15:56:53 STEP 2530 H=2530:00 +CPU= 25.235
  15:57:18 STEP 2531 H=2531:00 +CPU= 24.471
  15:57:57 STEP 2532 H=2532:00 +CPU= 39.538
Moving to projects directory: /boinc_data/boinc/slots/24/ICMGGhnw4+002532
Moving to projects directory: /boinc_data/boinc/slots/24/ICMSHhnw4+002532
Moving to projects directory: /boinc_data/boinc/slots/24/ICMUAhnw4+002532
  15:58:21 STEP 2533 H=2533:00 +CPU= 24.013
  15:58:46 STEP 2534 H=2534:00 +CPU= 24.563
  15:59:23 STEP 2535 H=2535:00 +CPU= 36.817
  15:59:47 STEP 2536 H=2536:00 +CPU= 24.323
  16:00:12 STEP 2537 H=2537:00 +CPU= 24.713
  16:00:49 STEP 2538 H=2538:00 +CPU= 36.614
  16:01:14 STEP 2539 H=2539:00 +CPU= 24.744
  16:01:39 STEP 2540 H=2540:00 +CPU= 24.999
  16:02:16 STEP 2541 H=2541:00 +CPU= 36.918
  16:02:42 STEP 2542 H=2542:00 +CPU= 25.736
  16:03:08 STEP 2543 H=2543:00 +CPU= 25.433
  16:03:52 STEP 2544 H=2544:00 +CPU= 44.028
Moving to projects directory: /boinc_data/boinc/slots/24/ICMGGhnw4+002544
Moving to projects directory: /boinc_data/boinc/slots/24/ICMSHhnw4+002544
Moving to projects directory: /boinc_data/boinc/slots/24/ICMUAhnw4+002544
  16:04:16 STEP 2545 H=2545:00 +CPU= 23.530
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMGGhnw4+002532
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMSHhnw4+002532
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMUAhnw4+002532
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMGGhnw4+002544
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMSHhnw4+002544
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMUAhnw4+002544
Zipping up the intermediate file: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_00jt_2019110100_123_993_12214151_0_r722857544_105.zip
Uploading the intermediate file: upload_file_105.zip
  16:04:41 STEP 2546 H=2546:00 +CPU= 25.463
  16:05:19 STEP 2547 H=2547:00 +CPU= 38.006
  16:05:46 STEP 2548 H=2548:00 +CPU= 26.443
  16:06:11 STEP 2549 H=2549:00 +CPU= 24.099
  16:06:51 STEP 2550 H=2550:00 +CPU= 38.729
  16:07:15 STEP 2551 H=2551:00 +CPU= 23.961
  16:07:40 STEP 2552 H=2552:00 +CPU= 24.894
  16:08:17 STEP 2553 H=2553:00 +CPU= 37.225
  16:08:43 STEP 2554 H=2554:00 +CPU= 25.269
  16:09:07 STEP 2555 H=2555:00 +CPU= 24.078
  16:09:47 STEP 2556 H=2556:00 +CPU= 40.380
Moving to projects directory: /boinc_data/boinc/slots/24/ICMGGhnw4+002556
Moving to projects directory: /boinc_data/boinc/slots/24/ICMSHhnw4+002556
Moving to projects directory: /boinc_data/boinc/slots/24/ICMUAhnw4+002556
  16:10:12 STEP 2557 H=2557:00 +CPU= 24.971
  16:10:38 STEP 2558 H=2558:00 +CPU= 25.531
  16:11:16 STEP 2559 H=2559:00 +CPU= 37.253
  16:11:40 STEP 2560 H=2560:00 +CPU= 24.875
  16:12:04 STEP 2561 H=2561:00 +CPU= 23.742
  16:12:40 STEP 2562 H=2562:00 +CPU= 35.557
  16:13:04 STEP 2563 H=2563:00 +CPU= 23.874
  16:13:28 STEP 2564 H=2564:00 +CPU= 23.950
  16:14:06 STEP 2565 H=2565:00 +CPU= 37.407
  16:14:32 STEP 2566 H=2566:00 +CPU= 25.879
  16:14:58 STEP 2567 H=2567:00 +CPU= 25.237
  16:15:41 STEP 2568 H=2568:00 +CPU= 42.361
Moving to projects directory: /boinc_data/boinc/slots/24/ICMGGhnw4+002568
Moving to projects directory: /boinc_data/boinc/slots/24/ICMSHhnw4+002568
Moving to projects directory: /boinc_data/boinc/slots/24/ICMUAhnw4+002568
  16:16:05 STEP 2569 H=2569:00 +CPU= 24.024
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMGGhnw4+002556
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMSHhnw4+002556
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMUAhnw4+002556
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMGGhnw4+002568
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMSHhnw4+002568
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMUAhnw4+002568
Zipping up the intermediate file: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_00jt_2019110100_123_993_12214151_0_r722857544_106.zip
Uploading the intermediate file: upload_file_106.zip
  16:16:30 STEP 2570 H=2570:00 +CPU= 24.893
  16:17:09 STEP 2571 H=2571:00 +CPU= 38.163
  16:17:35 STEP 2572 H=2572:00 +CPU= 25.680
  16:17:59 STEP 2573 H=2573:00 +CPU= 24.533
  16:18:36 STEP 2574 H=2574:00 +CPU= 36.905
  16:19:01 STEP 2575 H=2575:00 +CPU= 23.945
  16:19:25 STEP 2576 H=2576:00 +CPU= 24.252
  16:20:00 STEP 2577 H=2577:00 +CPU= 35.227
  16:20:25 STEP 2578 H=2578:00 +CPU= 24.107
  16:20:49 STEP 2579 H=2579:00 +CPU= 23.836
  16:21:28 STEP 2580 H=2580:00 +CPU= 38.910
Moving to projects directory: /boinc_data/boinc/slots/24/ICMGGhnw4+002580
Moving to projects directory: /boinc_data/boinc/slots/24/ICMSHhnw4+002580
Moving to projects directory: /boinc_data/boinc/slots/24/ICMUAhnw4+002580
  16:21:53 STEP 2581 H=2581:00 +CPU= 24.886
  16:22:18 STEP 2582 H=2582:00 +CPU= 25.453
  16:22:55 STEP 2583 H=2583:00 +CPU= 36.257
  16:23:20 STEP 2584 H=2584:00 +CPU= 24.905
  16:23:45 STEP 2585 H=2585:00 +CPU= 24.908
  16:24:23 STEP 2586 H=2586:00 +CPU= 38.360
  16:24:49 STEP 2587 H=2587:00 +CPU= 25.070
  16:25:14 STEP 2588 H=2588:00 +CPU= 25.367
  16:25:52 STEP 2589 H=2589:00 +CPU= 38.003
  16:26:19 STEP 2590 H=2590:00 +CPU= 26.391
  16:26:46 STEP 2591 H=2591:00 +CPU= 26.570
  16:27:32 STEP 2592 H=2592:00 +CPU= 45.234
Moving to projects directory: /boinc_data/boinc/slots/24/ICMGGhnw4+002592
Moving to projects directory: /boinc_data/boinc/slots/24/ICMSHhnw4+002592
Moving to projects directory: /boinc_data/boinc/slots/24/ICMUAhnw4+002592
  16:27:57 STEP 2593 H=2593:00 +CPU= 24.812
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMGGhnw4+002580
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMSHhnw4+002580
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMUAhnw4+002580
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMGGhnw4+002592
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMSHhnw4+002592
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMUAhnw4+002592
Zipping up the intermediate file: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_00jt_2019110100_123_993_12214151_0_r722857544_107.zip
Uploading the intermediate file: upload_file_107.zip
  16:28:23 STEP 2594 H=2594:00 +CPU= 26.062
  16:29:01 STEP 2595 H=2595:00 +CPU= 37.617
  16:29:26 STEP 2596 H=2596:00 +CPU= 25.391
  16:29:51 STEP 2597 H=2597:00 +CPU= 25.231
  16:30:31 STEP 2598 H=2598:00 +CPU= 38.813
  16:30:56 STEP 2599 H=2599:00 +CPU= 25.169
  16:31:21 STEP 2600 H=2600:00 +CPU= 24.583
  16:31:58 STEP 2601 H=2601:00 +CPU= 37.159
  16:32:23 STEP 2602 H=2602:00 +CPU= 25.028
  16:32:47 STEP 2603 H=2603:00 +CPU= 24.317
  16:33:29 STEP 2604 H=2604:00 +CPU= 41.730
Moving to projects directory: /boinc_data/boinc/slots/24/ICMGGhnw4+002604
Moving to projects directory: /boinc_data/boinc/slots/24/ICMSHhnw4+002604
Moving to projects directory: /boinc_data/boinc/slots/24/ICMUAhnw4+002604
  16:33:54 STEP 2605 H=2605:00 +CPU= 24.667
  16:34:20 STEP 2606 H=2606:00 +CPU= 25.734
  16:34:58 STEP 2607 H=2607:00 +CPU= 38.133
  16:35:24 STEP 2608 H=2608:00 +CPU= 25.226
  16:35:49 STEP 2609 H=2609:00 +CPU= 24.745
  16:36:29 STEP 2610 H=2610:00 +CPU= 39.255
  16:36:55 STEP 2611 H=2611:00 +CPU= 24.815
  16:37:19 STEP 2612 H=2612:00 +CPU= 23.649
  16:37:56 STEP 2613 H=2613:00 +CPU= 36.558
  16:38:21 STEP 2614 H=2614:00 +CPU= 25.638
  16:38:46 STEP 2615 H=2615:00 +CPU= 24.992
  16:39:31 STEP 2616 H=2616:00 +CPU= 43.814
Moving to projects directory: /boinc_data/boinc/slots/24/ICMGGhnw4+002616
Moving to projects directory: /boinc_data/boinc/slots/24/ICMSHhnw4+002616
Moving to projects directory: /boinc_data/boinc/slots/24/ICMUAhnw4+002616
  16:39:55 STEP 2617 H=2617:00 +CPU= 24.306
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMGGhnw4+002604
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMSHhnw4+002604
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMUAhnw4+002604
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMGGhnw4+002616
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMSHhnw4+002616
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMUAhnw4+002616
Zipping up the intermediate file: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_00jt_2019110100_123_993_12214151_0_r722857544_108.zip
Uploading the intermediate file: upload_file_108.zip
  16:40:21 STEP 2618 H=2618:00 +CPU= 25.609
Uploading trickle at timestep: 9421200
  16:40:59 STEP 2619 H=2619:00 +CPU= 37.986
  16:41:24 STEP 2620 H=2620:00 +CPU= 25.402
  16:41:49 STEP 2621 H=2621:00 +CPU= 24.386
  16:42:27 STEP 2622 H=2622:00 +CPU= 38.216
  16:42:53 STEP 2623 H=2623:00 +CPU= 25.721
  16:43:19 STEP 2624 H=2624:00 +CPU= 25.579
  16:43:56 STEP 2625 H=2625:00 +CPU= 37.491
  16:44:22 STEP 2626 H=2626:00 +CPU= 25.419
  16:44:48 STEP 2627 H=2627:00 +CPU= 25.412
  16:45:33 STEP 2628 H=2628:00 +CPU= 44.601
Moving to projects directory: /boinc_data/boinc/slots/24/ICMGGhnw4+002628
Moving to projects directory: /boinc_data/boinc/slots/24/ICMSHhnw4+002628
Moving to projects directory: /boinc_data/boinc/slots/24/ICMUAhnw4+002628
  16:45:59 STEP 2629 H=2629:00 +CPU= 25.877
  16:46:25 STEP 2630 H=2630:00 +CPU= 26.213
  16:47:02 STEP 2631 H=2631:00 +CPU= 37.173
  16:47:27 STEP 2632 H=2632:00 +CPU= 24.212
  16:47:50 STEP 2633 H=2633:00 +CPU= 23.004
  16:48:28 STEP 2634 H=2634:00 +CPU= 37.630
  16:48:52 STEP 2635 H=2635:00 +CPU= 24.306
  16:49:18 STEP 2636 H=2636:00 +CPU= 25.427
  16:49:55 STEP 2637 H=2637:00 +CPU= 36.609
  16:50:21 STEP 2638 H=2638:00 +CPU= 25.498
  16:50:45 STEP 2639 H=2639:00 +CPU= 24.590
  16:51:30 STEP 2640 H=2640:00 +CPU= 43.982
Moving to projects directory: /boinc_data/boinc/slots/24/ICMGGhnw4+002640
Moving to projects directory: /boinc_data/boinc/slots/24/ICMSHhnw4+002640
Moving to projects directory: /boinc_data/boinc/slots/24/ICMUAhnw4+002640
  16:51:55 STEP 2641 H=2641:00 +CPU= 24.940
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMGGhnw4+002628
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMSHhnw4+002628
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMUAhnw4+002628
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMGGhnw4+002640
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMSHhnw4+002640
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMUAhnw4+002640
Zipping up the intermediate file: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_00jt_2019110100_123_993_12214151_0_r722857544_109.zip
Uploading the intermediate file: upload_file_109.zip
  16:52:20 STEP 2642 H=2642:00 +CPU= 25.166
  16:52:57 STEP 2643 H=2643:00 +CPU= 36.619
  16:53:22 STEP 2644 H=2644:00 +CPU= 25.314
  16:53:47 STEP 2645 H=2645:00 +CPU= 25.055
  16:54:25 STEP 2646 H=2646:00 +CPU= 37.795
  16:54:50 STEP 2647 H=2647:00 +CPU= 24.807
  16:55:16 STEP 2648 H=2648:00 +CPU= 25.529
  16:55:52 STEP 2649 H=2649:00 +CPU= 35.375
  16:56:17 STEP 2650 H=2650:00 +CPU= 24.572
  16:56:40 STEP 2651 H=2651:00 +CPU= 23.656
  16:57:20 STEP 2652 H=2652:00 +CPU= 39.752
Moving to projects directory: /boinc_data/boinc/slots/24/ICMGGhnw4+002652
Moving to projects directory: /boinc_data/boinc/slots/24/ICMSHhnw4+002652
Moving to projects directory: /boinc_data/boinc/slots/24/ICMUAhnw4+002652
  16:57:45 STEP 2653 H=2653:00 +CPU= 24.462
  16:58:11 STEP 2654 H=2654:00 +CPU= 25.976
  16:58:48 STEP 2655 H=2655:00 +CPU= 36.895
  16:59:12 STEP 2656 H=2656:00 +CPU= 24.180
  16:59:36 STEP 2657 H=2657:00 +CPU= 23.755
  17:00:14 STEP 2658 H=2658:00 +CPU= 37.398
  17:00:38 STEP 2659 H=2659:00 +CPU= 24.439
  17:01:03 STEP 2660 H=2660:00 +CPU= 24.116
  17:01:39 STEP 2661 H=2661:00 +CPU= 35.814
  17:02:04 STEP 2662 H=2662:00 +CPU= 24.866
  17:02:29 STEP 2663 H=2663:00 +CPU= 24.695
  17:03:12 STEP 2664 H=2664:00 +CPU= 43.065
Moving to projects directory: /boinc_data/boinc/slots/24/ICMGGhnw4+002664
Moving to projects directory: /boinc_data/boinc/slots/24/ICMSHhnw4+002664
Moving to projects directory: /boinc_data/boinc/slots/24/ICMUAhnw4+002664
  17:03:37 STEP 2665 H=2665:00 +CPU= 24.833
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMGGhnw4+002652
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMSHhnw4+002652
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMUAhnw4+002652
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMGGhnw4+002664
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMSHhnw4+002664
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMUAhnw4+002664
Zipping up the intermediate file: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_00jt_2019110100_123_993_12214151_0_r722857544_110.zip
Uploading the intermediate file: upload_file_110.zip
  17:04:02 STEP 2666 H=2666:00 +CPU= 25.108
  17:04:40 STEP 2667 H=2667:00 +CPU= 37.055
  17:05:06 STEP 2668 H=2668:00 +CPU= 26.033
  17:05:32 STEP 2669 H=2669:00 +CPU= 25.745
  17:06:10 STEP 2670 H=2670:00 +CPU= 37.758
  17:06:34 STEP 2671 H=2671:00 +CPU= 24.173
  17:06:59 STEP 2672 H=2672:00 +CPU= 24.876
  17:07:38 STEP 2673 H=2673:00 +CPU= 38.696
  17:08:04 STEP 2674 H=2674:00 +CPU= 25.661
  17:08:27 STEP 2675 H=2675:00 +CPU= 23.252
  17:09:07 STEP 2676 H=2676:00 +CPU= 39.814
Moving to projects directory: /boinc_data/boinc/slots/24/ICMGGhnw4+002676
Moving to projects directory: /boinc_data/boinc/slots/24/ICMSHhnw4+002676
Moving to projects directory: /boinc_data/boinc/slots/24/ICMUAhnw4+002676
  17:09:30 STEP 2677 H=2677:00 +CPU= 22.952
  17:09:55 STEP 2678 H=2678:00 +CPU= 24.382
  17:10:33 STEP 2679 H=2679:00 +CPU= 37.478
  17:10:58 STEP 2680 H=2680:00 +CPU= 25.276
  17:11:22 STEP 2681 H=2681:00 +CPU= 23.706
  17:12:01 STEP 2682 H=2682:00 +CPU= 39.182
  17:12:26 STEP 2683 H=2683:00 +CPU= 24.708
  17:12:52 STEP 2684 H=2684:00 +CPU= 26.084
  17:13:29 STEP 2685 H=2685:00 +CPU= 37.054
  17:13:55 STEP 2686 H=2686:00 +CPU= 25.070
  17:14:20 STEP 2687 H=2687:00 +CPU= 25.387
  17:15:05 STEP 2688 H=2688:00 +CPU= 43.989
Moving to projects directory: /boinc_data/boinc/slots/24/ICMGGhnw4+002688
Moving to projects directory: /boinc_data/boinc/slots/24/ICMSHhnw4+002688
Moving to projects directory: /boinc_data/boinc/slots/24/ICMUAhnw4+002688
  17:15:30 STEP 2689 H=2689:00 +CPU= 24.550
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMGGhnw4+002676
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMSHhnw4+002676
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMUAhnw4+002676
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMGGhnw4+002688
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMSHhnw4+002688
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMUAhnw4+002688
Zipping up the intermediate file: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_00jt_2019110100_123_993_12214151_0_r722857544_111.zip
Uploading the intermediate file: upload_file_111.zip
  17:15:57 STEP 2690 H=2690:00 +CPU= 26.513
  17:16:35 STEP 2691 H=2691:00 +CPU= 37.768
  17:17:00 STEP 2692 H=2692:00 +CPU= 24.907
  17:17:25 STEP 2693 H=2693:00 +CPU= 24.258
  17:18:02 STEP 2694 H=2694:00 +CPU= 37.190
  17:18:27 STEP 2695 H=2695:00 +CPU= 24.660
  17:18:51 STEP 2696 H=2696:00 +CPU= 24.498
  17:19:29 STEP 2697 H=2697:00 +CPU= 37.028
  17:19:52 STEP 2698 H=2698:00 +CPU= 23.031
  17:20:17 STEP 2699 H=2699:00 +CPU= 25.132
  17:20:58 STEP 2700 H=2700:00 +CPU= 40.732
Moving to projects directory: /boinc_data/boinc/slots/24/ICMGGhnw4+002700
Moving to projects directory: /boinc_data/boinc/slots/24/ICMSHhnw4+002700
Moving to projects directory: /boinc_data/boinc/slots/24/ICMUAhnw4+002700
  17:21:22 STEP 2701 H=2701:00 +CPU= 24.084
  17:21:48 STEP 2702 H=2702:00 +CPU= 24.882
  17:22:25 STEP 2703 H=2703:00 +CPU= 37.646
  17:22:51 STEP 2704 H=2704:00 +CPU= 25.386
  17:23:15 STEP 2705 H=2705:00 +CPU= 23.996
  17:23:54 STEP 2706 H=2706:00 +CPU= 38.791
  17:24:19 STEP 2707 H=2707:00 +CPU= 24.468
  17:24:44 STEP 2708 H=2708:00 +CPU= 25.187
  17:25:21 STEP 2709 H=2709:00 +CPU= 36.653
  17:25:46 STEP 2710 H=2710:00 +CPU= 24.868
  17:26:11 STEP 2711 H=2711:00 +CPU= 25.020
  17:26:56 STEP 2712 H=2712:00 +CPU= 44.902
Moving to projects directory: /boinc_data/boinc/slots/24/ICMGGhnw4+002712
Moving to projects directory: /boinc_data/boinc/slots/24/ICMSHhnw4+002712
Moving to projects directory: /boinc_data/boinc/slots/24/ICMUAhnw4+002712
  17:27:21 STEP 2713 H=2713:00 +CPU= 24.206
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMGGhnw4+002700
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMSHhnw4+002700
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMUAhnw4+002700
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMGGhnw4+002712
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMSHhnw4+002712
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMUAhnw4+002712
Zipping up the intermediate file: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_00jt_2019110100_123_993_12214151_0_r722857544_112.zip
Uploading the intermediate file: upload_file_112.zip
  17:27:47 STEP 2714 H=2714:00 +CPU= 25.685
  17:28:24 STEP 2715 H=2715:00 +CPU= 37.119
  17:28:49 STEP 2716 H=2716:00 +CPU= 25.204
  17:29:14 STEP 2717 H=2717:00 +CPU= 23.979
  17:29:51 STEP 2718 H=2718:00 +CPU= 37.419
  17:30:16 STEP 2719 H=2719:00 +CPU= 24.073
  17:30:42 STEP 2720 H=2720:00 +CPU= 26.173
  17:31:21 STEP 2721 H=2721:00 +CPU= 39.251
  17:31:45 STEP 2722 H=2722:00 +CPU= 24.072
  17:32:09 STEP 2723 H=2723:00 +CPU= 23.257
  17:32:52 STEP 2724 H=2724:00 +CPU= 42.646
Moving to projects directory: /boinc_data/boinc/slots/24/ICMGGhnw4+002724
Moving to projects directory: /boinc_data/boinc/slots/24/ICMSHhnw4+002724
Moving to projects directory: /boinc_data/boinc/slots/24/ICMUAhnw4+002724
  17:33:17 STEP 2725 H=2725:00 +CPU= 25.085
  17:33:42 STEP 2726 H=2726:00 +CPU= 25.432
  17:34:20 STEP 2727 H=2727:00 +CPU= 37.474
  17:34:45 STEP 2728 H=2728:00 +CPU= 24.598
  17:35:09 STEP 2729 H=2729:00 +CPU= 24.183
  17:35:48 STEP 2730 H=2730:00 +CPU= 38.917
  17:36:13 STEP 2731 H=2731:00 +CPU= 24.989
  17:36:40 STEP 2732 H=2732:00 +CPU= 25.978
  17:37:18 STEP 2733 H=2733:00 +CPU= 38.200
  17:37:44 STEP 2734 H=2734:00 +CPU= 25.193
  17:38:09 STEP 2735 H=2735:00 +CPU= 25.674
  17:38:53 STEP 2736 H=2736:00 +CPU= 43.082
Moving to projects directory: /boinc_data/boinc/slots/24/ICMGGhnw4+002736
Moving to projects directory: /boinc_data/boinc/slots/24/ICMSHhnw4+002736
Moving to projects directory: /boinc_data/boinc/slots/24/ICMUAhnw4+002736
  17:39:18 STEP 2737 H=2737:00 +CPU= 25.363
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMGGhnw4+002724
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMSHhnw4+002724
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMUAhnw4+002724
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMGGhnw4+002736
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMSHhnw4+002736
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMUAhnw4+002736
Zipping up the intermediate file: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_00jt_2019110100_123_993_12214151_0_r722857544_113.zip
Uploading the intermediate file: upload_file_113.zip
  17:39:43 STEP 2738 H=2738:00 +CPU= 24.293
  17:40:20 STEP 2739 H=2739:00 +CPU= 36.776
  17:40:45 STEP 2740 H=2740:00 +CPU= 24.550
  17:41:09 STEP 2741 H=2741:00 +CPU= 24.119
  17:41:48 STEP 2742 H=2742:00 +CPU= 38.643
  17:42:14 STEP 2743 H=2743:00 +CPU= 25.431
  17:42:39 STEP 2744 H=2744:00 +CPU= 25.432
  17:43:17 STEP 2745 H=2745:00 +CPU= 37.783
  17:43:42 STEP 2746 H=2746:00 +CPU= 24.391
  17:44:06 STEP 2747 H=2747:00 +CPU= 24.417
  17:44:48 STEP 2748 H=2748:00 +CPU= 41.554
Moving to projects directory: /boinc_data/boinc/slots/24/ICMGGhnw4+002748
Moving to projects directory: /boinc_data/boinc/slots/24/ICMSHhnw4+002748
Moving to projects directory: /boinc_data/boinc/slots/24/ICMUAhnw4+002748
  17:45:13 STEP 2749 H=2749:00 +CPU= 24.301
  17:45:39 STEP 2750 H=2750:00 +CPU= 25.451
  17:46:17 STEP 2751 H=2751:00 +CPU= 37.834
  17:46:43 STEP 2752 H=2752:00 +CPU= 25.698
  17:47:08 STEP 2753 H=2753:00 +CPU= 25.195
  17:47:49 STEP 2754 H=2754:00 +CPU= 40.458
  17:48:14 STEP 2755 H=2755:00 +CPU= 24.949
  17:48:40 STEP 2756 H=2756:00 +CPU= 26.016
  17:49:19 STEP 2757 H=2757:00 +CPU= 38.234
  17:49:44 STEP 2758 H=2758:00 +CPU= 25.750
  17:50:10 STEP 2759 H=2759:00 +CPU= 25.746
  17:50:56 STEP 2760 H=2760:00 +CPU= 44.689
Moving to projects directory: /boinc_data/boinc/slots/24/ICMGGhnw4+002760
Moving to projects directory: /boinc_data/boinc/slots/24/ICMSHhnw4+002760
Moving to projects directory: /boinc_data/boinc/slots/24/ICMUAhnw4+002760
  17:51:21 STEP 2761 H=2761:00 +CPU= 24.984
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMGGhnw4+002748
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMSHhnw4+002748
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMUAhnw4+002748
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMGGhnw4+002760
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMSHhnw4+002760
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMUAhnw4+002760
Zipping up the intermediate file: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_00jt_2019110100_123_993_12214151_0_r722857544_114.zip
Uploading the intermediate file: upload_file_114.zip
  17:51:46 STEP 2762 H=2762:00 +CPU= 25.136
  17:52:25 STEP 2763 H=2763:00 +CPU= 38.347
  17:52:50 STEP 2764 H=2764:00 +CPU= 25.212
  17:53:15 STEP 2765 H=2765:00 +CPU= 24.436
  17:53:52 STEP 2766 H=2766:00 +CPU= 37.648
  17:54:17 STEP 2767 H=2767:00 +CPU= 24.899
  17:54:43 STEP 2768 H=2768:00 +CPU= 25.165
  17:55:21 STEP 2769 H=2769:00 +CPU= 38.418
  17:55:47 STEP 2770 H=2770:00 +CPU= 25.155
  17:56:12 STEP 2771 H=2771:00 +CPU= 25.182
  17:56:54 STEP 2772 H=2772:00 +CPU= 41.246
Moving to projects directory: /boinc_data/boinc/slots/24/ICMGGhnw4+002772
Moving to projects directory: /boinc_data/boinc/slots/24/ICMSHhnw4+002772
Moving to projects directory: /boinc_data/boinc/slots/24/ICMUAhnw4+002772
  17:57:20 STEP 2773 H=2773:00 +CPU= 25.605
  17:57:47 STEP 2774 H=2774:00 +CPU= 26.767
  17:58:25 STEP 2775 H=2775:00 +CPU= 38.772
  17:58:50 STEP 2776 H=2776:00 +CPU= 24.458
  17:59:13 STEP 2777 H=2777:00 +CPU= 23.289
  17:59:51 STEP 2778 H=2778:00 +CPU= 37.443
  18:00:16 STEP 2779 H=2779:00 +CPU= 24.880
  18:00:42 STEP 2780 H=2780:00 +CPU= 25.809
  18:01:20 STEP 2781 H=2781:00 +CPU= 37.596
  18:01:45 STEP 2782 H=2782:00 +CPU= 24.870
  18:02:11 STEP 2783 H=2783:00 +CPU= 25.500
  18:02:56 STEP 2784 H=2784:00 +CPU= 45.182
Moving to projects directory: /boinc_data/boinc/slots/24/ICMGGhnw4+002784
Moving to projects directory: /boinc_data/boinc/slots/24/ICMSHhnw4+002784
Moving to projects directory: /boinc_data/boinc/slots/24/ICMUAhnw4+002784
  18:03:20 STEP 2785 H=2785:00 +CPU= 24.265
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMGGhnw4+002772
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMSHhnw4+002772
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMUAhnw4+002772
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMGGhnw4+002784
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMSHhnw4+002784
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMUAhnw4+002784
Zipping up the intermediate file: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_00jt_2019110100_123_993_12214151_0_r722857544_115.zip
Uploading the intermediate file: upload_file_115.zip
  18:03:45 STEP 2786 H=2786:00 +CPU= 24.823
  18:04:22 STEP 2787 H=2787:00 +CPU= 36.857
  18:04:47 STEP 2788 H=2788:00 +CPU= 24.341
  18:05:11 STEP 2789 H=2789:00 +CPU= 24.175
  18:05:48 STEP 2790 H=2790:00 +CPU= 36.525
  18:06:13 STEP 2791 H=2791:00 +CPU= 24.989
  18:06:39 STEP 2792 H=2792:00 +CPU= 25.750
  18:07:19 STEP 2793 H=2793:00 +CPU= 38.788
  18:07:45 STEP 2794 H=2794:00 +CPU= 25.924
  18:08:10 STEP 2795 H=2795:00 +CPU= 24.823
  18:08:51 STEP 2796 H=2796:00 +CPU= 40.676
Moving to projects directory: /boinc_data/boinc/slots/24/ICMGGhnw4+002796
Moving to projects directory: /boinc_data/boinc/slots/24/ICMSHhnw4+002796
Moving to projects directory: /boinc_data/boinc/slots/24/ICMUAhnw4+002796
  18:09:16 STEP 2797 H=2797:00 +CPU= 25.310
  18:09:41 STEP 2798 H=2798:00 +CPU= 25.061
  18:10:20 STEP 2799 H=2799:00 +CPU= 37.752
  18:10:45 STEP 2800 H=2800:00 +CPU= 25.505
  18:11:09 STEP 2801 H=2801:00 +CPU= 23.315
  18:11:47 STEP 2802 H=2802:00 +CPU= 38.603
  18:12:13 STEP 2803 H=2803:00 +CPU= 25.184
  18:12:38 STEP 2804 H=2804:00 +CPU= 25.656
  18:13:15 STEP 2805 H=2805:00 +CPU= 36.650
  18:13:40 STEP 2806 H=2806:00 +CPU= 24.385
  18:14:05 STEP 2807 H=2807:00 +CPU= 25.618
  18:14:49 STEP 2808 H=2808:00 +CPU= 43.497
Moving to projects directory: /boinc_data/boinc/slots/24/ICMGGhnw4+002808
Moving to projects directory: /boinc_data/boinc/slots/24/ICMSHhnw4+002808
Moving to projects directory: /boinc_data/boinc/slots/24/ICMUAhnw4+002808
  18:15:13 STEP 2809 H=2809:00 +CPU= 23.463
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMGGhnw4+002796
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMSHhnw4+002796
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMUAhnw4+002796
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMGGhnw4+002808
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMSHhnw4+002808
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMUAhnw4+002808
Zipping up the intermediate file: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_00jt_2019110100_123_993_12214151_0_r722857544_116.zip
Uploading the intermediate file: upload_file_116.zip
  18:15:37 STEP 2810 H=2810:00 +CPU= 24.252
  18:16:15 STEP 2811 H=2811:00 +CPU= 37.388
  18:16:42 STEP 2812 H=2812:00 +CPU= 25.565
  18:17:08 STEP 2813 H=2813:00 +CPU= 24.880
  18:17:47 STEP 2814 H=2814:00 +CPU= 39.446
  18:18:13 STEP 2815 H=2815:00 +CPU= 25.588
  18:18:39 STEP 2816 H=2816:00 +CPU= 25.456
  18:19:16 STEP 2817 H=2817:00 +CPU= 37.510
  18:19:42 STEP 2818 H=2818:00 +CPU= 25.477
  18:20:05 STEP 2819 H=2819:00 +CPU= 23.057
  18:20:44 STEP 2820 H=2820:00 +CPU= 39.032
Moving to projects directory: /boinc_data/boinc/slots/24/ICMGGhnw4+002820
Moving to projects directory: /boinc_data/boinc/slots/24/ICMSHhnw4+002820
Moving to projects directory: /boinc_data/boinc/slots/24/ICMUAhnw4+002820
  18:21:08 STEP 2821 H=2821:00 +CPU= 23.810
  18:21:32 STEP 2822 H=2822:00 +CPU= 23.841
  18:22:09 STEP 2823 H=2823:00 +CPU= 36.311
  18:22:34 STEP 2824 H=2824:00 +CPU= 25.263
  18:22:59 STEP 2825 H=2825:00 +CPU= 24.174
  18:23:37 STEP 2826 H=2826:00 +CPU= 38.235
  18:24:01 STEP 2827 H=2827:00 +CPU= 24.146
  18:24:26 STEP 2828 H=2828:00 +CPU= 25.035
  18:25:06 STEP 2829 H=2829:00 +CPU= 39.145
  18:25:33 STEP 2830 H=2830:00 +CPU= 26.692
  18:26:00 STEP 2831 H=2831:00 +CPU= 26.412
  18:26:42 STEP 2832 H=2832:00 +CPU= 42.437
Moving to projects directory: /boinc_data/boinc/slots/24/ICMGGhnw4+002832
Moving to projects directory: /boinc_data/boinc/slots/24/ICMSHhnw4+002832
Moving to projects directory: /boinc_data/boinc/slots/24/ICMUAhnw4+002832
  18:27:07 STEP 2833 H=2833:00 +CPU= 24.386
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMGGhnw4+002820
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMSHhnw4+002820
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMUAhnw4+002820
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMGGhnw4+002832
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMSHhnw4+002832
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMUAhnw4+002832
Zipping up the intermediate file: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_00jt_2019110100_123_993_12214151_0_r722857544_117.zip
Uploading the intermediate file: upload_file_117.zip
  18:27:32 STEP 2834 H=2834:00 +CPU= 24.602
  18:28:11 STEP 2835 H=2835:00 +CPU= 38.731
  18:28:37 STEP 2836 H=2836:00 +CPU= 26.334
  18:29:01 STEP 2837 H=2837:00 +CPU= 24.108
  18:29:40 STEP 2838 H=2838:00 +CPU= 38.046
  18:30:03 STEP 2839 H=2839:00 +CPU= 23.760
  18:30:29 STEP 2840 H=2840:00 +CPU= 25.833
  18:31:07 STEP 2841 H=2841:00 +CPU= 37.713
  18:31:32 STEP 2842 H=2842:00 +CPU= 24.687
  18:31:57 STEP 2843 H=2843:00 +CPU= 24.411
  18:32:38 STEP 2844 H=2844:00 +CPU= 40.734
Moving to projects directory: /boinc_data/boinc/slots/24/ICMGGhnw4+002844
Moving to projects directory: /boinc_data/boinc/slots/24/ICMSHhnw4+002844
Moving to projects directory: /boinc_data/boinc/slots/24/ICMUAhnw4+002844
  18:33:03 STEP 2845 H=2845:00 +CPU= 25.057
  18:33:29 STEP 2846 H=2846:00 +CPU= 25.456
  18:34:07 STEP 2847 H=2847:00 +CPU= 38.100
  18:34:33 STEP 2848 H=2848:00 +CPU= 25.678
  18:34:59 STEP 2849 H=2849:00 +CPU= 25.857
  18:35:40 STEP 2850 H=2850:00 +CPU= 40.026
  18:36:05 STEP 2851 H=2851:00 +CPU= 24.785
  18:36:29 STEP 2852 H=2852:00 +CPU= 24.785
  18:37:07 STEP 2853 H=2853:00 +CPU= 36.944
  18:37:33 STEP 2854 H=2854:00 +CPU= 26.261
  18:37:57 STEP 2855 H=2855:00 +CPU= 24.024
  18:38:42 STEP 2856 H=2856:00 +CPU= 44.556
Moving to projects directory: /boinc_data/boinc/slots/24/ICMGGhnw4+002856
Moving to projects directory: /boinc_data/boinc/slots/24/ICMSHhnw4+002856
Moving to projects directory: /boinc_data/boinc/slots/24/ICMUAhnw4+002856
  18:39:08 STEP 2857 H=2857:00 +CPU= 25.592
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMGGhnw4+002844
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMSHhnw4+002844
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMUAhnw4+002844
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMGGhnw4+002856
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMSHhnw4+002856
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMUAhnw4+002856
Zipping up the intermediate file: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_00jt_2019110100_123_993_12214151_0_r722857544_118.zip
Uploading the intermediate file: upload_file_118.zip
  18:39:33 STEP 2858 H=2858:00 +CPU= 25.262
Uploading trickle at timestep: 10285200
  18:40:11 STEP 2859 H=2859:00 +CPU= 38.113
  18:40:37 STEP 2860 H=2860:00 +CPU= 25.714
  18:41:02 STEP 2861 H=2861:00 +CPU= 24.496
  18:41:40 STEP 2862 H=2862:00 +CPU= 38.281
  18:42:05 STEP 2863 H=2863:00 +CPU= 24.587
  18:42:30 STEP 2864 H=2864:00 +CPU= 24.964
  18:43:07 STEP 2865 H=2865:00 +CPU= 36.773
  18:43:32 STEP 2866 H=2866:00 +CPU= 24.916
  18:43:56 STEP 2867 H=2867:00 +CPU= 23.964
  18:44:37 STEP 2868 H=2868:00 +CPU= 40.514
Moving to projects directory: /boinc_data/boinc/slots/24/ICMGGhnw4+002868
Moving to projects directory: /boinc_data/boinc/slots/24/ICMSHhnw4+002868
Moving to projects directory: /boinc_data/boinc/slots/24/ICMUAhnw4+002868
  18:45:02 STEP 2869 H=2869:00 +CPU= 24.137
  18:45:26 STEP 2870 H=2870:00 +CPU= 24.620
  18:46:05 STEP 2871 H=2871:00 +CPU= 37.674
  18:46:33 STEP 2872 H=2872:00 +CPU= 27.332
  18:46:59 STEP 2873 H=2873:00 +CPU= 26.254
  18:47:38 STEP 2874 H=2874:00 +CPU= 38.834
  18:48:04 STEP 2875 H=2875:00 +CPU= 25.262
  18:48:30 STEP 2876 H=2876:00 +CPU= 25.456
  18:49:07 STEP 2877 H=2877:00 +CPU= 37.033
  18:49:30 STEP 2878 H=2878:00 +CPU= 23.523
  18:49:56 STEP 2879 H=2879:00 +CPU= 25.439
  18:50:40 STEP 2880 H=2880:00 +CPU= 43.235
Moving to projects directory: /boinc_data/boinc/slots/24/ICMGGhnw4+002880
Moving to projects directory: /boinc_data/boinc/slots/24/ICMSHhnw4+002880
Moving to projects directory: /boinc_data/boinc/slots/24/ICMUAhnw4+002880
  18:51:05 STEP 2881 H=2881:00 +CPU= 24.869
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMGGhnw4+002868
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMSHhnw4+002868
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMUAhnw4+002868
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMGGhnw4+002880
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMSHhnw4+002880
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMUAhnw4+002880
Zipping up the intermediate file: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_00jt_2019110100_123_993_12214151_0_r722857544_119.zip
Uploading the intermediate file: upload_file_119.zip
  18:51:29 STEP 2882 H=2882:00 +CPU= 24.244
  18:52:07 STEP 2883 H=2883:00 +CPU= 37.715
  18:52:33 STEP 2884 H=2884:00 +CPU= 25.305
  18:52:58 STEP 2885 H=2885:00 +CPU= 25.740
  18:53:38 STEP 2886 H=2886:00 +CPU= 39.564
  18:54:04 STEP 2887 H=2887:00 +CPU= 25.556
  18:54:30 STEP 2888 H=2888:00 +CPU= 25.791
  18:55:07 STEP 2889 H=2889:00 +CPU= 37.247
  18:55:34 STEP 2890 H=2890:00 +CPU= 25.931
  18:55:59 STEP 2891 H=2891:00 +CPU= 25.416
  18:56:42 STEP 2892 H=2892:00 +CPU= 43.022
Moving to projects directory: /boinc_data/boinc/slots/24/ICMGGhnw4+002892
Moving to projects directory: /boinc_data/boinc/slots/24/ICMSHhnw4+002892
Moving to projects directory: /boinc_data/boinc/slots/24/ICMUAhnw4+002892
  18:57:06 STEP 2893 H=2893:00 +CPU= 24.125
  18:57:33 STEP 2894 H=2894:00 +CPU= 26.627
  18:58:11 STEP 2895 H=2895:00 +CPU= 37.884
  18:58:36 STEP 2896 H=2896:00 +CPU= 24.845
  18:59:02 STEP 2897 H=2897:00 +CPU= 25.392
  18:59:40 STEP 2898 H=2898:00 +CPU= 38.478
  19:00:07 STEP 2899 H=2899:00 +CPU= 26.229
  19:00:32 STEP 2900 H=2900:00 +CPU= 25.768
  19:01:11 STEP 2901 H=2901:00 +CPU= 38.378
  19:01:36 STEP 2902 H=2902:00 +CPU= 24.964
  19:02:01 STEP 2903 H=2903:00 +CPU= 24.982
  19:02:46 STEP 2904 H=2904:00 +CPU= 44.425
Moving to projects directory: /boinc_data/boinc/slots/24/ICMGGhnw4+002904
Moving to projects directory: /boinc_data/boinc/slots/24/ICMSHhnw4+002904
Moving to projects directory: /boinc_data/boinc/slots/24/ICMUAhnw4+002904
  19:03:10 STEP 2905 H=2905:00 +CPU= 24.179
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMGGhnw4+002892
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMSHhnw4+002892
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMUAhnw4+002892
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMGGhnw4+002904
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMSHhnw4+002904
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMUAhnw4+002904
Zipping up the intermediate file: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_00jt_2019110100_123_993_12214151_0_r722857544_120.zip
Uploading the intermediate file: upload_file_120.zip
  19:03:36 STEP 2906 H=2906:00 +CPU= 25.365
  19:04:13 STEP 2907 H=2907:00 +CPU= 37.062
  19:04:38 STEP 2908 H=2908:00 +CPU= 24.914
  19:05:03 STEP 2909 H=2909:00 +CPU= 24.793
  19:05:42 STEP 2910 H=2910:00 +CPU= 38.997
  19:06:09 STEP 2911 H=2911:00 +CPU= 26.295
  19:06:35 STEP 2912 H=2912:00 +CPU= 25.909
  19:07:13 STEP 2913 H=2913:00 +CPU= 37.644
  19:07:38 STEP 2914 H=2914:00 +CPU= 25.199
  19:08:02 STEP 2915 H=2915:00 +CPU= 23.737
  19:08:43 STEP 2916 H=2916:00 +CPU= 40.212
Moving to projects directory: /boinc_data/boinc/slots/24/ICMGGhnw4+002916
Moving to projects directory: /boinc_data/boinc/slots/24/ICMSHhnw4+002916
Moving to projects directory: /boinc_data/boinc/slots/24/ICMUAhnw4+002916
  19:09:07 STEP 2917 H=2917:00 +CPU= 24.393
  19:09:36 STEP 2918 H=2918:00 +CPU= 27.922
  19:10:17 STEP 2919 H=2919:00 +CPU= 40.203
  19:10:44 STEP 2920 H=2920:00 +CPU= 26.670
  19:11:09 STEP 2921 H=2921:00 +CPU= 24.876
  19:11:47 STEP 2922 H=2922:00 +CPU= 38.391
  19:12:13 STEP 2923 H=2923:00 +CPU= 24.939
  19:12:38 STEP 2924 H=2924:00 +CPU= 25.151
  19:13:18 STEP 2925 H=2925:00 +CPU= 39.828
  19:13:45 STEP 2926 H=2926:00 +CPU= 26.765
  19:14:10 STEP 2927 H=2927:00 +CPU= 25.170
  19:14:55 STEP 2928 H=2928:00 +CPU= 44.190
Moving to projects directory: /boinc_data/boinc/slots/24/ICMGGhnw4+002928
Moving to projects directory: /boinc_data/boinc/slots/24/ICMSHhnw4+002928
Moving to projects directory: /boinc_data/boinc/slots/24/ICMUAhnw4+002928
  19:15:19 STEP 2929 H=2929:00 +CPU= 24.743
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMGGhnw4+002916
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMSHhnw4+002916
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMUAhnw4+002916
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMGGhnw4+002928
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMSHhnw4+002928
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMUAhnw4+002928
Zipping up the intermediate file: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_00jt_2019110100_123_993_12214151_0_r722857544_121.zip
Uploading the intermediate file: upload_file_121.zip
  19:15:45 STEP 2930 H=2930:00 +CPU= 25.400
  19:16:24 STEP 2931 H=2931:00 +CPU= 38.350
  19:16:49 STEP 2932 H=2932:00 +CPU= 25.649
  19:17:14 STEP 2933 H=2933:00 +CPU= 24.540
  19:17:52 STEP 2934 H=2934:00 +CPU= 38.197
  19:18:16 STEP 2935 H=2935:00 +CPU= 23.986
  19:18:43 STEP 2936 H=2936:00 +CPU= 26.083
  19:19:22 STEP 2937 H=2937:00 +CPU= 39.562
  19:19:48 STEP 2938 H=2938:00 +CPU= 25.460
  19:20:14 STEP 2939 H=2939:00 +CPU= 25.652
  19:20:58 STEP 2940 H=2940:00 +CPU= 42.906
Moving to projects directory: /boinc_data/boinc/slots/24/ICMGGhnw4+002940
Moving to projects directory: /boinc_data/boinc/slots/24/ICMSHhnw4+002940
Moving to projects directory: /boinc_data/boinc/slots/24/ICMUAhnw4+002940
  19:21:23 STEP 2941 H=2941:00 +CPU= 25.261
  19:21:48 STEP 2942 H=2942:00 +CPU= 24.688
  19:22:27 STEP 2943 H=2943:00 +CPU= 38.888
  19:22:54 STEP 2944 H=2944:00 +CPU= 27.044
  19:23:21 STEP 2945 H=2945:00 +CPU= 26.280
  19:24:01 STEP 2946 H=2946:00 +CPU= 40.008
  19:24:25 STEP 2947 H=2947:00 +CPU= 23.943
  19:24:51 STEP 2948 H=2948:00 +CPU= 25.599
  19:25:28 STEP 2949 H=2949:00 +CPU= 36.944
  19:25:53 STEP 2950 H=2950:00 +CPU= 24.657
  19:26:18 STEP 2951 H=2951:00 +CPU= 25.447
  19:27:03 STEP 2952 H=2952:00 +CPU= 44.220
..The child process terminated with status: 0
>>> Printing last 70 lines from file: NODE.001_01

 STEPO, YSPSDT_AR1     (  1,...), SP NORMS 2D: 4.865283731608E-01 1.865286143845E-01 6.362598939121E-02
 STEPO, YSPSDT_AR1     (SUM,...), SP NORMS 2D: 5.260318722649E-01
 STEPO, YGPSDT         (  1,...), Grid Point Norms
  FIELD LEV        AVERAGE               MINIMUM               MAXIMUM       
    101  60  3.48989191816039E-02 -1.72994898449055E+00  1.89761310057317E+00
 STEPO, YGPSDT[clipped](  1,...), Grid Point Norms
  FIELD LEV        AVERAGE               MINIMUM               MAXIMUM       
    101  60  3.19925486346338E-02 -1.00000000000000E+00  1.00000000000000E+00
  MAXGPFV : MAX. VALUE =    1.17214994980834     
  MAXGPFV : MAX. VALUE =   0.748229716469210     
 NSTEP =  2952 SCAN2M_HPOS  P
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ICMGGhnw4+002952 MODE=a
 IO-STREAM CLOSED - ICMGGhnw4+002952
 MPI-TASK:   1 -    348003146 BYTES IN      8 RECORDS TRANSFERRED IN    0.0008 SECONDS435003.9 Mbytes/s, TOTAL TIME=    0.0123( 6.5%)
   !!!!!!!!!!!!!! WAVE FIELDS INTEGRATED.  DATE IS: 20200302231500  !!!!!!!!!!!! 
   !!!!!!!!!!!!!! WAVE FIELDS INTEGRATED.  DATE IS: 20200302233000  !!!!!!!!!!!! 
   !!!!!!!!!!!!!! WAVE FIELDS INTEGRATED.  DATE IS: 20200302234500  !!!!!!!!!!!! 
   !!!!!!!!!!!!!! WAVE FIELDS INTEGRATED.  DATE IS: 20200303000000  !!!!!!!!!!!! 
 19:27:03 STEP 2952 H=2952:00 +CPU= 44.220
 END CNT3
 NSTEP =  2952 CNT0   000000000  38.737  38.737  39.018   0.993               0               0               0               0               0
IO-STREAM STATISTICS, TOTAL NO OF REC/BYTES READ           35/   804992476 WRITTEN         8579/************
 -TASK-OPENED-OPEN-RECS IN -KBYTE IN    -RECS OUT-KBYTE OUT-WALL   -WALL IN-WALL
  OU-TOT IN -TOT OUT
   1    210    0       35       786125     8579 ************    48.1     1.2    46.9     1.8    69.2
===-=== START OF TIMING STATISTICS ===-===
 
STATS FOR ALL TASKS
 NUM ROUTINE                                     CALLS  MEAN(ms)   MAX(ms)   FRAC(%)  UNBAL(%)
   0 CNT0     - COMPLETE EXECUTION                   1 ********* *********    100.00      0.00
   1 CNT4     - FORWARD INTEGRATION                  1 ********* *********     99.98      0.00
   8 SCAN2M - GRID-POINT DYNAMICS                  599    2923.4    2923.4     10.66      0.00
   9 SPCM     - SPECTRAL COMP.                     552     346.3     346.3      1.16      0.00
  10 SCAN2M - PHYSICS                              553   12583.8   12583.8     42.35      0.00
  11 IOPACK   - OUTPUT P.P. RESULTS                 46    1419.5    1419.5      0.40      0.00
  12 SPNORM   - SPECTRAL NORM COMP.                 24      18.2      18.2      0.00      0.00
  13 SCAN2M - RADIATION CALC.                      185   12916.9   12916.9     14.54      0.00
  17 GRIDFPOS IN CNT4                               46      54.4      54.4      0.02      0.00
  27 RESTART FILES - WRITING                        23    2838.9    2838.9      0.40      0.00
  28 RESTART FILES - READING                         1    2381.8    2381.8      0.01      0.00
  29 SU4FPOS IN CNT4                                46       0.1       0.1      0.00      0.00
  30 DYNFPOS IN CNT4                                46    2477.5    2477.5      0.69      0.00
  31 POSDDH IN STEPO                                 3       1.0       1.0      0.00      0.00
  37 CPGLAG   - SL COMPUTATIONS                    553    4034.1    4034.1     13.58      0.00
  38 WAM      - TOTAL COST OF WAVE MODEL           553    3145.0    3145.0     10.58      0.00
  39 SU0YOMB                                         1     306.8     306.8      0.00      0.00
  51 SCAN2M   - SL COMM. PART 1                    553      15.4      15.4      0.05      0.00
  54 SPCM     - M TO S/S TO M TRANSP.              552     230.9     230.9      0.78      0.00
  55 SPCIMPF  - S TO M/M TO S TRANSP.              552      30.2      30.2      0.10      0.00
  56 SPNORM   - SPECTRAL NORM COMM.                 24       0.1       0.1      0.00      0.00
 102 LTINV_CTL   - INVERSE LEGENDRE TRANSFORM     1892     172.7     172.7      1.99      0.00
 103 LTDIR_CTL   - DIRECT LEGENDRE TRANSFORM      1150     159.0     159.0      1.11      0.00
 106 FTDIR_CTL   - DIRECT FOURIER TRANSFORM       1150      65.1      65.1      0.46      0.00
 107 FTINV_CTL   - INVERSE FOURIER TRANSFORM      1892      52.8      52.8      0.61      0.00
 140 SULEG       - COMP. OF LEGENDRE POL.            2      24.1      24.1      0.00      0.00
 152 LTINV_CTL   - M TO L TRANSPOSITION           1892      22.2      22.2      0.26      0.00
 153 LTDIR_CTL   - L TO M TRANSPOSITION           1150      18.9      18.9      0.13      0.00
 157 FTINV_CTL   - L TO G TRANSPOSITION           1892      28.0      28.0      0.32      0.00
 158 FTDIR_CTL   - G TO L TRANSPOSITION           1150      21.6      21.6      0.15      0.00
 400 GSTATS                                     110985       0.0       0.0      0.00      0.00
 401 GSTATS HOOK                                106239       0.0       0.0      0.00      0.00
TOTAL MEASURED IMBALANCE =       0.0 SECONDS,  0.0 PERCENT
TOTAL WALLCLOCK TIME    16431.4 CPU TIME   16320.0 VECTOR TIME    16320.0


===-=== END   OF TIMING STATISTICS ===-===
FORECAST DAYS PER DAY      646.9
  *** END CNT0 *** 
------------------------------------------------
Moving to projects directory: /boinc_data/boinc/slots/24/ICMGGhnw4+002952
Moving to projects directory: /boinc_data/boinc/slots/24/ICMSHhnw4+002952
Moving to projects directory: /boinc_data/boinc/slots/24/ICMUAhnw4+002952
Adding to the zip: /boinc_data/boinc/slots/24/NODE.001_01
Adding to the zip: /boinc_data/boinc/slots/24/ifs.stat
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMUAhnw4+002952
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMGGhnw4+002940
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMGGhnw4+002952
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMUAhnw4+002940
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMGGhnw4+002400
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMSHhnw4+002940
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_12214151/ICMSHhnw4+002952
Zipping up the final file: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_00jt_2019110100_123_993_12214151_0_r722857544_122.zip
Uploading the final file: upload_file_122.zip
Uploading trickle at timestep: 10623600
19:30:25 (269144): called boinc_finish(0)

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 May 2023 16:54:10 1496046 22315509 oifs_43r3_00jt_2019110100_123_993_12214151_0 10,623,600 83,728 0.0079
09 May 2023 16:54:08 1496046 22315509 oifs_43r3_00jt_2019110100_123_993_12214151_0 10,285,200 80,878 0.0079
09 May 2023 16:54:06 1496046 22315509 oifs_43r3_00jt_2019110100_123_993_12214151_0 9,421,200 73,774 0.0078
09 May 2023 16:53:55 1496046 22315509 oifs_43r3_00jt_2019110100_123_993_12214151_0 8,035,200 62,521 0.0078
09 May 2023 16:53:49 1496046 22315509 oifs_43r3_00jt_2019110100_123_993_12214151_0 7,171,200 55,562 0.0077
09 May 2023 16:53:43 1496046 22315509 oifs_43r3_00jt_2019110100_123_993_12214151_0 6,307,200 48,688 0.0077
09 May 2023 16:53:37 1496046 22315509 oifs_43r3_00jt_2019110100_123_993_12214151_0 5,443,200 41,866 0.0077
09 May 2023 16:53:31 1496046 22315509 oifs_43r3_00jt_2019110100_123_993_12214151_0 4,320,000 32,617 0.0076
09 May 2023 16:53:26 1496046 22315509 oifs_43r3_00jt_2019110100_123_993_12214151_0 3,456,000 25,990 0.0075
09 May 2023 16:53:19 1496046 22315509 oifs_43r3_00jt_2019110100_123_993_12214151_0 2,592,000 19,502 0.0075
09 May 2023 16:53:12 1496046 22315509 oifs_43r3_00jt_2019110100_123_993_12214151_0 1,728,000 13,003 0.0075
09 May 2023 16:53:05 1496046 22315509 oifs_43r3_00jt_2019110100_123_993_12214151_0 864,000 6,491 0.0075


©2024 climateprediction.net