climateprediction.net home page
Task 22279661

Task 22279661

Name oifs_43r3_ps_0203_2008050100_123_977_12193847_0
Workunit 12193847
Created 23 Dec 2022, 17:49:30 UTC
Sent 26 Dec 2022, 14:42:24 UTC
Report deadline 25 Jan 2023, 14:42:24 UTC
Received 5 Jan 2023, 6:00:47 UTC
Server state Over
Outcome Success
Client state Done
Exit status 0 (0x00000000)
Computer ID 1537490
Run time 15 hours 29 min 50 sec
CPU time 15 hours 10 min 55 sec
Validate state Valid
Credit 14,873.04
Device peak FLOPS 1.00 GFLOPS
Application version OpenIFS 43r3 Perturbed Surface v1.05
x86_64-pc-linux-gnu
Peak working set size 4,415.03 MB
Peak swap size 4,977.89 MB
Peak disk usage 3,745.26 MB
Stderr
<core_client_version>7.20.2</core_client_version>
<![CDATA[
<stderr_txt>
 2377 H=2377:00 +CPU= 14.741
  04:51:36 STEP 2378 H=2378:00 +CPU= 15.358
  04:52:00 STEP 2379 H=2379:00 +CPU= 23.159
  04:52:15 STEP 2380 H=2380:00 +CPU= 15.307
  04:52:30 STEP 2381 H=2381:00 +CPU= 14.974
  04:52:54 STEP 2382 H=2382:00 +CPU= 23.697
  04:53:09 STEP 2383 H=2383:00 +CPU= 14.914
  04:53:24 STEP 2384 H=2384:00 +CPU= 15.323
  04:53:48 STEP 2385 H=2385:00 +CPU= 23.182
  04:54:03 STEP 2386 H=2386:00 +CPU= 15.397
  04:54:18 STEP 2387 H=2387:00 +CPU= 14.921
  04:54:45 STEP 2388 H=2388:00 +CPU= 25.922
Moving to projects directory: /home/mjo/slots/7/ICMGGhq0f+002388
Moving to projects directory: /home/mjo/slots/7/ICMSHhq0f+002388
Moving to projects directory: /home/mjo/slots/7/ICMUAhq0f+002388
  04:55:00 STEP 2389 H=2389:00 +CPU= 15.129
  04:55:15 STEP 2390 H=2390:00 +CPU= 15.256
  04:55:38 STEP 2391 H=2391:00 +CPU= 23.112
  04:55:54 STEP 2392 H=2392:00 +CPU= 15.344
  04:56:09 STEP 2393 H=2393:00 +CPU= 14.836
  04:56:33 STEP 2394 H=2394:00 +CPU= 23.530
  04:56:48 STEP 2395 H=2395:00 +CPU= 14.917
  04:57:03 STEP 2396 H=2396:00 +CPU= 15.301
  04:57:26 STEP 2397 H=2397:00 +CPU= 23.155
  04:57:42 STEP 2398 H=2398:00 +CPU= 15.304
  04:57:57 STEP 2399 H=2399:00 +CPU= 15.229
  04:58:25 STEP 2400 H=2400:00 +CPU= 27.354
Moving to projects directory: /home/mjo/slots/7/ICMGGhq0f+002400
Moving to projects directory: /home/mjo/slots/7/ICMSHhq0f+002400
Moving to projects directory: /home/mjo/slots/7/ICMUAhq0f+002400
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMGGhq0f+002376
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMSHhq0f+002376
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMUAhq0f+002376
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMGGhq0f+002388
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMSHhq0f+002388
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMUAhq0f+002388
Zipping up the intermediate file: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_0203_2008050100_123_977_12193847_0_r2002178832_99.zip
Uploading the intermediate file: upload_file_99.zip
  04:58:40 STEP 2401 H=2401:00 +CPU= 14.893
Uploading trickle at timestep: 8640000
  04:58:55 STEP 2402 H=2402:00 +CPU= 15.415
  04:59:19 STEP 2403 H=2403:00 +CPU= 23.264
  04:59:34 STEP 2404 H=2404:00 +CPU= 15.257
  04:59:49 STEP 2405 H=2405:00 +CPU= 14.884
  05:00:13 STEP 2406 H=2406:00 +CPU= 23.625
  05:00:28 STEP 2407 H=2407:00 +CPU= 15.039
  05:00:44 STEP 2408 H=2408:00 +CPU= 15.487
  05:01:07 STEP 2409 H=2409:00 +CPU= 23.296
  05:01:23 STEP 2410 H=2410:00 +CPU= 15.400
  05:01:38 STEP 2411 H=2411:00 +CPU= 14.880
  05:02:04 STEP 2412 H=2412:00 +CPU= 25.714
Moving to projects directory: /home/mjo/slots/7/ICMGGhq0f+002412
Moving to projects directory: /home/mjo/slots/7/ICMSHhq0f+002412
Moving to projects directory: /home/mjo/slots/7/ICMUAhq0f+002412
  05:02:19 STEP 2413 H=2413:00 +CPU= 15.003
  05:02:34 STEP 2414 H=2414:00 +CPU= 15.453
  05:02:58 STEP 2415 H=2415:00 +CPU= 23.174
  05:03:13 STEP 2416 H=2416:00 +CPU= 15.346
  05:03:28 STEP 2417 H=2417:00 +CPU= 14.810
  05:03:52 STEP 2418 H=2418:00 +CPU= 23.579
  05:04:07 STEP 2419 H=2419:00 +CPU= 14.936
  05:04:22 STEP 2420 H=2420:00 +CPU= 15.277
  05:04:46 STEP 2421 H=2421:00 +CPU= 23.162
  05:05:01 STEP 2422 H=2422:00 +CPU= 15.313
  05:05:16 STEP 2423 H=2423:00 +CPU= 15.269
  05:05:44 STEP 2424 H=2424:00 +CPU= 27.144
Moving to projects directory: /home/mjo/slots/7/ICMGGhq0f+002424
Moving to projects directory: /home/mjo/slots/7/ICMSHhq0f+002424
Moving to projects directory: /home/mjo/slots/7/ICMUAhq0f+002424
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMGGhq0f+002400
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMSHhq0f+002400
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMUAhq0f+002400
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMGGhq0f+002412
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMSHhq0f+002412
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMUAhq0f+002412
Zipping up the intermediate file: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_0203_2008050100_123_977_12193847_0_r2002178832_100.zip
Uploading the intermediate file: upload_file_100.zip
  05:05:59 STEP 2425 H=2425:00 +CPU= 14.884
  05:06:14 STEP 2426 H=2426:00 +CPU= 15.356
  05:06:38 STEP 2427 H=2427:00 +CPU= 23.312
  05:06:53 STEP 2428 H=2428:00 +CPU= 15.365
  05:07:08 STEP 2429 H=2429:00 +CPU= 14.849
  05:07:32 STEP 2430 H=2430:00 +CPU= 23.749
  05:07:47 STEP 2431 H=2431:00 +CPU= 14.979
  05:08:03 STEP 2432 H=2432:00 +CPU= 15.358
  05:08:26 STEP 2433 H=2433:00 +CPU= 23.418
  05:08:42 STEP 2434 H=2434:00 +CPU= 15.546
  05:08:57 STEP 2435 H=2435:00 +CPU= 14.969
  05:09:23 STEP 2436 H=2436:00 +CPU= 25.644
Moving to projects directory: /home/mjo/slots/7/ICMGGhq0f+002436
Moving to projects directory: /home/mjo/slots/7/ICMSHhq0f+002436
Moving to projects directory: /home/mjo/slots/7/ICMUAhq0f+002436
  05:09:38 STEP 2437 H=2437:00 +CPU= 15.011
  05:09:54 STEP 2438 H=2438:00 +CPU= 15.676
  05:10:17 STEP 2439 H=2439:00 +CPU= 23.033
  05:10:33 STEP 2440 H=2440:00 +CPU= 15.344
  05:10:48 STEP 2441 H=2441:00 +CPU= 14.936
  05:11:11 STEP 2442 H=2442:00 +CPU= 23.517
  05:11:26 STEP 2443 H=2443:00 +CPU= 14.991
  05:11:42 STEP 2444 H=2444:00 +CPU= 15.232
  05:12:05 STEP 2445 H=2445:00 +CPU= 23.089
  05:12:20 STEP 2446 H=2446:00 +CPU= 15.384
  05:12:36 STEP 2447 H=2447:00 +CPU= 15.289
  05:13:03 STEP 2448 H=2448:00 +CPU= 27.227
Moving to projects directory: /home/mjo/slots/7/ICMGGhq0f+002448
Moving to projects directory: /home/mjo/slots/7/ICMSHhq0f+002448
Moving to projects directory: /home/mjo/slots/7/ICMUAhq0f+002448
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMGGhq0f+002424
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMSHhq0f+002424
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMUAhq0f+002424
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMGGhq0f+002436
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMSHhq0f+002436
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMUAhq0f+002436
Zipping up the intermediate file: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_0203_2008050100_123_977_12193847_0_r2002178832_101.zip
Uploading the intermediate file: upload_file_101.zip
  05:13:18 STEP 2449 H=2449:00 +CPU= 14.832
  05:13:34 STEP 2450 H=2450:00 +CPU= 15.366
  05:13:57 STEP 2451 H=2451:00 +CPU= 23.358
  05:14:13 STEP 2452 H=2452:00 +CPU= 15.262
  05:14:28 STEP 2453 H=2453:00 +CPU= 14.934
  05:14:52 STEP 2454 H=2454:00 +CPU= 23.687
  05:15:07 STEP 2455 H=2455:00 +CPU= 14.880
  05:15:22 STEP 2456 H=2456:00 +CPU= 15.371
  05:15:45 STEP 2457 H=2457:00 +CPU= 23.120
  05:16:01 STEP 2458 H=2458:00 +CPU= 15.467
  05:16:16 STEP 2459 H=2459:00 +CPU= 15.059
  05:16:42 STEP 2460 H=2460:00 +CPU= 25.568
Moving to projects directory: /home/mjo/slots/7/ICMGGhq0f+002460
Moving to projects directory: /home/mjo/slots/7/ICMSHhq0f+002460
Moving to projects directory: /home/mjo/slots/7/ICMUAhq0f+002460
  05:16:57 STEP 2461 H=2461:00 +CPU= 14.875
  05:17:13 STEP 2462 H=2462:00 +CPU= 15.397
  05:17:36 STEP 2463 H=2463:00 +CPU= 23.123
  05:17:51 STEP 2464 H=2464:00 +CPU= 15.387
  05:18:06 STEP 2465 H=2465:00 +CPU= 14.808
  05:18:30 STEP 2466 H=2466:00 +CPU= 23.492
  05:18:45 STEP 2467 H=2467:00 +CPU= 14.946
  05:19:00 STEP 2468 H=2468:00 +CPU= 15.366
  05:19:24 STEP 2469 H=2469:00 +CPU= 23.095
  05:19:39 STEP 2470 H=2470:00 +CPU= 15.286
  05:19:55 STEP 2471 H=2471:00 +CPU= 15.387
  05:20:22 STEP 2472 H=2472:00 +CPU= 27.254
Moving to projects directory: /home/mjo/slots/7/ICMGGhq0f+002472
Moving to projects directory: /home/mjo/slots/7/ICMSHhq0f+002472
Moving to projects directory: /home/mjo/slots/7/ICMUAhq0f+002472
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMGGhq0f+002448
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMSHhq0f+002448
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMUAhq0f+002448
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMGGhq0f+002460
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMSHhq0f+002460
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMUAhq0f+002460
Zipping up the intermediate file: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_0203_2008050100_123_977_12193847_0_r2002178832_102.zip
Uploading the intermediate file: upload_file_102.zip
  05:20:37 STEP 2473 H=2473:00 +CPU= 14.824
  05:20:52 STEP 2474 H=2474:00 +CPU= 15.249
  05:21:16 STEP 2475 H=2475:00 +CPU= 23.201
  05:21:31 STEP 2476 H=2476:00 +CPU= 15.307
  05:21:46 STEP 2477 H=2477:00 +CPU= 14.952
  05:22:10 STEP 2478 H=2478:00 +CPU= 23.524
  05:22:25 STEP 2479 H=2479:00 +CPU= 14.889
  05:22:40 STEP 2480 H=2480:00 +CPU= 15.293
  05:23:04 STEP 2481 H=2481:00 +CPU= 23.128
  05:23:19 STEP 2482 H=2482:00 +CPU= 15.404
  05:23:34 STEP 2483 H=2483:00 +CPU= 14.818
  05:24:00 STEP 2484 H=2484:00 +CPU= 25.662
Moving to projects directory: /home/mjo/slots/7/ICMGGhq0f+002484
Moving to projects directory: /home/mjo/slots/7/ICMSHhq0f+002484
Moving to projects directory: /home/mjo/slots/7/ICMUAhq0f+002484
  05:24:15 STEP 2485 H=2485:00 +CPU= 15.002
  05:24:31 STEP 2486 H=2486:00 +CPU= 15.201
  05:24:54 STEP 2487 H=2487:00 +CPU= 23.160
  05:25:09 STEP 2488 H=2488:00 +CPU= 15.324
  05:25:24 STEP 2489 H=2489:00 +CPU= 14.758
  05:25:48 STEP 2490 H=2490:00 +CPU= 23.568
  05:26:03 STEP 2491 H=2491:00 +CPU= 14.926
  05:26:18 STEP 2492 H=2492:00 +CPU= 15.236
  05:26:42 STEP 2493 H=2493:00 +CPU= 23.221
  05:26:57 STEP 2494 H=2494:00 +CPU= 15.388
  05:27:13 STEP 2495 H=2495:00 +CPU= 15.331
  05:27:40 STEP 2496 H=2496:00 +CPU= 27.417
Moving to projects directory: /home/mjo/slots/7/ICMGGhq0f+002496
Moving to projects directory: /home/mjo/slots/7/ICMSHhq0f+002496
Moving to projects directory: /home/mjo/slots/7/ICMUAhq0f+002496
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMGGhq0f+002472
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMSHhq0f+002472
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMUAhq0f+002472
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMGGhq0f+002484
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMSHhq0f+002484
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMUAhq0f+002484
Zipping up the intermediate file: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_0203_2008050100_123_977_12193847_0_r2002178832_103.zip
Uploading the intermediate file: upload_file_103.zip
  05:27:55 STEP 2497 H=2497:00 +CPU= 14.864
  05:28:11 STEP 2498 H=2498:00 +CPU= 15.470
  05:28:34 STEP 2499 H=2499:00 +CPU= 23.149
  05:28:50 STEP 2500 H=2500:00 +CPU= 15.342
  05:29:05 STEP 2501 H=2501:00 +CPU= 14.832
  05:29:28 STEP 2502 H=2502:00 +CPU= 23.541
  05:29:43 STEP 2503 H=2503:00 +CPU= 14.836
  05:29:59 STEP 2504 H=2504:00 +CPU= 15.227
  05:30:22 STEP 2505 H=2505:00 +CPU= 23.042
  05:30:37 STEP 2506 H=2506:00 +CPU= 15.353
  05:30:52 STEP 2507 H=2507:00 +CPU= 14.819
  05:31:18 STEP 2508 H=2508:00 +CPU= 25.614
Moving to projects directory: /home/mjo/slots/7/ICMGGhq0f+002508
Moving to projects directory: /home/mjo/slots/7/ICMSHhq0f+002508
Moving to projects directory: /home/mjo/slots/7/ICMUAhq0f+002508
  05:31:33 STEP 2509 H=2509:00 +CPU= 14.982
  05:31:49 STEP 2510 H=2510:00 +CPU= 15.354
  05:32:12 STEP 2511 H=2511:00 +CPU= 23.374
  05:32:28 STEP 2512 H=2512:00 +CPU= 15.362
  05:32:43 STEP 2513 H=2513:00 +CPU= 14.860
  05:33:07 STEP 2514 H=2514:00 +CPU= 23.787
  05:33:22 STEP 2515 H=2515:00 +CPU= 14.983
  05:33:37 STEP 2516 H=2516:00 +CPU= 15.257
  05:34:01 STEP 2517 H=2517:00 +CPU= 23.429
  05:34:16 STEP 2518 H=2518:00 +CPU= 15.395
  05:34:32 STEP 2519 H=2519:00 +CPU= 15.285
  05:34:59 STEP 2520 H=2520:00 +CPU= 27.205
Moving to projects directory: /home/mjo/slots/7/ICMGGhq0f+002520
Moving to projects directory: /home/mjo/slots/7/ICMSHhq0f+002520
Moving to projects directory: /home/mjo/slots/7/ICMUAhq0f+002520
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMGGhq0f+002496
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMSHhq0f+002496
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMUAhq0f+002496
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMGGhq0f+002508
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMSHhq0f+002508
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMUAhq0f+002508
Zipping up the intermediate file: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_0203_2008050100_123_977_12193847_0_r2002178832_104.zip
Uploading the intermediate file: upload_file_104.zip
  05:35:14 STEP 2521 H=2521:00 +CPU= 14.860
  05:35:30 STEP 2522 H=2522:00 +CPU= 15.296
  05:35:53 STEP 2523 H=2523:00 +CPU= 23.236
  05:36:09 STEP 2524 H=2524:00 +CPU= 15.411
  05:36:24 STEP 2525 H=2525:00 +CPU= 14.872
  05:36:47 STEP 2526 H=2526:00 +CPU= 23.728
  05:37:03 STEP 2527 H=2527:00 +CPU= 14.956
  05:37:18 STEP 2528 H=2528:00 +CPU= 15.325
  05:37:41 STEP 2529 H=2529:00 +CPU= 23.191
  05:37:57 STEP 2530 H=2530:00 +CPU= 15.439
  05:38:12 STEP 2531 H=2531:00 +CPU= 15.110
  05:38:38 STEP 2532 H=2532:00 +CPU= 25.668
Moving to projects directory: /home/mjo/slots/7/ICMGGhq0f+002532
Moving to projects directory: /home/mjo/slots/7/ICMSHhq0f+002532
Moving to projects directory: /home/mjo/slots/7/ICMUAhq0f+002532
  05:38:53 STEP 2533 H=2533:00 +CPU= 15.046
  05:39:09 STEP 2534 H=2534:00 +CPU= 15.346
  05:39:32 STEP 2535 H=2535:00 +CPU= 23.160
  05:39:47 STEP 2536 H=2536:00 +CPU= 15.311
  05:40:03 STEP 2537 H=2537:00 +CPU= 14.907
  05:40:26 STEP 2538 H=2538:00 +CPU= 23.569
  05:40:41 STEP 2539 H=2539:00 +CPU= 14.881
  05:40:57 STEP 2540 H=2540:00 +CPU= 15.312
  05:41:20 STEP 2541 H=2541:00 +CPU= 23.214
  05:41:35 STEP 2542 H=2542:00 +CPU= 15.400
  05:41:51 STEP 2543 H=2543:00 +CPU= 15.498
  05:42:19 STEP 2544 H=2544:00 +CPU= 27.245
Moving to projects directory: /home/mjo/slots/7/ICMGGhq0f+002544
Moving to projects directory: /home/mjo/slots/7/ICMSHhq0f+002544
Moving to projects directory: /home/mjo/slots/7/ICMUAhq0f+002544
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMGGhq0f+002520
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMSHhq0f+002520
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMUAhq0f+002520
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMGGhq0f+002532
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMSHhq0f+002532
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMUAhq0f+002532
Zipping up the intermediate file: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_0203_2008050100_123_977_12193847_0_r2002178832_105.zip
Uploading the intermediate file: upload_file_105.zip
  05:42:34 STEP 2545 H=2545:00 +CPU= 14.857
  05:42:49 STEP 2546 H=2546:00 +CPU= 15.280
  05:43:13 STEP 2547 H=2547:00 +CPU= 23.269
  05:43:28 STEP 2548 H=2548:00 +CPU= 15.303
  05:43:43 STEP 2549 H=2549:00 +CPU= 14.888
  05:44:07 STEP 2550 H=2550:00 +CPU= 23.970
  05:44:22 STEP 2551 H=2551:00 +CPU= 15.132
  05:44:38 STEP 2552 H=2552:00 +CPU= 15.358
  05:45:01 STEP 2553 H=2553:00 +CPU= 23.138
  05:45:17 STEP 2554 H=2554:00 +CPU= 15.593
  05:45:32 STEP 2555 H=2555:00 +CPU= 14.848
  05:45:58 STEP 2556 H=2556:00 +CPU= 25.692
Moving to projects directory: /home/mjo/slots/7/ICMGGhq0f+002556
Moving to projects directory: /home/mjo/slots/7/ICMSHhq0f+002556
Moving to projects directory: /home/mjo/slots/7/ICMUAhq0f+002556
  05:46:13 STEP 2557 H=2557:00 +CPU= 15.027
  05:46:28 STEP 2558 H=2558:00 +CPU= 15.324
  05:46:52 STEP 2559 H=2559:00 +CPU= 23.257
  05:47:07 STEP 2560 H=2560:00 +CPU= 15.296
  05:47:22 STEP 2561 H=2561:00 +CPU= 14.774
  05:47:46 STEP 2562 H=2562:00 +CPU= 23.729
  05:48:01 STEP 2563 H=2563:00 +CPU= 14.883
  05:48:16 STEP 2564 H=2564:00 +CPU= 15.282
  05:48:40 STEP 2565 H=2565:00 +CPU= 23.336
  05:48:55 STEP 2566 H=2566:00 +CPU= 15.269
  05:49:11 STEP 2567 H=2567:00 +CPU= 15.421
  05:49:38 STEP 2568 H=2568:00 +CPU= 27.430
Moving to projects directory: /home/mjo/slots/7/ICMGGhq0f+002568
Moving to projects directory: /home/mjo/slots/7/ICMSHhq0f+002568
Moving to projects directory: /home/mjo/slots/7/ICMUAhq0f+002568
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMGGhq0f+002544
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMSHhq0f+002544
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMUAhq0f+002544
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMGGhq0f+002556
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMSHhq0f+002556
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMUAhq0f+002556
Zipping up the intermediate file: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_0203_2008050100_123_977_12193847_0_r2002178832_106.zip
Uploading the intermediate file: upload_file_106.zip
  05:49:54 STEP 2569 H=2569:00 +CPU= 15.179
  05:50:09 STEP 2570 H=2570:00 +CPU= 15.500
  05:50:33 STEP 2571 H=2571:00 +CPU= 23.133
  05:50:48 STEP 2572 H=2572:00 +CPU= 15.343
  05:51:03 STEP 2573 H=2573:00 +CPU= 15.103
  05:51:27 STEP 2574 H=2574:00 +CPU= 23.745
  05:51:42 STEP 2575 H=2575:00 +CPU= 14.883
  05:51:58 STEP 2576 H=2576:00 +CPU= 15.311
  05:52:21 STEP 2577 H=2577:00 +CPU= 23.196
  05:52:36 STEP 2578 H=2578:00 +CPU= 15.308
  05:52:52 STEP 2579 H=2579:00 +CPU= 15.035
  05:53:17 STEP 2580 H=2580:00 +CPU= 25.689
Moving to projects directory: /home/mjo/slots/7/ICMGGhq0f+002580
Moving to projects directory: /home/mjo/slots/7/ICMSHhq0f+002580
Moving to projects directory: /home/mjo/slots/7/ICMUAhq0f+002580
  05:53:33 STEP 2581 H=2581:00 +CPU= 14.934
  05:53:48 STEP 2582 H=2582:00 +CPU= 15.559
  05:54:11 STEP 2583 H=2583:00 +CPU= 23.070
  05:54:27 STEP 2584 H=2584:00 +CPU= 15.221
  05:54:42 STEP 2585 H=2585:00 +CPU= 14.867
  05:55:06 STEP 2586 H=2586:00 +CPU= 23.890
  05:55:21 STEP 2587 H=2587:00 +CPU= 14.859
  05:55:36 STEP 2588 H=2588:00 +CPU= 15.526
  05:56:00 STEP 2589 H=2589:00 +CPU= 23.093
  05:56:15 STEP 2590 H=2590:00 +CPU= 15.375
  05:56:31 STEP 2591 H=2591:00 +CPU= 15.468
  05:56:58 STEP 2592 H=2592:00 +CPU= 27.288
Moving to projects directory: /home/mjo/slots/7/ICMGGhq0f+002592
Moving to projects directory: /home/mjo/slots/7/ICMSHhq0f+002592
Moving to projects directory: /home/mjo/slots/7/ICMUAhq0f+002592
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMGGhq0f+002568
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMSHhq0f+002568
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMUAhq0f+002568
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMGGhq0f+002580
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMSHhq0f+002580
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMUAhq0f+002580
Zipping up the intermediate file: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_0203_2008050100_123_977_12193847_0_r2002178832_107.zip
Uploading the intermediate file: upload_file_107.zip
  05:57:13 STEP 2593 H=2593:00 +CPU= 14.837
  05:57:28 STEP 2594 H=2594:00 +CPU= 15.188
  05:57:52 STEP 2595 H=2595:00 +CPU= 23.054
  05:58:07 STEP 2596 H=2596:00 +CPU= 15.214
  05:58:22 STEP 2597 H=2597:00 +CPU= 14.904
  05:58:46 STEP 2598 H=2598:00 +CPU= 23.482
  05:59:01 STEP 2599 H=2599:00 +CPU= 14.886
  05:59:16 STEP 2600 H=2600:00 +CPU= 15.187
  05:59:39 STEP 2601 H=2601:00 +CPU= 23.194
  05:59:55 STEP 2602 H=2602:00 +CPU= 15.292
  06:00:10 STEP 2603 H=2603:00 +CPU= 14.952
  06:00:36 STEP 2604 H=2604:00 +CPU= 25.646
Moving to projects directory: /home/mjo/slots/7/ICMGGhq0f+002604
Moving to projects directory: /home/mjo/slots/7/ICMSHhq0f+002604
Moving to projects directory: /home/mjo/slots/7/ICMUAhq0f+002604
  06:00:51 STEP 2605 H=2605:00 +CPU= 15.025
  06:01:06 STEP 2606 H=2606:00 +CPU= 15.322
  06:01:30 STEP 2607 H=2607:00 +CPU= 23.249
  06:01:45 STEP 2608 H=2608:00 +CPU= 15.267
  06:02:00 STEP 2609 H=2609:00 +CPU= 14.845
  06:02:24 STEP 2610 H=2610:00 +CPU= 23.561
  06:02:39 STEP 2611 H=2611:00 +CPU= 14.854
  06:02:54 STEP 2612 H=2612:00 +CPU= 15.219
  06:03:17 STEP 2613 H=2613:00 +CPU= 23.134
  06:03:33 STEP 2614 H=2614:00 +CPU= 15.196
  06:03:48 STEP 2615 H=2615:00 +CPU= 15.342
  06:04:16 STEP 2616 H=2616:00 +CPU= 27.411
Moving to projects directory: /home/mjo/slots/7/ICMGGhq0f+002616
Moving to projects directory: /home/mjo/slots/7/ICMSHhq0f+002616
Moving to projects directory: /home/mjo/slots/7/ICMUAhq0f+002616
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMGGhq0f+002592
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMSHhq0f+002592
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMUAhq0f+002592
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMGGhq0f+002604
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMSHhq0f+002604
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMUAhq0f+002604
Zipping up the intermediate file: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_0203_2008050100_123_977_12193847_0_r2002178832_108.zip
Uploading the intermediate file: upload_file_108.zip
  06:04:31 STEP 2617 H=2617:00 +CPU= 14.919
  06:04:46 STEP 2618 H=2618:00 +CPU= 15.312
  06:05:10 STEP 2619 H=2619:00 +CPU= 23.118
  06:05:25 STEP 2620 H=2620:00 +CPU= 15.289
  06:05:40 STEP 2621 H=2621:00 +CPU= 14.883
  06:06:04 STEP 2622 H=2622:00 +CPU= 23.605
  06:06:19 STEP 2623 H=2623:00 +CPU= 14.912
  06:06:34 STEP 2624 H=2624:00 +CPU= 15.268
  06:06:58 STEP 2625 H=2625:00 +CPU= 23.242
  06:07:13 STEP 2626 H=2626:00 +CPU= 15.283
  06:07:28 STEP 2627 H=2627:00 +CPU= 14.968
  06:07:54 STEP 2628 H=2628:00 +CPU= 25.707
Moving to projects directory: /home/mjo/slots/7/ICMGGhq0f+002628
Moving to projects directory: /home/mjo/slots/7/ICMSHhq0f+002628
Moving to projects directory: /home/mjo/slots/7/ICMUAhq0f+002628
  06:08:09 STEP 2629 H=2629:00 +CPU= 14.873
  06:08:25 STEP 2630 H=2630:00 +CPU= 15.331
  06:08:48 STEP 2631 H=2631:00 +CPU= 23.085
  06:09:03 STEP 2632 H=2632:00 +CPU= 15.287
  06:09:18 STEP 2633 H=2633:00 +CPU= 14.831
  06:09:42 STEP 2634 H=2634:00 +CPU= 23.618
  06:09:57 STEP 2635 H=2635:00 +CPU= 14.970
  06:10:13 STEP 2636 H=2636:00 +CPU= 15.339
  06:10:36 STEP 2637 H=2637:00 +CPU= 23.209
  06:10:51 STEP 2638 H=2638:00 +CPU= 15.266
  06:11:07 STEP 2639 H=2639:00 +CPU= 15.299
  06:11:34 STEP 2640 H=2640:00 +CPU= 27.224
Moving to projects directory: /home/mjo/slots/7/ICMGGhq0f+002640
Moving to projects directory: /home/mjo/slots/7/ICMSHhq0f+002640
Moving to projects directory: /home/mjo/slots/7/ICMUAhq0f+002640
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMGGhq0f+002616
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMSHhq0f+002616
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMUAhq0f+002616
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMGGhq0f+002628
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMSHhq0f+002628
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMUAhq0f+002628
Zipping up the intermediate file: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_0203_2008050100_123_977_12193847_0_r2002178832_109.zip
Uploading the intermediate file: upload_file_109.zip
  06:11:49 STEP 2641 H=2641:00 +CPU= 14.904
Uploading trickle at timestep: 9504000
  06:12:05 STEP 2642 H=2642:00 +CPU= 15.305
  06:12:28 STEP 2643 H=2643:00 +CPU= 23.114
  06:12:43 STEP 2644 H=2644:00 +CPU= 15.331
  06:12:58 STEP 2645 H=2645:00 +CPU= 14.741
  06:13:22 STEP 2646 H=2646:00 +CPU= 23.659
  06:13:37 STEP 2647 H=2647:00 +CPU= 14.896
  06:13:53 STEP 2648 H=2648:00 +CPU= 15.432
  06:14:16 STEP 2649 H=2649:00 +CPU= 23.180
  06:14:32 STEP 2650 H=2650:00 +CPU= 15.352
  06:14:47 STEP 2651 H=2651:00 +CPU= 14.986
  06:15:12 STEP 2652 H=2652:00 +CPU= 25.672
Moving to projects directory: /home/mjo/slots/7/ICMGGhq0f+002652
Moving to projects directory: /home/mjo/slots/7/ICMSHhq0f+002652
Moving to projects directory: /home/mjo/slots/7/ICMUAhq0f+002652
  06:15:28 STEP 2653 H=2653:00 +CPU= 14.913
  06:15:43 STEP 2654 H=2654:00 +CPU= 15.289
  06:16:06 STEP 2655 H=2655:00 +CPU= 23.040
  06:16:22 STEP 2656 H=2656:00 +CPU= 15.402
  06:16:37 STEP 2657 H=2657:00 +CPU= 14.965
  06:17:01 STEP 2658 H=2658:00 +CPU= 23.890
  06:17:16 STEP 2659 H=2659:00 +CPU= 14.932
  06:17:31 STEP 2660 H=2660:00 +CPU= 15.217
  06:17:55 STEP 2661 H=2661:00 +CPU= 23.279
  06:18:10 STEP 2662 H=2662:00 +CPU= 15.469
  06:18:26 STEP 2663 H=2663:00 +CPU= 15.448
  06:18:54 STEP 2664 H=2664:00 +CPU= 27.342
Moving to projects directory: /home/mjo/slots/7/ICMGGhq0f+002664
Moving to projects directory: /home/mjo/slots/7/ICMSHhq0f+002664
Moving to projects directory: /home/mjo/slots/7/ICMUAhq0f+002664
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMGGhq0f+002640
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMSHhq0f+002640
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMUAhq0f+002640
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMGGhq0f+002652
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMSHhq0f+002652
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMUAhq0f+002652
Zipping up the intermediate file: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_0203_2008050100_123_977_12193847_0_r2002178832_110.zip
Uploading the intermediate file: upload_file_110.zip
  06:19:09 STEP 2665 H=2665:00 +CPU= 14.843
  06:19:24 STEP 2666 H=2666:00 +CPU= 15.295
  06:19:48 STEP 2667 H=2667:00 +CPU= 23.278
  06:20:03 STEP 2668 H=2668:00 +CPU= 15.310
  06:20:18 STEP 2669 H=2669:00 +CPU= 15.007
  06:20:42 STEP 2670 H=2670:00 +CPU= 23.800
  06:20:57 STEP 2671 H=2671:00 +CPU= 14.856
  06:21:12 STEP 2672 H=2672:00 +CPU= 15.201
  06:21:36 STEP 2673 H=2673:00 +CPU= 23.225
  06:21:51 STEP 2674 H=2674:00 +CPU= 15.192
  06:22:06 STEP 2675 H=2675:00 +CPU= 14.824
  06:22:32 STEP 2676 H=2676:00 +CPU= 25.766
Moving to projects directory: /home/mjo/slots/7/ICMGGhq0f+002676
Moving to projects directory: /home/mjo/slots/7/ICMSHhq0f+002676
Moving to projects directory: /home/mjo/slots/7/ICMUAhq0f+002676
  06:22:47 STEP 2677 H=2677:00 +CPU= 14.909
  06:23:03 STEP 2678 H=2678:00 +CPU= 15.456
  06:23:26 STEP 2679 H=2679:00 +CPU= 22.995
  06:23:41 STEP 2680 H=2680:00 +CPU= 15.234
  06:23:56 STEP 2681 H=2681:00 +CPU= 14.834
  06:24:20 STEP 2682 H=2682:00 +CPU= 23.558
  06:24:35 STEP 2683 H=2683:00 +CPU= 14.867
  06:24:50 STEP 2684 H=2684:00 +CPU= 15.242
  06:25:14 STEP 2685 H=2685:00 +CPU= 22.992
  06:25:29 STEP 2686 H=2686:00 +CPU= 15.329
  06:25:44 STEP 2687 H=2687:00 +CPU= 15.380
  06:26:12 STEP 2688 H=2688:00 +CPU= 27.236
Moving to projects directory: /home/mjo/slots/7/ICMGGhq0f+002688
Moving to projects directory: /home/mjo/slots/7/ICMSHhq0f+002688
Moving to projects directory: /home/mjo/slots/7/ICMUAhq0f+002688
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMGGhq0f+002664
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMSHhq0f+002664
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMUAhq0f+002664
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMGGhq0f+002676
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMSHhq0f+002676
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMUAhq0f+002676
Zipping up the intermediate file: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_0203_2008050100_123_977_12193847_0_r2002178832_111.zip
Uploading the intermediate file: upload_file_111.zip
  06:26:27 STEP 2689 H=2689:00 +CPU= 14.811
  06:26:42 STEP 2690 H=2690:00 +CPU= 15.282
  06:27:06 STEP 2691 H=2691:00 +CPU= 23.175
  06:27:21 STEP 2692 H=2692:00 +CPU= 15.192
  06:27:36 STEP 2693 H=2693:00 +CPU= 14.953
  06:28:00 STEP 2694 H=2694:00 +CPU= 23.644
  06:28:15 STEP 2695 H=2695:00 +CPU= 14.960
  06:28:30 STEP 2696 H=2696:00 +CPU= 15.346
  06:28:54 STEP 2697 H=2697:00 +CPU= 23.199
  06:29:09 STEP 2698 H=2698:00 +CPU= 15.319
  06:29:24 STEP 2699 H=2699:00 +CPU= 14.889
  06:29:50 STEP 2700 H=2700:00 +CPU= 25.669
Moving to projects directory: /home/mjo/slots/7/ICMGGhq0f+002700
Moving to projects directory: /home/mjo/slots/7/ICMSHhq0f+002700
Moving to projects directory: /home/mjo/slots/7/ICMUAhq0f+002700
  06:30:05 STEP 2701 H=2701:00 +CPU= 14.927
  06:30:21 STEP 2702 H=2702:00 +CPU= 15.284
  06:30:44 STEP 2703 H=2703:00 +CPU= 22.953
  06:30:59 STEP 2704 H=2704:00 +CPU= 15.169
  06:31:14 STEP 2705 H=2705:00 +CPU= 14.973
  06:31:38 STEP 2706 H=2706:00 +CPU= 23.609
  06:31:53 STEP 2707 H=2707:00 +CPU= 14.840
  06:32:08 STEP 2708 H=2708:00 +CPU= 15.245
  06:32:32 STEP 2709 H=2709:00 +CPU= 23.131
  06:32:47 STEP 2710 H=2710:00 +CPU= 15.266
  06:33:02 STEP 2711 H=2711:00 +CPU= 15.266
  06:33:30 STEP 2712 H=2712:00 +CPU= 27.354
Moving to projects directory: /home/mjo/slots/7/ICMGGhq0f+002712
Moving to projects directory: /home/mjo/slots/7/ICMSHhq0f+002712
Moving to projects directory: /home/mjo/slots/7/ICMUAhq0f+002712
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMGGhq0f+002688
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMSHhq0f+002688
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMUAhq0f+002688
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMGGhq0f+002700
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMSHhq0f+002700
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMUAhq0f+002700
Zipping up the intermediate file: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_0203_2008050100_123_977_12193847_0_r2002178832_112.zip
Uploading the intermediate file: upload_file_112.zip
  06:33:45 STEP 2713 H=2713:00 +CPU= 14.888
  06:34:01 STEP 2714 H=2714:00 +CPU= 15.418
  06:34:24 STEP 2715 H=2715:00 +CPU= 23.244
  06:34:40 STEP 2716 H=2716:00 +CPU= 15.446
  06:34:55 STEP 2717 H=2717:00 +CPU= 14.904
  06:35:18 STEP 2718 H=2718:00 +CPU= 23.560
  06:35:33 STEP 2719 H=2719:00 +CPU= 14.935
  06:35:49 STEP 2720 H=2720:00 +CPU= 15.321
  06:36:12 STEP 2721 H=2721:00 +CPU= 23.284
  06:36:28 STEP 2722 H=2722:00 +CPU= 15.371
  06:36:43 STEP 2723 H=2723:00 +CPU= 14.978
  06:37:09 STEP 2724 H=2724:00 +CPU= 25.526
Moving to projects directory: /home/mjo/slots/7/ICMGGhq0f+002724
Moving to projects directory: /home/mjo/slots/7/ICMSHhq0f+002724
Moving to projects directory: /home/mjo/slots/7/ICMUAhq0f+002724
  06:37:24 STEP 2725 H=2725:00 +CPU= 15.017
  06:37:39 STEP 2726 H=2726:00 +CPU= 15.334
  06:38:03 STEP 2727 H=2727:00 +CPU= 23.096
  06:38:18 STEP 2728 H=2728:00 +CPU= 15.329
  06:38:33 STEP 2729 H=2729:00 +CPU= 14.823
  06:38:57 STEP 2730 H=2730:00 +CPU= 23.523
  06:39:12 STEP 2731 H=2731:00 +CPU= 15.115
  06:39:27 STEP 2732 H=2732:00 +CPU= 15.281
  06:39:51 STEP 2733 H=2733:00 +CPU= 23.373
  06:40:06 STEP 2734 H=2734:00 +CPU= 15.203
  06:40:21 STEP 2735 H=2735:00 +CPU= 15.287
  06:40:49 STEP 2736 H=2736:00 +CPU= 27.233
Moving to projects directory: /home/mjo/slots/7/ICMGGhq0f+002736
Moving to projects directory: /home/mjo/slots/7/ICMSHhq0f+002736
Moving to projects directory: /home/mjo/slots/7/ICMUAhq0f+002736
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMGGhq0f+002712
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMSHhq0f+002712
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMUAhq0f+002712
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMGGhq0f+002724
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMSHhq0f+002724
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMUAhq0f+002724
Zipping up the intermediate file: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_0203_2008050100_123_977_12193847_0_r2002178832_113.zip
Uploading the intermediate file: upload_file_113.zip
  06:41:04 STEP 2737 H=2737:00 +CPU= 14.725
  06:41:19 STEP 2738 H=2738:00 +CPU= 15.230
  06:41:42 STEP 2739 H=2739:00 +CPU= 22.913
  06:41:58 STEP 2740 H=2740:00 +CPU= 15.281
  06:42:13 STEP 2741 H=2741:00 +CPU= 14.893
  06:42:36 STEP 2742 H=2742:00 +CPU= 23.557
  06:42:51 STEP 2743 H=2743:00 +CPU= 14.933
  06:43:07 STEP 2744 H=2744:00 +CPU= 15.439
  06:43:30 STEP 2745 H=2745:00 +CPU= 23.029
  06:43:46 STEP 2746 H=2746:00 +CPU= 15.285
  06:44:01 STEP 2747 H=2747:00 +CPU= 14.969
  06:44:27 STEP 2748 H=2748:00 +CPU= 25.796
Moving to projects directory: /home/mjo/slots/7/ICMGGhq0f+002748
Moving to projects directory: /home/mjo/slots/7/ICMSHhq0f+002748
Moving to projects directory: /home/mjo/slots/7/ICMUAhq0f+002748
  06:44:42 STEP 2749 H=2749:00 +CPU= 14.950
  06:44:57 STEP 2750 H=2750:00 +CPU= 15.338
  06:45:21 STEP 2751 H=2751:00 +CPU= 23.092
  06:45:36 STEP 2752 H=2752:00 +CPU= 15.351
  06:45:51 STEP 2753 H=2753:00 +CPU= 14.912
  06:46:15 STEP 2754 H=2754:00 +CPU= 23.603
  06:46:30 STEP 2755 H=2755:00 +CPU= 14.935
  06:46:46 STEP 2756 H=2756:00 +CPU= 15.442
  06:47:09 STEP 2757 H=2757:00 +CPU= 23.418
  06:47:24 STEP 2758 H=2758:00 +CPU= 15.226
  06:47:40 STEP 2759 H=2759:00 +CPU= 15.355
  06:48:08 STEP 2760 H=2760:00 +CPU= 27.423
Moving to projects directory: /home/mjo/slots/7/ICMGGhq0f+002760
Moving to projects directory: /home/mjo/slots/7/ICMSHhq0f+002760
Moving to projects directory: /home/mjo/slots/7/ICMUAhq0f+002760
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMGGhq0f+002736
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMSHhq0f+002736
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMUAhq0f+002736
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMGGhq0f+002748
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMSHhq0f+002748
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMUAhq0f+002748
Zipping up the intermediate file: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_0203_2008050100_123_977_12193847_0_r2002178832_114.zip
Uploading the intermediate file: upload_file_114.zip
  06:48:23 STEP 2761 H=2761:00 +CPU= 14.836
  06:48:38 STEP 2762 H=2762:00 +CPU= 15.256
  06:49:01 STEP 2763 H=2763:00 +CPU= 23.066
  06:49:17 STEP 2764 H=2764:00 +CPU= 15.334
  06:49:32 STEP 2765 H=2765:00 +CPU= 14.762
  06:49:55 STEP 2766 H=2766:00 +CPU= 23.452
  06:50:10 STEP 2767 H=2767:00 +CPU= 14.753
  06:50:25 STEP 2768 H=2768:00 +CPU= 15.207
  06:50:49 STEP 2769 H=2769:00 +CPU= 23.168
  06:51:04 STEP 2770 H=2770:00 +CPU= 15.345
  06:51:19 STEP 2771 H=2771:00 +CPU= 14.949
  06:51:45 STEP 2772 H=2772:00 +CPU= 25.803
Moving to projects directory: /home/mjo/slots/7/ICMGGhq0f+002772
Moving to projects directory: /home/mjo/slots/7/ICMSHhq0f+002772
Moving to projects directory: /home/mjo/slots/7/ICMUAhq0f+002772
  06:52:01 STEP 2773 H=2773:00 +CPU= 15.079
  06:52:16 STEP 2774 H=2774:00 +CPU= 15.404
  06:52:40 STEP 2775 H=2775:00 +CPU= 23.338
  06:52:56 STEP 2776 H=2776:00 +CPU= 15.713
  06:53:11 STEP 2777 H=2777:00 +CPU= 15.009
  06:53:34 STEP 2778 H=2778:00 +CPU= 23.577
  06:53:49 STEP 2779 H=2779:00 +CPU= 14.923
  06:54:05 STEP 2780 H=2780:00 +CPU= 15.362
  06:54:28 STEP 2781 H=2781:00 +CPU= 23.125
  06:54:44 STEP 2782 H=2782:00 +CPU= 15.362
  06:54:59 STEP 2783 H=2783:00 +CPU= 15.285
  06:55:26 STEP 2784 H=2784:00 +CPU= 27.089
Moving to projects directory: /home/mjo/slots/7/ICMGGhq0f+002784
Moving to projects directory: /home/mjo/slots/7/ICMSHhq0f+002784
Moving to projects directory: /home/mjo/slots/7/ICMUAhq0f+002784
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMGGhq0f+002760
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMSHhq0f+002760
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMUAhq0f+002760
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMGGhq0f+002772
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMSHhq0f+002772
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMUAhq0f+002772
Zipping up the intermediate file: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_0203_2008050100_123_977_12193847_0_r2002178832_115.zip
Uploading the intermediate file: upload_file_115.zip
  06:55:41 STEP 2785 H=2785:00 +CPU= 14.952
  06:55:57 STEP 2786 H=2786:00 +CPU= 15.204
  06:56:20 STEP 2787 H=2787:00 +CPU= 23.431
  06:56:36 STEP 2788 H=2788:00 +CPU= 15.367
  06:56:51 STEP 2789 H=2789:00 +CPU= 14.847
  06:57:14 STEP 2790 H=2790:00 +CPU= 23.418
  06:57:29 STEP 2791 H=2791:00 +CPU= 14.848
  06:57:45 STEP 2792 H=2792:00 +CPU= 15.209
  06:58:08 STEP 2793 H=2793:00 +CPU= 23.055
  06:58:24 STEP 2794 H=2794:00 +CPU= 15.352
  06:58:39 STEP 2795 H=2795:00 +CPU= 15.049
  06:59:05 STEP 2796 H=2796:00 +CPU= 25.679
Moving to projects directory: /home/mjo/slots/7/ICMGGhq0f+002796
Moving to projects directory: /home/mjo/slots/7/ICMSHhq0f+002796
Moving to projects directory: /home/mjo/slots/7/ICMUAhq0f+002796
  06:59:20 STEP 2797 H=2797:00 +CPU= 14.845
Suspend request received from the BOINC client, suspending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 2008050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0203
(argv4) batchid: 977
(argv5) wuid: 12193847
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /home/mjo/slots/7
Project directory is: /home/mjo/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847
..mkdir for temp folder for results failed
Copying: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /home/mjo/slots/7/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /home/mjo/slots/7/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_0b9eeb553a84bac8bcd38c8e4b187183 to: /home/mjo/slots/7/oifs_43r3_ps_0203_2008050100_123_977_12193847.zip
Unzipping the namelist zip file: /home/mjo/slots/7/oifs_43r3_ps_0203_2008050100_123_977_12193847.zip
ic_ancil_file: ic_ancil_12193847
ifsdata_file: ifsdata_12193847
climate_data_file: clim_data_12193847
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_e993ac67de0ac3e03b350a8496cf1bc4 to: /home/mjo/slots/7/ic_ancil_12193847.zip
Unzipping the IC ancils zip file: /home/mjo/slots/7/ic_ancil_12193847.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_760b10cdd533f9bc0a24208b23649378 to: /home/mjo/slots/7/ifsdata/ifsdata_12193847.zip
Unzipping the ifsdata_zip file: /home/mjo/slots/7/ifsdata/ifsdata_12193847.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /home/mjo/slots/7/159l_2/clim_data_12193847.zip
Unzipping the climate data zip file: /home/mjo/slots/7/159l_2/clim_data_12193847.zip
Checking for progress XML file: /home/mjo/slots/7/progress_file_12193847.xml
Opened progress file ok : /home/mjo/slots/7/progress_file_12193847.xml
-- Model is restarting --
Adjusting last_iter, 2798, to previous model restart step.
Creating progress file: /home/mjo/slots/7/progress_file_12193847.xml
last_cpu_time: 51506
upload_file_number: 116
last_iter: 2783
last_upload: 10022400
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0203_2008050100_123_977_12193847_0_r2002178832
The child process has been launched with process id: 3214
Executing the command: /home/mjo/slots/7/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:mjo-pc-fedora:1:1:3214:3214] [20221227:204413:1672170253:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:mjo-pc-fedora:1:1:3214:3214] [20221227:204413:1672170253:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:mjo-pc-fedora:1:1:3214:3214] [20221227:204413:1672170253:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:mjo-pc-fedora:1:1:3214:3214] [20221227:204413:1672170253:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:mjo-pc-fedora:1:1:3214:3214] [20221227:204413:1672170253:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:mjo-pc-fedora:1:1:3214:3214] [20221227:204413:1672170253:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:mjo-pc-fedora:1:1:3214:3214] [20221227:204413:1672170253:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:mjo-pc-fedora:1:1:3214:3214] [20221227:204413:1672170253:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:mjo-pc-fedora:1:1:3214:3214] [20221227:204413:1672170253:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:mjo-pc-fedora:1:1:3214:3214] [20221227:204413:1672170253:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:mjo-pc-fedora:1:1:3214:3214] [20221227:204413:1672170253:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:mjo-pc-fedora:1:1:3214:3214] [20221227:204413:1672170253:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:mjo-pc-fedora:1:1:3214:3214] [20221227:204413:1672170253:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:mjo-pc-fedora:1:1:3214:3214] [20221227:204413:1672170253:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:mjo-pc-fedora:1:1:3214:3214] [20221227:204413:1672170253:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:mjo-pc-fedora:1:1:3214:3214] [20221227:204413:1672170253:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:mjo-pc-fedora:1:1:3214:3214] [20221227:204413:1672170253:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:mjo-pc-fedora:1:1:3214:3214] [20221227:204413:1672170253:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:mjo-pc-fedora:1:1:3214:3214] [20221227:204413:1672170253:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:mjo-pc-fedora:1:1:3214:3214] [20221227:204413:1672170253:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:mjo-pc-fedora:1:1:3214:3214] [20221227:204413:1672170253:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:mjo-pc-fedora:1:1:3214:3214] [20221227:204413:1672170253:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:mjo-pc-fedora:1:1:3214:3214] [20221227:204413:1672170253:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:mjo-pc-fedora:1:1:3214:3214] [20221227:204413:1672170253:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:mjo-pc-fedora:1:1:3214:3214] [20221227:204413:1672170253:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:mjo-pc-fedora:1:1:3214:3214] [20221227:204413:1672170253:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:mjo-pc-fedora:1:1:3214:3214] [20221227:204413:1672170253:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:mjo-pc-fedora:1:1:3214:3214] [20221227:204413:1672170253:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:mjo-pc-fedora:1:1:3214:3214] [20221227:204413:1672170253:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:mjo-pc-fedora:1:1:3214:3214] [20221227:204413:1672170253:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  20:44:46 STEP 2784 H=2784:00 +CPU= 27.716
  20:45:02 STEP 2785 H=2785:00 +CPU= 15.529
  20:45:18 STEP 2786 H=2786:00 +CPU= 15.819
  20:45:42 STEP 2787 H=2787:00 +CPU= 23.397
  20:45:58 STEP 2788 H=2788:00 +CPU= 15.680
  20:46:14 STEP 2789 H=2789:00 +CPU= 15.268
  20:46:38 STEP 2790 H=2790:00 +CPU= 24.026
  20:46:54 STEP 2791 H=2791:00 +CPU= 15.321
  20:47:11 STEP 2792 H=2792:00 +CPU= 16.197
  20:47:36 STEP 2793 H=2793:00 +CPU= 23.693
  20:47:52 STEP 2794 H=2794:00 +CPU= 15.700
  20:48:07 STEP 2795 H=2795:00 +CPU= 15.307
  20:48:34 STEP 2796 H=2796:00 +CPU= 25.960
Moving to projects directory: /home/mjo/slots/7/ICMGGhq0f+002796
Moving to projects directory: /home/mjo/slots/7/ICMSHhq0f+002796
Moving to projects directory: /home/mjo/slots/7/ICMUAhq0f+002796
  20:48:50 STEP 2797 H=2797:00 +CPU= 15.393
  20:49:06 STEP 2798 H=2798:00 +CPU= 15.786
  20:49:30 STEP 2799 H=2799:00 +CPU= 23.654
  20:49:46 STEP 2800 H=2800:00 +CPU= 15.677
  20:50:01 STEP 2801 H=2801:00 +CPU= 15.053
  20:50:25 STEP 2802 H=2802:00 +CPU= 23.837
  20:50:40 STEP 2803 H=2803:00 +CPU= 14.922
  20:50:56 STEP 2804 H=2804:00 +CPU= 15.390
  20:51:19 STEP 2805 H=2805:00 +CPU= 23.093
  20:51:34 STEP 2806 H=2806:00 +CPU= 15.332
  20:51:50 STEP 2807 H=2807:00 +CPU= 15.486
  20:52:18 STEP 2808 H=2808:00 +CPU= 27.292
Moving to projects directory: /home/mjo/slots/7/ICMGGhq0f+002808
Moving to projects directory: /home/mjo/slots/7/ICMSHhq0f+002808
Moving to projects directory: /home/mjo/slots/7/ICMUAhq0f+002808
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMGGhq0f+002784
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMSHhq0f+002784
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMUAhq0f+002784
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMGGhq0f+002796
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMSHhq0f+002796
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMUAhq0f+002796
Zipping up the intermediate file: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_0203_2008050100_123_977_12193847_0_r2002178832_116.zip
Uploading the intermediate file: upload_file_116.zip
  20:52:33 STEP 2809 H=2809:00 +CPU= 14.947
  20:52:49 STEP 2810 H=2810:00 +CPU= 15.486
  20:53:12 STEP 2811 H=2811:00 +CPU= 23.143
  20:53:28 STEP 2812 H=2812:00 +CPU= 15.578
  20:53:44 STEP 2813 H=2813:00 +CPU= 15.549
  20:54:09 STEP 2814 H=2814:00 +CPU= 24.194
  20:54:25 STEP 2815 H=2815:00 +CPU= 15.159
  20:54:41 STEP 2816 H=2816:00 +CPU= 15.591
  20:55:04 STEP 2817 H=2817:00 +CPU= 23.068
  20:55:20 STEP 2818 H=2818:00 +CPU= 15.546
  20:55:36 STEP 2819 H=2819:00 +CPU= 15.613
  20:56:04 STEP 2820 H=2820:00 +CPU= 26.394
Moving to projects directory: /home/mjo/slots/7/ICMGGhq0f+002820
Moving to projects directory: /home/mjo/slots/7/ICMSHhq0f+002820
Moving to projects directory: /home/mjo/slots/7/ICMUAhq0f+002820
  20:56:20 STEP 2821 H=2821:00 +CPU= 15.284
  20:56:35 STEP 2822 H=2822:00 +CPU= 15.584
  20:56:59 STEP 2823 H=2823:00 +CPU= 23.261
  20:57:15 STEP 2824 H=2824:00 +CPU= 15.689
  20:57:31 STEP 2825 H=2825:00 +CPU= 15.088
  20:57:55 STEP 2826 H=2826:00 +CPU= 23.825
  20:58:10 STEP 2827 H=2827:00 +CPU= 15.103
  20:58:26 STEP 2828 H=2828:00 +CPU= 15.672
  20:58:49 STEP 2829 H=2829:00 +CPU= 23.182
  20:59:05 STEP 2830 H=2830:00 +CPU= 15.382
  20:59:20 STEP 2831 H=2831:00 +CPU= 15.332
  20:59:49 STEP 2832 H=2832:00 +CPU= 27.418
Moving to projects directory: /home/mjo/slots/7/ICMGGhq0f+002832
Moving to projects directory: /home/mjo/slots/7/ICMSHhq0f+002832
Moving to projects directory: /home/mjo/slots/7/ICMUAhq0f+002832
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMGGhq0f+002808
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMSHhq0f+002808
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMUAhq0f+002808
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMGGhq0f+002820
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMSHhq0f+002820
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMUAhq0f+002820
Zipping up the intermediate file: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_0203_2008050100_123_977_12193847_0_r2002178832_117.zip
Uploading the intermediate file: upload_file_117.zip
  21:00:04 STEP 2833 H=2833:00 +CPU= 15.054
  21:00:19 STEP 2834 H=2834:00 +CPU= 15.298
  21:00:42 STEP 2835 H=2835:00 +CPU= 23.013
  21:00:58 STEP 2836 H=2836:00 +CPU= 15.327
  21:01:13 STEP 2837 H=2837:00 +CPU= 14.965
  21:01:37 STEP 2838 H=2838:00 +CPU= 23.378
  21:01:51 STEP 2839 H=2839:00 +CPU= 14.836
  21:02:07 STEP 2840 H=2840:00 +CPU= 15.281
  21:02:30 STEP 2841 H=2841:00 +CPU= 22.914
  21:02:45 STEP 2842 H=2842:00 +CPU= 15.207
  21:03:00 STEP 2843 H=2843:00 +CPU= 14.756
  21:03:26 STEP 2844 H=2844:00 +CPU= 25.426
Moving to projects directory: /home/mjo/slots/7/ICMGGhq0f+002844
Moving to projects directory: /home/mjo/slots/7/ICMSHhq0f+002844
Moving to projects directory: /home/mjo/slots/7/ICMUAhq0f+002844
  21:03:41 STEP 2845 H=2845:00 +CPU= 14.898
  21:03:57 STEP 2846 H=2846:00 +CPU= 15.421
  21:04:21 STEP 2847 H=2847:00 +CPU= 23.793
  21:04:37 STEP 2848 H=2848:00 +CPU= 15.548
  21:04:53 STEP 2849 H=2849:00 +CPU= 15.084
  21:05:18 STEP 2850 H=2850:00 +CPU= 24.494
  21:05:34 STEP 2851 H=2851:00 +CPU= 15.383
  21:05:51 STEP 2852 H=2852:00 +CPU= 15.803
  21:06:15 STEP 2853 H=2853:00 +CPU= 23.572
  21:06:32 STEP 2854 H=2854:00 +CPU= 16.062
  21:06:49 STEP 2855 H=2855:00 +CPU= 15.985
  21:07:19 STEP 2856 H=2856:00 +CPU= 28.107
Moving to projects directory: /home/mjo/slots/7/ICMGGhq0f+002856
Moving to projects directory: /home/mjo/slots/7/ICMSHhq0f+002856
Moving to projects directory: /home/mjo/slots/7/ICMUAhq0f+002856
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMGGhq0f+002832
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMSHhq0f+002832
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMUAhq0f+002832
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMGGhq0f+002844
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMSHhq0f+002844
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMUAhq0f+002844
Zipping up the intermediate file: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_0203_2008050100_123_977_12193847_0_r2002178832_118.zip
Uploading the intermediate file: upload_file_118.zip
  21:07:35 STEP 2857 H=2857:00 +CPU= 15.657
  21:07:51 STEP 2858 H=2858:00 +CPU= 15.636
  21:08:16 STEP 2859 H=2859:00 +CPU= 23.537
  21:08:32 STEP 2860 H=2860:00 +CPU= 15.805
  21:08:48 STEP 2861 H=2861:00 +CPU= 15.524
  21:09:14 STEP 2862 H=2862:00 +CPU= 24.510
  21:09:31 STEP 2863 H=2863:00 +CPU= 15.852
  21:09:48 STEP 2864 H=2864:00 +CPU= 16.091
  21:10:13 STEP 2865 H=2865:00 +CPU= 23.939
  21:10:30 STEP 2866 H=2866:00 +CPU= 16.213
  21:10:46 STEP 2867 H=2867:00 +CPU= 15.598
  21:11:14 STEP 2868 H=2868:00 +CPU= 26.335
Moving to projects directory: /home/mjo/slots/7/ICMGGhq0f+002868
Moving to projects directory: /home/mjo/slots/7/ICMSHhq0f+002868
Moving to projects directory: /home/mjo/slots/7/ICMUAhq0f+002868
  21:11:30 STEP 2869 H=2869:00 +CPU= 15.516
  21:11:47 STEP 2870 H=2870:00 +CPU= 15.939
  21:12:12 STEP 2871 H=2871:00 +CPU= 23.695
  21:12:28 STEP 2872 H=2872:00 +CPU= 15.906
  21:12:44 STEP 2873 H=2873:00 +CPU= 15.462
  21:13:10 STEP 2874 H=2874:00 +CPU= 24.092
  21:13:26 STEP 2875 H=2875:00 +CPU= 15.409
  21:13:42 STEP 2876 H=2876:00 +CPU= 15.856
  21:14:07 STEP 2877 H=2877:00 +CPU= 23.541
  21:14:23 STEP 2878 H=2878:00 +CPU= 15.685
  21:14:39 STEP 2879 H=2879:00 +CPU= 15.966
  21:15:09 STEP 2880 H=2880:00 +CPU= 27.833
Moving to projects directory: /home/mjo/slots/7/ICMGGhq0f+002880
Moving to projects directory: /home/mjo/slots/7/ICMSHhq0f+002880
Moving to projects directory: /home/mjo/slots/7/ICMUAhq0f+002880
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMGGhq0f+002856
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMSHhq0f+002856
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMUAhq0f+002856
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMGGhq0f+002868
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMSHhq0f+002868
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMUAhq0f+002868
Zipping up the intermediate file: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_0203_2008050100_123_977_12193847_0_r2002178832_119.zip
Uploading the intermediate file: upload_file_119.zip
  21:15:24 STEP 2881 H=2881:00 +CPU= 15.141
  21:15:40 STEP 2882 H=2882:00 +CPU= 15.554
  21:16:05 STEP 2883 H=2883:00 +CPU= 23.607
  21:16:22 STEP 2884 H=2884:00 +CPU= 15.812
  21:16:38 STEP 2885 H=2885:00 +CPU= 15.398
  21:17:03 STEP 2886 H=2886:00 +CPU= 24.155
  21:17:19 STEP 2887 H=2887:00 +CPU= 15.466
  21:17:36 STEP 2888 H=2888:00 +CPU= 15.791
  21:18:01 STEP 2889 H=2889:00 +CPU= 23.651
  21:18:17 STEP 2890 H=2890:00 +CPU= 15.833
  21:18:34 STEP 2891 H=2891:00 +CPU= 15.672
  21:19:01 STEP 2892 H=2892:00 +CPU= 26.204
Moving to projects directory: /home/mjo/slots/7/ICMGGhq0f+002892
Moving to projects directory: /home/mjo/slots/7/ICMSHhq0f+002892
Moving to projects directory: /home/mjo/slots/7/ICMUAhq0f+002892
  21:19:18 STEP 2893 H=2893:00 +CPU= 15.691
  21:19:34 STEP 2894 H=2894:00 +CPU= 15.825
  21:19:59 STEP 2895 H=2895:00 +CPU= 23.656
  21:20:16 STEP 2896 H=2896:00 +CPU= 15.812
  21:20:32 STEP 2897 H=2897:00 +CPU= 15.413
  21:20:57 STEP 2898 H=2898:00 +CPU= 24.205
  21:21:14 STEP 2899 H=2899:00 +CPU= 15.594
  21:21:30 STEP 2900 H=2900:00 +CPU= 15.972
  21:21:55 STEP 2901 H=2901:00 +CPU= 23.789
  21:22:12 STEP 2902 H=2902:00 +CPU= 15.924
  21:22:28 STEP 2903 H=2903:00 +CPU= 15.977
  21:22:58 STEP 2904 H=2904:00 +CPU= 28.194
Moving to projects directory: /home/mjo/slots/7/ICMGGhq0f+002904
Moving to projects directory: /home/mjo/slots/7/ICMSHhq0f+002904
Moving to projects directory: /home/mjo/slots/7/ICMUAhq0f+002904
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMGGhq0f+002880
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMSHhq0f+002880
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMUAhq0f+002880
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMGGhq0f+002892
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMSHhq0f+002892
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMUAhq0f+002892
Zipping up the intermediate file: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_0203_2008050100_123_977_12193847_0_r2002178832_120.zip
Uploading the intermediate file: upload_file_120.zip
  21:23:15 STEP 2905 H=2905:00 +CPU= 15.457
  21:23:31 STEP 2906 H=2906:00 +CPU= 15.971
  21:23:56 STEP 2907 H=2907:00 +CPU= 23.887
  21:24:12 STEP 2908 H=2908:00 +CPU= 15.916
  21:24:29 STEP 2909 H=2909:00 +CPU= 15.598
  21:24:54 STEP 2910 H=2910:00 +CPU= 24.380
  21:25:10 STEP 2911 H=2911:00 +CPU= 15.516
  21:25:27 STEP 2912 H=2912:00 +CPU= 15.972
  21:25:52 STEP 2913 H=2913:00 +CPU= 23.989
  21:26:08 STEP 2914 H=2914:00 +CPU= 15.960
  21:26:24 STEP 2915 H=2915:00 +CPU= 15.536
  21:26:52 STEP 2916 H=2916:00 +CPU= 26.416
Moving to projects directory: /home/mjo/slots/7/ICMGGhq0f+002916
Moving to projects directory: /home/mjo/slots/7/ICMSHhq0f+002916
Moving to projects directory: /home/mjo/slots/7/ICMUAhq0f+002916
  21:27:08 STEP 2917 H=2917:00 +CPU= 15.551
  21:27:25 STEP 2918 H=2918:00 +CPU= 16.049
  21:27:50 STEP 2919 H=2919:00 +CPU= 23.923
  21:28:06 STEP 2920 H=2920:00 +CPU= 16.014
  21:28:22 STEP 2921 H=2921:00 +CPU= 15.472
  21:28:48 STEP 2922 H=2922:00 +CPU= 24.482
  21:29:04 STEP 2923 H=2923:00 +CPU= 15.611
  21:29:21 STEP 2924 H=2924:00 +CPU= 16.037
  21:29:46 STEP 2925 H=2925:00 +CPU= 23.916
  21:30:03 STEP 2926 H=2926:00 +CPU= 16.474
  21:30:21 STEP 2927 H=2927:00 +CPU= 16.343
  21:30:50 STEP 2928 H=2928:00 +CPU= 27.993
Moving to projects directory: /home/mjo/slots/7/ICMGGhq0f+002928
Moving to projects directory: /home/mjo/slots/7/ICMSHhq0f+002928
Moving to projects directory: /home/mjo/slots/7/ICMUAhq0f+002928
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMGGhq0f+002904
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMSHhq0f+002904
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMUAhq0f+002904
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMGGhq0f+002916
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMSHhq0f+002916
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMUAhq0f+002916
Zipping up the intermediate file: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_0203_2008050100_123_977_12193847_0_r2002178832_121.zip
Uploading the intermediate file: upload_file_121.zip
  21:31:07 STEP 2929 H=2929:00 +CPU= 15.379
  21:31:23 STEP 2930 H=2930:00 +CPU= 15.600
  21:31:48 STEP 2931 H=2931:00 +CPU= 23.521
  21:32:04 STEP 2932 H=2932:00 +CPU= 15.559
  21:32:20 STEP 2933 H=2933:00 +CPU= 15.344
  21:32:47 STEP 2934 H=2934:00 +CPU= 25.442
  21:33:04 STEP 2935 H=2935:00 +CPU= 16.128
  21:33:21 STEP 2936 H=2936:00 +CPU= 16.156
  21:33:46 STEP 2937 H=2937:00 +CPU= 23.433
  21:34:03 STEP 2938 H=2938:00 +CPU= 15.944
  21:34:19 STEP 2939 H=2939:00 +CPU= 15.493
  21:34:47 STEP 2940 H=2940:00 +CPU= 26.496
Moving to projects directory: /home/mjo/slots/7/ICMGGhq0f+002940
Moving to projects directory: /home/mjo/slots/7/ICMSHhq0f+002940
Moving to projects directory: /home/mjo/slots/7/ICMUAhq0f+002940
  21:35:03 STEP 2941 H=2941:00 +CPU= 15.332
  21:35:20 STEP 2942 H=2942:00 +CPU= 15.884
  21:35:45 STEP 2943 H=2943:00 +CPU= 23.562
  21:36:01 STEP 2944 H=2944:00 +CPU= 15.783
  21:36:17 STEP 2945 H=2945:00 +CPU= 15.121
  21:36:42 STEP 2946 H=2946:00 +CPU= 23.820
  21:36:58 STEP 2947 H=2947:00 +CPU= 15.221
  21:37:14 STEP 2948 H=2948:00 +CPU= 15.496
  21:37:38 STEP 2949 H=2949:00 +CPU= 23.347
  21:37:54 STEP 2950 H=2950:00 +CPU= 15.374
  21:38:09 STEP 2951 H=2951:00 +CPU= 15.369
  21:38:37 STEP 2952 H=2952:00 +CPU= 26.440
The child process terminated with status: 0
Moving to projects directory: /home/mjo/slots/7/ICMGGhq0f+002952
Moving to projects directory: /home/mjo/slots/7/ICMSHhq0f+002952
Moving to projects directory: /home/mjo/slots/7/ICMUAhq0f+002952
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMGGhq0f+002928
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMSHhq0f+002928
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMUAhq0f+002928
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMGGhq0f+002940
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMSHhq0f+002940
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMUAhq0f+002940
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMGGhq0f+002952
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMSHhq0f+002952
Adding to the zip: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_12193847/ICMUAhq0f+002952
Zipping up the final file: /home/mjo/projects/climateprediction.net/oifs_43r3_ps_0203_2008050100_123_977_12193847_0_r2002178832_122.zip
Uploading the final file: upload_file_122.zip
Uploading trickle at timestep: 10623600
21:41:59 (3182): called boinc_finish(0)

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 May 2023 15:59:13 1537490 22279661 oifs_43r3_ps_0203_2008050100_123_977_12193847_0 10,623,600 54,655 0.0051
09 May 2023 15:58:12 1537490 22279661 oifs_43r3_ps_0203_2008050100_123_977_12193847_0 9,504,000 48,661 0.0051
09 May 2023 15:58:06 1537490 22279661 oifs_43r3_ps_0203_2008050100_123_977_12193847_0 8,640,000 44,302 0.0051
09 May 2023 15:58:03 1537490 22279661 oifs_43r3_ps_0203_2008050100_123_977_12193847_0 7,776,000 39,944 0.0051
09 May 2023 15:58:03 1537490 22279661 oifs_43r3_ps_0203_2008050100_123_977_12193847_0 7,776,000 39,944 0.0051
09 May 2023 15:57:58 1537490 22279661 oifs_43r3_ps_0203_2008050100_123_977_12193847_0 6,912,000 35,596 0.0051
09 May 2023 15:57:54 1537490 22279661 oifs_43r3_ps_0203_2008050100_123_977_12193847_0 6,048,000 31,248 0.0052
09 May 2023 15:57:50 1537490 22279661 oifs_43r3_ps_0203_2008050100_123_977_12193847_0 5,184,000 26,901 0.0052
09 May 2023 15:57:41 1537490 22279661 oifs_43r3_ps_0203_2008050100_123_977_12193847_0 4,320,000 22,519 0.0052
09 May 2023 15:57:36 1537490 22279661 oifs_43r3_ps_0203_2008050100_123_977_12193847_0 3,456,000 17,982 0.0052
09 May 2023 15:57:31 1537490 22279661 oifs_43r3_ps_0203_2008050100_123_977_12193847_0 2,592,000 13,597 0.0052
09 May 2023 15:57:24 1537490 22279661 oifs_43r3_ps_0203_2008050100_123_977_12193847_0 1,728,000 9,151 0.0053
09 May 2023 15:57:18 1537490 22279661 oifs_43r3_ps_0203_2008050100_123_977_12193847_0 864,000 4,549 0.0053


©2024 cpdn.org