climateprediction.net home page
Task 22295225

Task 22295225

Name oifs_43r3_ps_0469_2006050100_123_975_12192113_1
Workunit 12192113
Created 2 Jan 2023, 16:22:50 UTC
Sent 23 Jan 2023, 11:48:07 UTC
Report deadline 24 Mar 2023, 11:48:07 UTC
Received 28 Jan 2023, 11:22:49 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 1 (0x00000001) Unknown error code
Computer ID 1536027
Run time 12 hours 31 min 31 sec
CPU time 12 hours 30 min 42 sec
Validate state Invalid
Credit 14,636.16
Device peak FLOPS 7.20 GFLOPS
Application version OpenIFS 43r3 Perturbed Surface v1.05
x86_64-pc-linux-gnu
Peak working set size 4,798.08 MB
Peak swap size 4,978.00 MB
Peak disk usage 2,828.99 MB
Stderr
<core_client_version>7.16.6</core_client_version>
<![CDATA[
<message>
process exited with code 1 (0x1, -255)</message>
<stderr_txt>
b/boinc-client/slots/3/ICMUAhq0f+002472
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMGGhq0f+002448
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMSHhq0f+002448
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMUAhq0f+002448
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMGGhq0f+002460
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMSHhq0f+002460
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMUAhq0f+002460
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0469_2006050100_123_975_12192113_1_r1500204407_102.zip
Uploading the intermediate file: upload_file_102.zip
  21:03:58 STEP 2473 H=2473:00 +CPU= 13.618
  21:04:11 STEP 2474 H=2474:00 +CPU= 12.822
  21:04:31 STEP 2475 H=2475:00 +CPU= 19.680
  21:04:44 STEP 2476 H=2476:00 +CPU= 13.221
  21:04:58 STEP 2477 H=2477:00 +CPU= 13.201
  21:05:19 STEP 2478 H=2478:00 +CPU= 20.712
  21:05:33 STEP 2479 H=2479:00 +CPU= 13.081
  21:05:46 STEP 2480 H=2480:00 +CPU= 13.001
  21:06:06 STEP 2481 H=2481:00 +CPU= 19.961
  21:06:19 STEP 2482 H=2482:00 +CPU= 13.178
  21:06:33 STEP 2483 H=2483:00 +CPU= 13.199
  21:06:55 STEP 2484 H=2484:00 +CPU= 21.693
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+002484
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+002484
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+002484
  21:07:12 STEP 2485 H=2485:00 +CPU= 14.785
  21:07:27 STEP 2486 H=2486:00 +CPU= 13.207
  21:07:48 STEP 2487 H=2487:00 +CPU= 20.966
  21:08:02 STEP 2488 H=2488:00 +CPU= 13.169
  21:08:15 STEP 2489 H=2489:00 +CPU= 12.950
  21:08:35 STEP 2490 H=2490:00 +CPU= 19.613
  21:08:48 STEP 2491 H=2491:00 +CPU= 12.668
  21:09:02 STEP 2492 H=2492:00 +CPU= 13.429
  21:09:20 STEP 2493 H=2493:00 +CPU= 18.441
  21:09:34 STEP 2494 H=2494:00 +CPU= 13.196
  21:09:47 STEP 2495 H=2495:00 +CPU= 12.871
  21:10:10 STEP 2496 H=2496:00 +CPU= 22.130
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+002496
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+002496
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+002496
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMGGhq0f+002472
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMSHhq0f+002472
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMUAhq0f+002472
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMGGhq0f+002484
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMSHhq0f+002484
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMUAhq0f+002484
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0469_2006050100_123_975_12192113_1_r1500204407_103.zip
Uploading the intermediate file: upload_file_103.zip
  21:10:23 STEP 2497 H=2497:00 +CPU= 13.282
  21:10:37 STEP 2498 H=2498:00 +CPU= 13.564
  21:10:57 STEP 2499 H=2499:00 +CPU= 19.083
  21:11:11 STEP 2500 H=2500:00 +CPU= 13.753
  21:11:24 STEP 2501 H=2501:00 +CPU= 12.907
  21:11:44 STEP 2502 H=2502:00 +CPU= 19.442
  21:11:57 STEP 2503 H=2503:00 +CPU= 12.617
  21:12:10 STEP 2504 H=2504:00 +CPU= 12.543
  21:12:29 STEP 2505 H=2505:00 +CPU= 19.131
  21:12:43 STEP 2506 H=2506:00 +CPU= 13.621
  21:12:56 STEP 2507 H=2507:00 +CPU= 12.699
  21:13:17 STEP 2508 H=2508:00 +CPU= 21.565
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+002508
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+002508
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+002508
  21:13:31 STEP 2509 H=2509:00 +CPU= 13.049
  21:13:45 STEP 2510 H=2510:00 +CPU= 13.440
  21:14:04 STEP 2511 H=2511:00 +CPU= 19.129
  21:14:17 STEP 2512 H=2512:00 +CPU= 12.986
  21:14:30 STEP 2513 H=2513:00 +CPU= 13.165
  21:14:50 STEP 2514 H=2514:00 +CPU= 19.701
  21:15:03 STEP 2515 H=2515:00 +CPU= 13.230
  21:15:17 STEP 2516 H=2516:00 +CPU= 13.272
  21:15:37 STEP 2517 H=2517:00 +CPU= 19.619
  21:15:50 STEP 2518 H=2518:00 +CPU= 12.914
  21:16:04 STEP 2519 H=2519:00 +CPU= 13.549
  21:16:27 STEP 2520 H=2520:00 +CPU= 22.514
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+002520
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+002520
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+002520
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMGGhq0f+002496
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMSHhq0f+002496
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMUAhq0f+002496
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMGGhq0f+002508
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMSHhq0f+002508
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMUAhq0f+002508
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0469_2006050100_123_975_12192113_1_r1500204407_104.zip
Uploading the intermediate file: upload_file_104.zip
  21:16:41 STEP 2521 H=2521:00 +CPU= 12.895
  21:16:54 STEP 2522 H=2522:00 +CPU= 13.305
  21:17:14 STEP 2523 H=2523:00 +CPU= 19.129
  21:17:27 STEP 2524 H=2524:00 +CPU= 13.577
  21:17:40 STEP 2525 H=2525:00 +CPU= 12.246
  21:18:00 STEP 2526 H=2526:00 +CPU= 19.655
  21:18:13 STEP 2527 H=2527:00 +CPU= 12.687
  21:18:26 STEP 2528 H=2528:00 +CPU= 12.638
  21:18:46 STEP 2529 H=2529:00 +CPU= 19.220
  21:19:00 STEP 2530 H=2530:00 +CPU= 13.816
  21:19:13 STEP 2531 H=2531:00 +CPU= 13.242
  21:19:35 STEP 2532 H=2532:00 +CPU= 21.331
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+002532
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+002532
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+002532
  21:19:48 STEP 2533 H=2533:00 +CPU= 12.844
  21:20:01 STEP 2534 H=2534:00 +CPU= 13.217
  21:20:20 STEP 2535 H=2535:00 +CPU= 19.150
  21:20:33 STEP 2536 H=2536:00 +CPU= 12.916
  21:20:47 STEP 2537 H=2537:00 +CPU= 13.240
  21:21:06 STEP 2538 H=2538:00 +CPU= 19.069
  21:21:18 STEP 2539 H=2539:00 +CPU= 12.511
  21:21:32 STEP 2540 H=2540:00 +CPU= 13.986
  21:21:52 STEP 2541 H=2541:00 +CPU= 19.121
  21:22:06 STEP 2542 H=2542:00 +CPU= 14.390
  21:22:19 STEP 2543 H=2543:00 +CPU= 12.690
  21:22:42 STEP 2544 H=2544:00 +CPU= 22.771
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+002544
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+002544
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+002544
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMGGhq0f+002520
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMSHhq0f+002520
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMUAhq0f+002520
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMGGhq0f+002532
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMSHhq0f+002532
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMUAhq0f+002532
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0469_2006050100_123_975_12192113_1_r1500204407_105.zip
Uploading the intermediate file: upload_file_105.zip
  21:22:55 STEP 2545 H=2545:00 +CPU= 12.746
  21:23:09 STEP 2546 H=2546:00 +CPU= 13.825
Uploading trickle at timestep: 9158400
  21:23:28 STEP 2547 H=2547:00 +CPU= 18.948
  21:23:41 STEP 2548 H=2548:00 +CPU= 13.404
  21:23:54 STEP 2549 H=2549:00 +CPU= 12.673
  21:24:13 STEP 2550 H=2550:00 +CPU= 19.223
  21:24:26 STEP 2551 H=2551:00 +CPU= 12.583
  21:24:39 STEP 2552 H=2552:00 +CPU= 13.259
  21:24:58 STEP 2553 H=2553:00 +CPU= 19.055
  21:25:12 STEP 2554 H=2554:00 +CPU= 13.147
  21:25:24 STEP 2555 H=2555:00 +CPU= 12.659
  21:25:45 STEP 2556 H=2556:00 +CPU= 20.450
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+002556
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+002556
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+002556
  21:25:57 STEP 2557 H=2557:00 +CPU= 12.357
  21:26:10 STEP 2558 H=2558:00 +CPU= 12.682
  21:26:29 STEP 2559 H=2559:00 +CPU= 19.102
  21:26:42 STEP 2560 H=2560:00 +CPU= 13.116
  21:26:55 STEP 2561 H=2561:00 +CPU= 12.272
  21:27:15 STEP 2562 H=2562:00 +CPU= 20.062
  21:27:26 STEP 2563 H=2563:00 +CPU= 11.678
  21:27:39 STEP 2564 H=2564:00 +CPU= 12.698
  21:27:59 STEP 2565 H=2565:00 +CPU= 20.189
  21:28:12 STEP 2566 H=2566:00 +CPU= 12.785
  21:28:26 STEP 2567 H=2567:00 +CPU= 13.780
  21:28:50 STEP 2568 H=2568:00 +CPU= 23.552
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+002568
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+002568
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+002568
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMGGhq0f+002544
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMSHhq0f+002544
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMUAhq0f+002544
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMGGhq0f+002556
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMSHhq0f+002556
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMUAhq0f+002556
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0469_2006050100_123_975_12192113_1_r1500204407_106.zip
Uploading the intermediate file: upload_file_106.zip
  21:29:03 STEP 2569 H=2569:00 +CPU= 12.969
  21:29:15 STEP 2570 H=2570:00 +CPU= 12.826
  21:29:35 STEP 2571 H=2571:00 +CPU= 19.286
  21:29:48 STEP 2572 H=2572:00 +CPU= 12.974
  21:30:01 STEP 2573 H=2573:00 +CPU= 12.793
  21:30:20 STEP 2574 H=2574:00 +CPU= 19.471
  21:30:32 STEP 2575 H=2575:00 +CPU= 11.972
  21:30:46 STEP 2576 H=2576:00 +CPU= 13.486
  21:31:04 STEP 2577 H=2577:00 +CPU= 18.798
  21:31:18 STEP 2578 H=2578:00 +CPU= 13.172
  21:31:31 STEP 2579 H=2579:00 +CPU= 12.879
  21:31:52 STEP 2580 H=2580:00 +CPU= 21.655
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+002580
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+002580
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+002580
  21:32:06 STEP 2581 H=2581:00 +CPU= 13.475
  21:32:19 STEP 2582 H=2582:00 +CPU= 12.914
  21:32:40 STEP 2583 H=2583:00 +CPU= 20.739
  21:32:55 STEP 2584 H=2584:00 +CPU= 14.736
  21:33:10 STEP 2585 H=2585:00 +CPU= 14.654
  21:33:32 STEP 2586 H=2586:00 +CPU= 21.564
  21:33:48 STEP 2587 H=2587:00 +CPU= 15.979
  21:34:04 STEP 2588 H=2588:00 +CPU= 15.326
  21:34:26 STEP 2589 H=2589:00 +CPU= 21.744
  21:34:41 STEP 2590 H=2590:00 +CPU= 15.144
  21:34:56 STEP 2591 H=2591:00 +CPU= 15.265
  21:35:21 STEP 2592 H=2592:00 +CPU= 24.947
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+002592
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+002592
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+002592
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMGGhq0f+002568
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMSHhq0f+002568
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMUAhq0f+002568
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMGGhq0f+002580
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMSHhq0f+002580
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMUAhq0f+002580
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0469_2006050100_123_975_12192113_1_r1500204407_107.zip
Uploading the intermediate file: upload_file_107.zip
  21:35:35 STEP 2593 H=2593:00 +CPU= 13.728
  21:35:50 STEP 2594 H=2594:00 +CPU= 15.232
  21:36:12 STEP 2595 H=2595:00 +CPU= 21.041
  21:36:27 STEP 2596 H=2596:00 +CPU= 14.844
  21:36:41 STEP 2597 H=2597:00 +CPU= 14.398
  21:37:04 STEP 2598 H=2598:00 +CPU= 22.141
  21:37:18 STEP 2599 H=2599:00 +CPU= 13.283
  21:37:34 STEP 2600 H=2600:00 +CPU= 15.116
  21:37:57 STEP 2601 H=2601:00 +CPU= 21.898
  21:38:12 STEP 2602 H=2602:00 +CPU= 14.860
  21:38:25 STEP 2603 H=2603:00 +CPU= 13.564
  21:38:50 STEP 2604 H=2604:00 +CPU= 24.163
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+002604
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+002604
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+002604
  21:39:04 STEP 2605 H=2605:00 +CPU= 13.513
  21:39:20 STEP 2606 H=2606:00 +CPU= 15.278
  21:39:42 STEP 2607 H=2607:00 +CPU= 21.441
  21:39:57 STEP 2608 H=2608:00 +CPU= 14.986
  21:40:12 STEP 2609 H=2609:00 +CPU= 14.664
  21:40:34 STEP 2610 H=2610:00 +CPU= 21.777
  21:40:48 STEP 2611 H=2611:00 +CPU= 13.871
  21:41:04 STEP 2612 H=2612:00 +CPU= 14.762
  21:41:25 STEP 2613 H=2613:00 +CPU= 21.414
  21:41:40 STEP 2614 H=2614:00 +CPU= 14.545
  21:41:56 STEP 2615 H=2615:00 +CPU= 15.387
  21:42:24 STEP 2616 H=2616:00 +CPU= 27.141
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+002616
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+002616
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+002616
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMGGhq0f+002592
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMSHhq0f+002592
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMUAhq0f+002592
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMGGhq0f+002604
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMSHhq0f+002604
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMUAhq0f+002604
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0469_2006050100_123_975_12192113_1_r1500204407_108.zip
Uploading the intermediate file: upload_file_108.zip
  21:42:38 STEP 2617 H=2617:00 +CPU= 14.126
  21:42:53 STEP 2618 H=2618:00 +CPU= 14.823
  21:43:16 STEP 2619 H=2619:00 +CPU= 22.370
  21:43:30 STEP 2620 H=2620:00 +CPU= 14.139
  21:43:46 STEP 2621 H=2621:00 +CPU= 15.009
  21:44:09 STEP 2622 H=2622:00 +CPU= 23.643
  21:44:24 STEP 2623 H=2623:00 +CPU= 14.615
  21:44:40 STEP 2624 H=2624:00 +CPU= 14.696
  21:45:02 STEP 2625 H=2625:00 +CPU= 21.653
  21:45:16 STEP 2626 H=2626:00 +CPU= 13.775
  21:45:31 STEP 2627 H=2627:00 +CPU= 14.641
  21:45:55 STEP 2628 H=2628:00 +CPU= 24.053
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+002628
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+002628
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+002628
  21:46:09 STEP 2629 H=2629:00 +CPU= 14.244
  21:46:24 STEP 2630 H=2630:00 +CPU= 14.491
  21:46:45 STEP 2631 H=2631:00 +CPU= 20.882
  21:46:59 STEP 2632 H=2632:00 +CPU= 13.735
  21:47:14 STEP 2633 H=2633:00 +CPU= 14.392
  21:47:36 STEP 2634 H=2634:00 +CPU= 21.731
  21:47:50 STEP 2635 H=2635:00 +CPU= 13.816
  21:48:05 STEP 2636 H=2636:00 +CPU= 14.402
  21:48:27 STEP 2637 H=2637:00 +CPU= 21.827
  21:48:42 STEP 2638 H=2638:00 +CPU= 14.506
  21:48:56 STEP 2639 H=2639:00 +CPU= 14.096
  21:49:19 STEP 2640 H=2640:00 +CPU= 22.738
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+002640
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+002640
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+002640
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMGGhq0f+002616
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMSHhq0f+002616
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMUAhq0f+002616
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMGGhq0f+002628
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMSHhq0f+002628
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMUAhq0f+002628
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0469_2006050100_123_975_12192113_1_r1500204407_109.zip
Uploading the intermediate file: upload_file_109.zip
  21:49:31 STEP 2641 H=2641:00 +CPU= 12.241
  21:49:44 STEP 2642 H=2642:00 +CPU= 12.864
  21:50:04 STEP 2643 H=2643:00 +CPU= 19.876
  21:50:17 STEP 2644 H=2644:00 +CPU= 12.417
  21:50:30 STEP 2645 H=2645:00 +CPU= 13.236
  21:50:50 STEP 2646 H=2646:00 +CPU= 20.129
  21:51:03 STEP 2647 H=2647:00 +CPU= 12.925
  21:51:17 STEP 2648 H=2648:00 +CPU= 13.652
  21:51:37 STEP 2649 H=2649:00 +CPU= 19.842
  21:51:50 STEP 2650 H=2650:00 +CPU= 13.492
  21:52:04 STEP 2651 H=2651:00 +CPU= 13.207
  21:52:26 STEP 2652 H=2652:00 +CPU= 22.245
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+002652
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+002652
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+002652
  21:52:38 STEP 2653 H=2653:00 +CPU= 12.355
  21:52:52 STEP 2654 H=2654:00 +CPU= 13.662
  21:53:12 STEP 2655 H=2655:00 +CPU= 20.140
  21:53:26 STEP 2656 H=2656:00 +CPU= 13.365
  21:53:40 STEP 2657 H=2657:00 +CPU= 14.011
  21:54:04 STEP 2658 H=2658:00 +CPU= 23.457
  21:54:17 STEP 2659 H=2659:00 +CPU= 13.262
  21:54:32 STEP 2660 H=2660:00 +CPU= 15.092
  21:54:54 STEP 2661 H=2661:00 +CPU= 21.920
  21:55:10 STEP 2662 H=2662:00 +CPU= 15.058
  21:55:26 STEP 2663 H=2663:00 +CPU= 15.619
  21:55:53 STEP 2664 H=2664:00 +CPU= 26.857
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+002664
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+002664
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+002664
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMGGhq0f+002640
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMSHhq0f+002640
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMUAhq0f+002640
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMGGhq0f+002652
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMSHhq0f+002652
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMUAhq0f+002652
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0469_2006050100_123_975_12192113_1_r1500204407_110.zip
Uploading the intermediate file: upload_file_110.zip
  21:56:07 STEP 2665 H=2665:00 +CPU= 14.274
  21:56:24 STEP 2666 H=2666:00 +CPU= 15.805
Suspend request received from the BOINC client, suspending the child process
Quit request received from the BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 2006050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0469
(argv4) batchid: 975
(argv5) wuid: 12192113
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/3
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/3/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/3/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_f9aba43a4e7fee8a159fb5a1d5c8ac20 to: /var/lib/boinc-client/slots/3/oifs_43r3_ps_0469_2006050100_123_975_12192113.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/3/oifs_43r3_ps_0469_2006050100_123_975_12192113.zip
ic_ancil_file: ic_ancil_12192113
ifsdata_file: ifsdata_12192113
climate_data_file: clim_data_12192113
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_5e348136aae1bf47aaf292701f00994f to: /var/lib/boinc-client/slots/3/ic_ancil_12192113.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/3/ic_ancil_12192113.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_15adc4f37233ba6efed8f5aa15fe8da1 to: /var/lib/boinc-client/slots/3/ifsdata/ifsdata_12192113.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/3/ifsdata/ifsdata_12192113.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/3/159l_2/clim_data_12192113.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/3/159l_2/clim_data_12192113.zip
Checking for progress XML file: /var/lib/boinc-client/slots/3/progress_file_12192113.xml
Opened progress file ok : /var/lib/boinc-client/slots/3/progress_file_12192113.xml
-- Model is restarting --
Adjusting last_iter, 2665, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/3/progress_file_12192113.xml
last_cpu_time: 40662
upload_file_number: 111
last_iter: 2665
last_upload: 9590400
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0469_2006050100_123_975_12192113_1_r1500204407
The child process has been launched with process id: 488
Executing the command: /var/lib/boinc-client/slots/3/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:Biggles:1:1:488:488] [20230128:092018:1674897618:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:Biggles:1:1:488:488] [20230128:092018:1674897618:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:Biggles:1:1:488:488] [20230128:092018:1674897618:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:Biggles:1:1:488:488] [20230128:092018:1674897618:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:Biggles:1:1:488:488] [20230128:092018:1674897618:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:Biggles:1:1:488:488] [20230128:092018:1674897618:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:Biggles:1:1:488:488] [20230128:092018:1674897618:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:Biggles:1:1:488:488] [20230128:092018:1674897618:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:Biggles:1:1:488:488] [20230128:092018:1674897618:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:Biggles:1:1:488:488] [20230128:092018:1674897618:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:Biggles:1:1:488:488] [20230128:092018:1674897618:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:Biggles:1:1:488:488] [20230128:092018:1674897618:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:Biggles:1:1:488:488] [20230128:092018:1674897618:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:Biggles:1:1:488:488] [20230128:092018:1674897618:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:Biggles:1:1:488:488] [20230128:092018:1674897618:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:Biggles:1:1:488:488] [20230128:092018:1674897618:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Biggles:1:1:488:488] [20230128:092018:1674897618:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:Biggles:1:1:488:488] [20230128:092018:1674897618:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Biggles:1:1:488:488] [20230128:092018:1674897618:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:Biggles:1:1:488:488] [20230128:092018:1674897618:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:Biggles:1:1:488:488] [20230128:092018:1674897618:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Biggles:1:1:488:488] [20230128:092018:1674897618:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Biggles:1:1:488:488] [20230128:092018:1674897618:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:Biggles:1:1:488:488] [20230128:092018:1674897618:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Biggles:1:1:488:488] [20230128:092018:1674897618:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Biggles:1:1:488:488] [20230128:092018:1674897618:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Biggles:1:1:488:488] [20230128:092018:1674897618:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:Biggles:1:1:488:488] [20230128:092018:1674897618:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:Biggles:1:1:488:488] [20230128:092018:1674897618:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:Biggles:1:1:488:488] [20230128:092018:1674897618:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  09:20:42 STEP 2664 H=2664:00 +CPU= 20.859
  09:20:54 STEP 2665 H=2665:00 +CPU= 12.175
  09:21:07 STEP 2666 H=2666:00 +CPU= 13.199
  09:21:25 STEP 2667 H=2667:00 +CPU= 17.881
  09:21:40 STEP 2668 H=2668:00 +CPU= 14.780
  09:21:55 STEP 2669 H=2669:00 +CPU= 15.215
  09:22:17 STEP 2670 H=2670:00 +CPU= 21.729
  09:22:29 STEP 2671 H=2671:00 +CPU= 12.794
  09:22:43 STEP 2672 H=2672:00 +CPU= 13.069
  09:23:01 STEP 2673 H=2673:00 +CPU= 18.820
  09:23:14 STEP 2674 H=2674:00 +CPU= 12.926
  09:23:26 STEP 2675 H=2675:00 +CPU= 12.121
  09:23:48 STEP 2676 H=2676:00 +CPU= 21.187
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+002676
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+002676
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+002676
  09:24:00 STEP 2677 H=2677:00 +CPU= 12.294
  09:24:13 STEP 2678 H=2678:00 +CPU= 12.976
  09:24:32 STEP 2679 H=2679:00 +CPU= 19.434
  09:24:46 STEP 2680 H=2680:00 +CPU= 13.209
  09:24:58 STEP 2681 H=2681:00 +CPU= 12.790
  09:25:17 STEP 2682 H=2682:00 +CPU= 18.069
  09:25:29 STEP 2683 H=2683:00 +CPU= 12.801
  09:25:42 STEP 2684 H=2684:00 +CPU= 12.426
  09:26:00 STEP 2685 H=2685:00 +CPU= 18.473
  09:26:13 STEP 2686 H=2686:00 +CPU= 12.278
  09:26:25 STEP 2687 H=2687:00 +CPU= 12.815
  09:26:48 STEP 2688 H=2688:00 +CPU= 22.126
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+002688
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+002688
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+002688
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMGGhq0f+002664
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMSHhq0f+002664
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMUAhq0f+002664
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMGGhq0f+002676
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMSHhq0f+002676
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMUAhq0f+002676
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0469_2006050100_123_975_12192113_1_r1500204407_111.zip
Uploading the intermediate file: upload_file_111.zip
  09:27:00 STEP 2689 H=2689:00 +CPU= 12.222
  09:27:13 STEP 2690 H=2690:00 +CPU= 13.010
  09:27:32 STEP 2691 H=2691:00 +CPU= 18.761
  09:27:45 STEP 2692 H=2692:00 +CPU= 13.496
  09:27:58 STEP 2693 H=2693:00 +CPU= 12.390
  09:28:16 STEP 2694 H=2694:00 +CPU= 18.621
  09:28:28 STEP 2695 H=2695:00 +CPU= 12.033
  09:28:41 STEP 2696 H=2696:00 +CPU= 12.891
  09:29:01 STEP 2697 H=2697:00 +CPU= 19.424
  09:29:14 STEP 2698 H=2698:00 +CPU= 13.081
  09:29:26 STEP 2699 H=2699:00 +CPU= 12.243
  09:29:46 STEP 2700 H=2700:00 +CPU= 20.133
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+002700
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+002700
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+002700
  09:29:58 STEP 2701 H=2701:00 +CPU= 12.014
  09:30:11 STEP 2702 H=2702:00 +CPU= 12.649
  09:30:30 STEP 2703 H=2703:00 +CPU= 18.471
  09:30:42 STEP 2704 H=2704:00 +CPU= 12.460
  09:30:54 STEP 2705 H=2705:00 +CPU= 12.329
  09:31:13 STEP 2706 H=2706:00 +CPU= 18.769
  09:31:25 STEP 2707 H=2707:00 +CPU= 11.782
  09:31:37 STEP 2708 H=2708:00 +CPU= 12.430
  09:31:55 STEP 2709 H=2709:00 +CPU= 17.944
  09:32:08 STEP 2710 H=2710:00 +CPU= 12.779
  09:32:21 STEP 2711 H=2711:00 +CPU= 13.177
  09:32:44 STEP 2712 H=2712:00 +CPU= 21.930
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+002712
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+002712
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+002712
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMGGhq0f+002688
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMSHhq0f+002688
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMUAhq0f+002688
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMGGhq0f+002700
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMSHhq0f+002700
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMUAhq0f+002700
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0469_2006050100_123_975_12192113_1_r1500204407_112.zip
Uploading the intermediate file: upload_file_112.zip
  09:32:56 STEP 2713 H=2713:00 +CPU= 12.244
  09:33:08 STEP 2714 H=2714:00 +CPU= 12.657
  09:33:27 STEP 2715 H=2715:00 +CPU= 18.589
  09:33:39 STEP 2716 H=2716:00 +CPU= 12.363
  09:33:51 STEP 2717 H=2717:00 +CPU= 11.928
  09:34:10 STEP 2718 H=2718:00 +CPU= 18.356
  09:34:22 STEP 2719 H=2719:00 +CPU= 12.375
  09:34:34 STEP 2720 H=2720:00 +CPU= 12.288
  09:34:53 STEP 2721 H=2721:00 +CPU= 18.614
  09:35:06 STEP 2722 H=2722:00 +CPU= 13.201
  09:35:19 STEP 2723 H=2723:00 +CPU= 12.621
  09:35:39 STEP 2724 H=2724:00 +CPU= 20.221
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+002724
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+002724
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+002724
  09:35:51 STEP 2725 H=2725:00 +CPU= 11.857
  09:36:03 STEP 2726 H=2726:00 +CPU= 11.716
  09:36:22 STEP 2727 H=2727:00 +CPU= 18.905
  09:36:34 STEP 2728 H=2728:00 +CPU= 12.271
  09:36:46 STEP 2729 H=2729:00 +CPU= 12.019
  09:37:05 STEP 2730 H=2730:00 +CPU= 18.508
  09:37:16 STEP 2731 H=2731:00 +CPU= 11.759
  09:37:28 STEP 2732 H=2732:00 +CPU= 11.706
  09:37:46 STEP 2733 H=2733:00 +CPU= 17.591
  09:37:58 STEP 2734 H=2734:00 +CPU= 12.150
  09:38:10 STEP 2735 H=2735:00 +CPU= 11.845
  09:38:31 STEP 2736 H=2736:00 +CPU= 21.120
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+002736
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+002736
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+002736
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMGGhq0f+002712
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMSHhq0f+002712
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMUAhq0f+002712
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMGGhq0f+002724
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMSHhq0f+002724
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMUAhq0f+002724
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0469_2006050100_123_975_12192113_1_r1500204407_113.zip
Uploading the intermediate file: upload_file_113.zip
  09:38:43 STEP 2737 H=2737:00 +CPU= 11.577
  09:38:56 STEP 2738 H=2738:00 +CPU= 12.827
  09:39:15 STEP 2739 H=2739:00 +CPU= 18.728
  09:39:27 STEP 2740 H=2740:00 +CPU= 12.124
  09:39:39 STEP 2741 H=2741:00 +CPU= 12.204
  09:39:58 STEP 2742 H=2742:00 +CPU= 19.182
  09:40:11 STEP 2743 H=2743:00 +CPU= 12.584
  09:40:24 STEP 2744 H=2744:00 +CPU= 12.759
  09:40:42 STEP 2745 H=2745:00 +CPU= 17.866
  09:40:55 STEP 2746 H=2746:00 +CPU= 12.917
  09:41:08 STEP 2747 H=2747:00 +CPU= 12.847
  09:41:28 STEP 2748 H=2748:00 +CPU= 20.698
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+002748
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+002748
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+002748
  09:41:42 STEP 2749 H=2749:00 +CPU= 13.312
  09:41:55 STEP 2750 H=2750:00 +CPU= 13.181
  09:42:14 STEP 2751 H=2751:00 +CPU= 18.997
  09:42:27 STEP 2752 H=2752:00 +CPU= 12.713
  09:42:39 STEP 2753 H=2753:00 +CPU= 12.666
  09:42:59 STEP 2754 H=2754:00 +CPU= 20.086
  09:43:12 STEP 2755 H=2755:00 +CPU= 12.384
  09:43:25 STEP 2756 H=2756:00 +CPU= 13.074
  09:43:44 STEP 2757 H=2757:00 +CPU= 19.363
  09:43:57 STEP 2758 H=2758:00 +CPU= 12.419
  09:44:09 STEP 2759 H=2759:00 +CPU= 12.673
  09:44:32 STEP 2760 H=2760:00 +CPU= 22.314
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+002760
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+002760
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+002760
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMGGhq0f+002736
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMSHhq0f+002736
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMUAhq0f+002736
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMGGhq0f+002748
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMSHhq0f+002748
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMUAhq0f+002748
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0469_2006050100_123_975_12192113_1_r1500204407_114.zip
Uploading the intermediate file: upload_file_114.zip
  09:44:45 STEP 2761 H=2761:00 +CPU= 12.891
  09:44:59 STEP 2762 H=2762:00 +CPU= 13.954
  09:45:21 STEP 2763 H=2763:00 +CPU= 21.708
  09:45:34 STEP 2764 H=2764:00 +CPU= 13.573
  09:45:46 STEP 2765 H=2765:00 +CPU= 12.348
  09:46:06 STEP 2766 H=2766:00 +CPU= 19.982
  09:46:18 STEP 2767 H=2767:00 +CPU= 11.950
  09:46:31 STEP 2768 H=2768:00 +CPU= 12.063
  09:46:49 STEP 2769 H=2769:00 +CPU= 18.448
  09:47:01 STEP 2770 H=2770:00 +CPU= 12.005
  09:47:13 STEP 2771 H=2771:00 +CPU= 12.406
  09:47:34 STEP 2772 H=2772:00 +CPU= 20.773
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+002772
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+002772
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+002772
  09:47:50 STEP 2773 H=2773:00 +CPU= 13.551
  09:48:03 STEP 2774 H=2774:00 +CPU= 12.538
  09:48:22 STEP 2775 H=2775:00 +CPU= 19.004
  09:48:35 STEP 2776 H=2776:00 +CPU= 12.681
  09:48:47 STEP 2777 H=2777:00 +CPU= 11.652
  09:49:06 STEP 2778 H=2778:00 +CPU= 19.211
  09:49:18 STEP 2779 H=2779:00 +CPU= 11.617
  09:49:30 STEP 2780 H=2780:00 +CPU= 12.102
  09:49:48 STEP 2781 H=2781:00 +CPU= 18.317
  09:50:01 STEP 2782 H=2782:00 +CPU= 12.171
  09:50:13 STEP 2783 H=2783:00 +CPU= 12.730
  09:50:35 STEP 2784 H=2784:00 +CPU= 21.737
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+002784
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+002784
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+002784
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMGGhq0f+002760
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMSHhq0f+002760
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMUAhq0f+002760
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMGGhq0f+002772
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMSHhq0f+002772
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMUAhq0f+002772
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0469_2006050100_123_975_12192113_1_r1500204407_115.zip
Uploading the intermediate file: upload_file_115.zip
  09:50:47 STEP 2785 H=2785:00 +CPU= 12.065
  09:50:59 STEP 2786 H=2786:00 +CPU= 11.934
  09:51:18 STEP 2787 H=2787:00 +CPU= 18.229
  09:51:30 STEP 2788 H=2788:00 +CPU= 12.460
  09:51:42 STEP 2789 H=2789:00 +CPU= 11.971
  09:52:01 STEP 2790 H=2790:00 +CPU= 18.603
  09:52:12 STEP 2791 H=2791:00 +CPU= 11.773
  09:52:24 STEP 2792 H=2792:00 +CPU= 11.946
  09:52:42 STEP 2793 H=2793:00 +CPU= 17.884
  09:52:54 STEP 2794 H=2794:00 +CPU= 12.212
  09:53:06 STEP 2795 H=2795:00 +CPU= 11.413
  09:53:27 STEP 2796 H=2796:00 +CPU= 20.607
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+002796
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+002796
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+002796
  09:53:38 STEP 2797 H=2797:00 +CPU= 11.841
  09:53:51 STEP 2798 H=2798:00 +CPU= 12.308
  09:54:08 STEP 2799 H=2799:00 +CPU= 17.626
  09:54:21 STEP 2800 H=2800:00 +CPU= 12.241
  09:54:32 STEP 2801 H=2801:00 +CPU= 11.584
  09:54:51 STEP 2802 H=2802:00 +CPU= 18.611
  09:55:03 STEP 2803 H=2803:00 +CPU= 12.239
  09:55:15 STEP 2804 H=2804:00 +CPU= 12.055
  09:55:33 STEP 2805 H=2805:00 +CPU= 18.147
  09:55:45 STEP 2806 H=2806:00 +CPU= 12.002
  09:55:58 STEP 2807 H=2807:00 +CPU= 12.220
  09:56:19 STEP 2808 H=2808:00 +CPU= 21.485
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+002808
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+002808
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+002808
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMGGhq0f+002784
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMSHhq0f+002784
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMUAhq0f+002784
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMGGhq0f+002796
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMSHhq0f+002796
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMUAhq0f+002796
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0469_2006050100_123_975_12192113_1_r1500204407_116.zip
Uploading the intermediate file: upload_file_116.zip
  09:56:31 STEP 2809 H=2809:00 +CPU= 11.751
  09:56:43 STEP 2810 H=2810:00 +CPU= 12.323
  09:57:02 STEP 2811 H=2811:00 +CPU= 18.328
  09:57:15 STEP 2812 H=2812:00 +CPU= 13.273
  09:57:27 STEP 2813 H=2813:00 +CPU= 11.848
  09:57:46 STEP 2814 H=2814:00 +CPU= 18.582
  09:57:59 STEP 2815 H=2815:00 +CPU= 13.066
  09:58:11 STEP 2816 H=2816:00 +CPU= 12.531
  09:58:29 STEP 2817 H=2817:00 +CPU= 17.930
  09:58:43 STEP 2818 H=2818:00 +CPU= 13.260
  09:58:54 STEP 2819 H=2819:00 +CPU= 11.694
  09:59:14 STEP 2820 H=2820:00 +CPU= 20.074
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+002820
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+002820
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+002820
  09:59:26 STEP 2821 H=2821:00 +CPU= 12.087
  09:59:39 STEP 2822 H=2822:00 +CPU= 12.294
  09:59:57 STEP 2823 H=2823:00 +CPU= 18.031
  10:00:10 STEP 2824 H=2824:00 +CPU= 12.706
  10:00:22 STEP 2825 H=2825:00 +CPU= 12.412
  10:00:41 STEP 2826 H=2826:00 +CPU= 18.686
  10:00:53 STEP 2827 H=2827:00 +CPU= 12.160
  10:01:06 STEP 2828 H=2828:00 +CPU= 12.553
  10:01:24 STEP 2829 H=2829:00 +CPU= 18.409
  10:01:38 STEP 2830 H=2830:00 +CPU= 13.346
  10:01:50 STEP 2831 H=2831:00 +CPU= 12.570
  10:02:13 STEP 2832 H=2832:00 +CPU= 21.872
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+002832
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+002832
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+002832
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMGGhq0f+002808
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMSHhq0f+002808
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMUAhq0f+002808
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMGGhq0f+002820
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMSHhq0f+002820
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMUAhq0f+002820
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0469_2006050100_123_975_12192113_1_r1500204407_117.zip
Uploading the intermediate file: upload_file_117.zip
  10:02:25 STEP 2833 H=2833:00 +CPU= 11.957
  10:02:38 STEP 2834 H=2834:00 +CPU= 13.506
  10:02:57 STEP 2835 H=2835:00 +CPU= 19.112
  10:03:10 STEP 2836 H=2836:00 +CPU= 12.931
  10:03:23 STEP 2837 H=2837:00 +CPU= 12.530
  10:03:42 STEP 2838 H=2838:00 +CPU= 19.022
  10:03:55 STEP 2839 H=2839:00 +CPU= 13.047
  10:04:07 STEP 2840 H=2840:00 +CPU= 12.332
  10:04:26 STEP 2841 H=2841:00 +CPU= 18.259
  10:04:38 STEP 2842 H=2842:00 +CPU= 12.743
  10:04:50 STEP 2843 H=2843:00 +CPU= 11.925
  10:05:13 STEP 2844 H=2844:00 +CPU= 21.568
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+002844
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+002844
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+002844
  10:05:25 STEP 2845 H=2845:00 +CPU= 11.934
  10:05:38 STEP 2846 H=2846:00 +CPU= 12.820
  10:05:57 STEP 2847 H=2847:00 +CPU= 18.363
  10:06:10 STEP 2848 H=2848:00 +CPU= 13.285
  10:06:23 STEP 2849 H=2849:00 +CPU= 12.728
  10:06:41 STEP 2850 H=2850:00 +CPU= 18.478
  10:06:54 STEP 2851 H=2851:00 +CPU= 12.410
  10:07:06 STEP 2852 H=2852:00 +CPU= 11.943
  10:07:25 STEP 2853 H=2853:00 +CPU= 19.428
  10:07:37 STEP 2854 H=2854:00 +CPU= 12.057
  10:07:50 STEP 2855 H=2855:00 +CPU= 12.895
  10:08:12 STEP 2856 H=2856:00 +CPU= 22.029
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+002856
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+002856
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+002856
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMGGhq0f+002832
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMSHhq0f+002832
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMUAhq0f+002832
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMGGhq0f+002844
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMSHhq0f+002844
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMUAhq0f+002844
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0469_2006050100_123_975_12192113_1_r1500204407_118.zip
Uploading the intermediate file: upload_file_118.zip
  10:08:27 STEP 2857 H=2857:00 +CPU= 13.943
  10:08:41 STEP 2858 H=2858:00 +CPU= 14.082
  10:09:02 STEP 2859 H=2859:00 +CPU= 20.665
  10:09:16 STEP 2860 H=2860:00 +CPU= 13.927
  10:09:30 STEP 2861 H=2861:00 +CPU= 14.273
  10:09:51 STEP 2862 H=2862:00 +CPU= 20.329
  10:10:04 STEP 2863 H=2863:00 +CPU= 13.428
  10:10:18 STEP 2864 H=2864:00 +CPU= 13.873
  10:10:38 STEP 2865 H=2865:00 +CPU= 20.139
  10:10:53 STEP 2866 H=2866:00 +CPU= 14.894
  10:11:07 STEP 2867 H=2867:00 +CPU= 12.962
  10:11:30 STEP 2868 H=2868:00 +CPU= 23.724
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+002868
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+002868
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+002868
  10:11:44 STEP 2869 H=2869:00 +CPU= 13.548
  10:11:58 STEP 2870 H=2870:00 +CPU= 14.106
  10:12:19 STEP 2871 H=2871:00 +CPU= 20.167
  10:12:32 STEP 2872 H=2872:00 +CPU= 13.835
  10:12:46 STEP 2873 H=2873:00 +CPU= 13.488
  10:13:07 STEP 2874 H=2874:00 +CPU= 20.189
  10:13:21 STEP 2875 H=2875:00 +CPU= 13.717
  10:13:35 STEP 2876 H=2876:00 +CPU= 14.177
  10:13:56 STEP 2877 H=2877:00 +CPU= 20.175
  10:14:09 STEP 2878 H=2878:00 +CPU= 13.647
  10:14:23 STEP 2879 H=2879:00 +CPU= 13.962
  10:14:47 STEP 2880 H=2880:00 +CPU= 23.525
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+002880
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+002880
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+002880
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMGGhq0f+002856
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMSHhq0f+002856
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMUAhq0f+002856
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMGGhq0f+002868
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMSHhq0f+002868
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMUAhq0f+002868
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0469_2006050100_123_975_12192113_1_r1500204407_119.zip
Uploading the intermediate file: upload_file_119.zip
  10:15:01 STEP 2881 H=2881:00 +CPU= 13.286
  10:15:14 STEP 2882 H=2882:00 +CPU= 13.681
  10:15:38 STEP 2883 H=2883:00 +CPU= 22.577
  10:15:52 STEP 2884 H=2884:00 +CPU= 14.215
  10:16:06 STEP 2885 H=2885:00 +CPU= 14.043
  10:16:27 STEP 2886 H=2886:00 +CPU= 20.883
  10:16:43 STEP 2887 H=2887:00 +CPU= 15.425
  10:16:57 STEP 2888 H=2888:00 +CPU= 13.658
  10:17:18 STEP 2889 H=2889:00 +CPU= 20.665
  10:17:32 STEP 2890 H=2890:00 +CPU= 13.666
  10:17:46 STEP 2891 H=2891:00 +CPU= 14.274
  10:18:09 STEP 2892 H=2892:00 +CPU= 22.679
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+002892
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+002892
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+002892
  10:18:23 STEP 2893 H=2893:00 +CPU= 13.679
  10:18:37 STEP 2894 H=2894:00 +CPU= 13.660
  10:18:58 STEP 2895 H=2895:00 +CPU= 20.622
  10:19:12 STEP 2896 H=2896:00 +CPU= 13.644
  10:19:26 STEP 2897 H=2897:00 +CPU= 13.798
  10:19:48 STEP 2898 H=2898:00 +CPU= 21.376
  10:20:03 STEP 2899 H=2899:00 +CPU= 14.991
  10:20:18 STEP 2900 H=2900:00 +CPU= 14.523
  10:20:40 STEP 2901 H=2901:00 +CPU= 21.014
  10:20:54 STEP 2902 H=2902:00 +CPU= 14.327
  10:21:09 STEP 2903 H=2903:00 +CPU= 14.214
  10:21:34 STEP 2904 H=2904:00 +CPU= 25.205
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+002904
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+002904
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+002904
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMGGhq0f+002880
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMSHhq0f+002880
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMUAhq0f+002880
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMGGhq0f+002892
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMSHhq0f+002892
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMUAhq0f+002892
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0469_2006050100_123_975_12192113_1_r1500204407_120.zip
Uploading the intermediate file: upload_file_120.zip
  10:21:49 STEP 2905 H=2905:00 +CPU= 13.904
  10:22:03 STEP 2906 H=2906:00 +CPU= 14.117
Uploading trickle at timestep: 10454400
  10:22:24 STEP 2907 H=2907:00 +CPU= 20.756
  10:22:38 STEP 2908 H=2908:00 +CPU= 13.876
  10:22:51 STEP 2909 H=2909:00 +CPU= 13.184
  10:23:13 STEP 2910 H=2910:00 +CPU= 20.787
  10:23:26 STEP 2911 H=2911:00 +CPU= 13.413
  10:23:41 STEP 2912 H=2912:00 +CPU= 13.907
  10:24:01 STEP 2913 H=2913:00 +CPU= 20.424
  10:24:14 STEP 2914 H=2914:00 +CPU= 12.385
  10:24:26 STEP 2915 H=2915:00 +CPU= 12.118
  10:24:46 STEP 2916 H=2916:00 +CPU= 20.007
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+002916
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+002916
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+002916
  10:24:57 STEP 2917 H=2917:00 +CPU= 11.629
  10:25:10 STEP 2918 H=2918:00 +CPU= 12.160
  10:25:27 STEP 2919 H=2919:00 +CPU= 17.735
  10:25:40 STEP 2920 H=2920:00 +CPU= 12.370
  10:25:52 STEP 2921 H=2921:00 +CPU= 12.375
  10:26:12 STEP 2922 H=2922:00 +CPU= 19.406
  10:26:25 STEP 2923 H=2923:00 +CPU= 13.276
  10:26:39 STEP 2924 H=2924:00 +CPU= 13.634
  10:26:57 STEP 2925 H=2925:00 +CPU= 18.379
  10:27:09 STEP 2926 H=2926:00 +CPU= 12.466
  10:27:22 STEP 2927 H=2927:00 +CPU= 12.725
  10:27:46 STEP 2928 H=2928:00 +CPU= 23.600
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+002928
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+002928
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+002928
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMGGhq0f+002904
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMSHhq0f+002904
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMUAhq0f+002904
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMGGhq0f+002916
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMSHhq0f+002916
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192113/ICMUAhq0f+002916
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0469_2006050100_123_975_12192113_1_r1500204407_121.zip
Uploading the intermediate file: upload_file_121.zip
  10:27:58 STEP 2929 H=2929:00 +CPU= 11.845
  10:28:11 STEP 2930 H=2930:00 +CPU= 13.268
  10:28:30 STEP 2931 H=2931:00 +CPU= 19.193
  10:28:43 STEP 2932 H=2932:00 +CPU= 12.687
  10:28:55 STEP 2933 H=2933:00 +CPU= 12.212
  10:29:15 STEP 2934 H=2934:00 +CPU= 19.328
  10:29:28 STEP 2935 H=2935:00 +CPU= 12.990
  10:29:41 STEP 2936 H=2936:00 +CPU= 12.896
  10:29:59 STEP 2937 H=2937:00 +CPU= 18.592
  10:30:12 STEP 2938 H=2938:00 +CPU= 12.678
  10:30:24 STEP 2939 H=2939:00 +CPU= 12.462
  10:30:46 STEP 2940 H=2940:00 +CPU= 21.893
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+002940
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+002940
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+002940
  10:30:59 STEP 2941 H=2941:00 +CPU= 12.420
  10:31:11 STEP 2942 H=2942:00 +CPU= 12.556
  10:31:30 STEP 2943 H=2943:00 +CPU= 19.078
  10:31:43 STEP 2944 H=2944:00 +CPU= 12.426
  10:31:56 STEP 2945 H=2945:00 +CPU= 12.787
  10:32:15 STEP 2946 H=2946:00 +CPU= 19.616
  10:32:27 STEP 2947 H=2947:00 +CPU= 11.944
  10:32:40 STEP 2948 H=2948:00 +CPU= 12.877
  10:32:59 STEP 2949 H=2949:00 +CPU= 18.586
  10:33:12 STEP 2950 H=2950:00 +CPU= 13.171
  10:33:25 STEP 2951 H=2951:00 +CPU= 12.648
  10:33:46 STEP 2952 H=2952:00 +CPU= 21.388
..The child process terminated with status: 0
>>> Printing last 70 lines from file: NODE.001_01


   !!!!!!!!!!!!!! WAVE FIELDS INTEGRATED.  DATE IS: 20060831234500  !!!!!!!!!!!! 

   !!!!!!!!!!!!!! WAVE FIELDS INTEGRATED.  DATE IS: 20060901000000  !!!!!!!!!!!! 
  
  GLOBAL NORM OF FIELDS RETURNED TO IFS :
  Charnock     1.330800774938021E-002  7.500100765173560E-003
  5.451423235854007E-002       26160
              HEX: 3F8B413A9077D030  3F7EB86CF80CCCD1  3FABE94A1A4AFEB6
  U-Stokes     3.492503543917805E-003 -0.246323661531350     
  0.311960485219513            26160
              HEX: 3F6C9C4F9012C4B9  BFCF8788A3410A22  3FD3F7291C6A5FE0
  V-Stokes     4.618530311143391E-003 -0.345126141474107     
  0.268330732481124            26160
              HEX: 3F72EAE14A4568B8  BFD6168BF4A80FBE  3FD12C54AA212969
  
  
 END OF WAVE MODEL CALL 
 ----------------------------------------------------------------------- 
 10:33:46 STEP 2952 H=2952:00 +CPU= 21.388
 END CNT3
 NSTEP =  2952 CNT0   000000000  18.488  18.488  18.510   0.999               0               0               0               0               0
IO-STREAM STATISTICS, TOTAL NO OF REC/BYTES READ           35/   804992476 WRITTEN         3972/************
 -TASK-OPENED-OPEN-RECS IN -KBYTE IN    -RECS OUT-KBYTE OUT-WALL   -WALL IN-WALL
  OU-TOT IN -TOT OUT
   1    159    0       35       786125     3972 ************    11.7     1.0    10.7     1.4    16.4
===-=== START OF TIMING STATISTICS ===-===
 
STATS FOR ALL TASKS
 NUM ROUTINE                                     CALLS  MEAN(ms)   MAX(ms)   FRAC(%)  UNBAL(%)
   0 CNT0     - COMPLETE EXECUTION                   1 4407890.6 4407890.6    100.00      0.00
   1 CNT4     - FORWARD INTEGRATION                  1 4405807.2 4405807.2     99.95      0.00
   8 SCAN2M - GRID-POINT DYNAMICS                  313    1274.7    1274.7      9.05      0.00
   9 SPCM     - SPECTRAL COMP.                     288     228.2     228.2      1.49      0.00
  10 SCAN2M - PHYSICS                              289    6507.0    6507.0     42.66      0.00
  11 IOPACK   - OUTPUT P.P. RESULTS                 24     635.6     635.6      0.35      0.00
  12 SPNORM   - SPECTRAL NORM COMP.                289      11.6      11.6      0.08      0.00
  13 SCAN2M - RADIATION CALC.                       97    6526.2    6526.2     14.36      0.00
  17 GRIDFPOS IN CNT4                               24       9.1       9.1      0.00      0.00
  27 RESTART FILES - WRITING                        12    1271.1    1271.1      0.35      0.00
  28 RESTART FILES - READING                         1    1882.9    1882.9      0.04      0.00
  29 SU4FPOS IN CNT4                                24       0.0       0.0      0.00      0.00
  30 DYNFPOS IN CNT4                                24    1558.5    1558.5      0.85      0.00
  31 POSDDH IN STEPO                                 1       0.6       0.6      0.00      0.00
  37 CPGLAG   - SL COMPUTATIONS                    289    2083.6    2083.6     13.66      0.00
  38 WAM      - TOTAL COST OF WAVE MODEL           289    1593.9    1593.9     10.45      0.00
  39 SU0YOMB                                         1     141.7     141.7      0.00      0.00
  51 SCAN2M   - SL COMM. PART 1                    289      17.9      17.9      0.12      0.00
  54 SPCM     - M TO S/S TO M TRANSP.              288     204.2     204.2      1.33      0.00
  55 SPCIMPF  - S TO M/M TO S TRANSP.              288      33.5      33.5      0.22      0.00
  56 SPNORM   - SPECTRAL NORM COMM.                289       0.1       0.1      0.00      0.00
 102 LTINV_CTL   - INVERSE LEGENDRE TRANSFORM      990      80.1      80.1      1.80      0.00
 103 LTDIR_CTL   - DIRECT LEGENDRE TRANSFORM       624      75.7      75.7      1.07      0.00
 106 FTDIR_CTL   - DIRECT FOURIER TRANSFORM        624      28.9      28.9      0.41      0.00
 107 FTINV_CTL   - INVERSE FOURIER TRANSFORM       990      32.4      32.4      0.73      0.00
 140 SULEG       - COMP. OF LEGENDRE POL.            2       9.3       9.3      0.00      0.00
 152 LTINV_CTL   - M TO L TRANSPOSITION            990      10.7      10.7      0.24      0.00
 153 LTDIR_CTL   - L TO M TRANSPOSITION            624       9.9       9.9      0.14      0.00
 157 FTINV_CTL   - L TO G TRANSPOSITION            990      14.3      14.3      0.32      0.00
 158 FTDIR_CTL   - G TO L TRANSPOSITION            624       9.3       9.3      0.13      0.00
 400 GSTATS                                      69509       0.0       0.0      0.00      0.00
 401 GSTATS HOOK                                 66698       0.0       0.0      0.00      0.00
TOTAL MEASURED IMBALANCE =       0.0 SECONDS,  0.0 PERCENT
TOTAL WALLCLOCK TIME     4407.9 CPU TIME    4380.7 VECTOR TIME     4380.7


===-=== END   OF TIMING STATISTICS ===-===
FORECAST DAYS PER DAY     2412.1
  *** END CNT0 *** 
------------------------------------------------
>>> Printing last 11 lines from file: rcf

 CSTEP   = "    2952",
 CTIME   = "01230000      ",
 NSTEPLPP        =         201,         202,          49,      228226,      228227, 5*0,
 IPRGPNSRES      =           1,
 IPRGPEWRES      =           1,
 IPRTRWRES       =           1,
 IPRTRVRES       =           1,
 GMASS0  =   98329.3682850359     ,
 GMASSI  =   98329.2664373923     
 /
------------------------------------------------
>>> Printing last 17 lines from file: waminfo

PARAM=10U/10V,                                                          
REPRES=GG,                                                              
ANALYSIS PERIOD =       0                                               
FORECAST PERIOD =    9590400                                            
WIND TIME STEP IN SECONDS =    3600                                     
ANALYSIS FROM 20060501000000 TO 20060501000000                          
ASSIMILATION NO                                                         
NEW FORECAST NO                                                         
STATUS STORM1= UNFINISHED                                               
STATUS STORM2= UNFINISHED                                               
STATUS STORM3= UNFINISHED                                               
DATE FOR OUTPUT OF BOTH RESTART FILES = 00000000000000                  
LAST DATE FOR SPECTRA FILE OUTPUT = 20060901000000                      
BEGIN DATE FOR USING SURFACE CURRENT = 20060501000000                   
PARALLEL RESTART = YES                                                  
NUMBER OF MPI TASKS USED =          1                                   
------------------------------------------------
>>> Printing last 8 lines from file: /var/lib/boinc-client/slots/3/progress_file_12192113.xml

<running_values>
  <last_cpu_time>45042.260000</last_cpu_time>
  <upload_file_number>122</upload_file_number>
  <last_iter>2952</last_iter>
  <last_upload>10540800</last_upload>
  <model_completed>0</model_completed>
</running_values>
------------------------------------------------
..Failed, model did not complete successfully

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 May 2023 16:44:10 1536027 22295225 oifs_43r3_ps_0469_2006050100_123_975_12192113_1 10,454,400 44,322 0.0042
09 May 2023 16:44:08 1536027 22295225 oifs_43r3_ps_0469_2006050100_123_975_12192113_1 9,158,400 38,694 0.0042
09 May 2023 16:43:39 1536027 22295225 oifs_43r3_ps_0469_2006050100_123_975_12192113_1 8,294,400 34,908 0.0042
09 May 2023 16:43:36 1536027 22295225 oifs_43r3_ps_0469_2006050100_123_975_12192113_1 7,430,400 31,169 0.0042
09 May 2023 16:43:33 1536027 22295225 oifs_43r3_ps_0469_2006050100_123_975_12192113_1 6,566,400 27,317 0.0042
09 May 2023 16:43:31 1536027 22295225 oifs_43r3_ps_0469_2006050100_123_975_12192113_1 5,702,400 23,640 0.0041
09 May 2023 16:43:28 1536027 22295225 oifs_43r3_ps_0469_2006050100_123_975_12192113_1 4,838,400 19,907 0.0041
09 May 2023 16:43:25 1536027 22295225 oifs_43r3_ps_0469_2006050100_123_975_12192113_1 3,974,400 16,185 0.0041
09 May 2023 16:42:30 1536027 22295225 oifs_43r3_ps_0469_2006050100_123_975_12192113_1 2,592,000 10,101 0.0039
09 May 2023 16:42:26 1536027 22295225 oifs_43r3_ps_0469_2006050100_123_975_12192113_1 1,728,000 6,723 0.0039
09 May 2023 16:42:23 1536027 22295225 oifs_43r3_ps_0469_2006050100_123_975_12192113_1 864,000 3,355 0.0039


©2024 climateprediction.net