climateprediction.net home page
Task 22247691

Task 22247691

Name oifs_43r3_ps_2578_2021050100_123_947_12165667_0
Workunit 12165667
Created 29 Nov 2022, 14:19:35 UTC
Sent 29 Nov 2022, 14:30:21 UTC
Report deadline 29 Dec 2022, 14:30:21 UTC
Received 30 Nov 2022, 21:17:46 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 0 (0x00000000)
Computer ID 1521562
Run time 22 hours 55 min 50 sec
CPU time 54 min 16 sec
Validate state Invalid
Credit 14,515.20
Device peak FLOPS 5.97 GFLOPS
Application version OpenIFS 43r3 Perturbed Surface v1.01
x86_64-pc-linux-gnu
Peak working set size 4,418.31 MB
Peak swap size 4,978.08 MB
Peak disk usage 2,066.56 MB
Stderr
<core_client_version>7.16.6</core_client_version>
<![CDATA[
<stderr_txt>
he zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMGGhpi1+002400
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMSHhpi1+002400
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMUAhpi1+002400
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMGGhpi1+002412
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMSHhpi1+002412
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMUAhpi1+002412
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_2578_2021050100_123_947_12165667_0_r335842750_100.zip
Uploading the intermediate file: upload_file_100.zip
Uploading trickle at timestep: 8726400
  02:15:48 STEP 2425 H=2425:00 +CPU= 29.390
  02:16:18 STEP 2426 H=2426:00 +CPU= 30.192
  02:17:02 STEP 2427 H=2427:00 +CPU= 42.970
  02:17:32 STEP 2428 H=2428:00 +CPU= 30.057
  02:18:01 STEP 2429 H=2429:00 +CPU= 29.603
  02:18:45 STEP 2430 H=2430:00 +CPU= 43.987
  02:19:15 STEP 2431 H=2431:00 +CPU= 29.008
  02:19:45 STEP 2432 H=2432:00 +CPU= 30.366
  02:20:28 STEP 2433 H=2433:00 +CPU= 43.165
  02:20:58 STEP 2434 H=2434:00 +CPU= 29.863
  02:21:27 STEP 2435 H=2435:00 +CPU= 29.030
  02:22:16 STEP 2436 H=2436:00 +CPU= 48.653
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhpi1+002436
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhpi1+002436
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhpi1+002436
  02:22:45 STEP 2437 H=2437:00 +CPU= 29.442
  02:23:16 STEP 2438 H=2438:00 +CPU= 30.297
  02:23:59 STEP 2439 H=2439:00 +CPU= 43.021
  02:24:29 STEP 2440 H=2440:00 +CPU= 30.143
  02:24:57 STEP 2441 H=2441:00 +CPU= 27.690
  02:25:42 STEP 2442 H=2442:00 +CPU= 45.072
  02:26:12 STEP 2443 H=2443:00 +CPU= 29.244
  02:26:42 STEP 2444 H=2444:00 +CPU= 29.538
  02:27:26 STEP 2445 H=2445:00 +CPU= 44.177
  02:27:57 STEP 2446 H=2446:00 +CPU= 29.764
  02:28:27 STEP 2447 H=2447:00 +CPU= 30.084
  02:29:19 STEP 2448 H=2448:00 +CPU= 52.434
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhpi1+002448
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhpi1+002448
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhpi1+002448
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMGGhpi1+002424
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMSHhpi1+002424
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMUAhpi1+002424
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMGGhpi1+002436
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMSHhpi1+002436
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMUAhpi1+002436
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_2578_2021050100_123_947_12165667_0_r335842750_101.zip
Uploading the intermediate file: upload_file_101.zip
  02:29:48 STEP 2449 H=2449:00 +CPU= 28.609
Uploading trickle at timestep: 8812800
  02:30:18 STEP 2450 H=2450:00 +CPU= 30.147
  02:31:02 STEP 2451 H=2451:00 +CPU= 44.012
  02:31:32 STEP 2452 H=2452:00 +CPU= 29.579
  02:32:02 STEP 2453 H=2453:00 +CPU= 29.643
  02:32:47 STEP 2454 H=2454:00 +CPU= 44.766
  02:33:15 STEP 2455 H=2455:00 +CPU= 28.574
  02:33:45 STEP 2456 H=2456:00 +CPU= 30.081
  02:34:29 STEP 2457 H=2457:00 +CPU= 43.624
  02:34:59 STEP 2458 H=2458:00 +CPU= 29.949
  02:35:29 STEP 2459 H=2459:00 +CPU= 29.480
  02:36:18 STEP 2460 H=2460:00 +CPU= 48.671
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhpi1+002460
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhpi1+002460
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhpi1+002460
  02:36:47 STEP 2461 H=2461:00 +CPU= 29.230
  02:37:18 STEP 2462 H=2462:00 +CPU= 30.417
  02:38:02 STEP 2463 H=2463:00 +CPU= 44.164
  02:38:32 STEP 2464 H=2464:00 +CPU= 29.593
  02:39:01 STEP 2465 H=2465:00 +CPU= 28.731
  02:39:45 STEP 2466 H=2466:00 +CPU= 44.011
  02:40:14 STEP 2467 H=2467:00 +CPU= 28.741
  02:40:43 STEP 2468 H=2468:00 +CPU= 29.437
  02:41:27 STEP 2469 H=2469:00 +CPU= 43.920
  02:41:57 STEP 2470 H=2470:00 +CPU= 29.757
  02:42:28 STEP 2471 H=2471:00 +CPU= 30.010
  02:43:20 STEP 2472 H=2472:00 +CPU= 51.791
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhpi1+002472
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhpi1+002472
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhpi1+002472
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMGGhpi1+002448
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMSHhpi1+002448
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMUAhpi1+002448
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMGGhpi1+002460
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMSHhpi1+002460
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMUAhpi1+002460
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_2578_2021050100_123_947_12165667_0_r335842750_102.zip
Uploading the intermediate file: upload_file_102.zip
Uploading trickle at timestep: 8899200
  02:43:49 STEP 2473 H=2473:00 +CPU= 29.178
  02:44:19 STEP 2474 H=2474:00 +CPU= 30.419
  02:45:04 STEP 2475 H=2475:00 +CPU= 44.496
  02:45:34 STEP 2476 H=2476:00 +CPU= 30.100
  02:46:04 STEP 2477 H=2477:00 +CPU= 29.407
  02:46:49 STEP 2478 H=2478:00 +CPU= 44.655
  02:47:18 STEP 2479 H=2479:00 +CPU= 29.185
  02:47:48 STEP 2480 H=2480:00 +CPU= 30.542
  02:48:32 STEP 2481 H=2481:00 +CPU= 43.902
  02:49:03 STEP 2482 H=2482:00 +CPU= 30.280
  02:49:32 STEP 2483 H=2483:00 +CPU= 29.221
  02:50:21 STEP 2484 H=2484:00 +CPU= 49.061
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhpi1+002484
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhpi1+002484
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhpi1+002484
  02:50:51 STEP 2485 H=2485:00 +CPU= 29.283
  02:51:20 STEP 2486 H=2486:00 +CPU= 29.883
  02:52:05 STEP 2487 H=2487:00 +CPU= 44.352
  02:52:35 STEP 2488 H=2488:00 +CPU= 29.765
  02:53:05 STEP 2489 H=2489:00 +CPU= 29.197
  02:53:50 STEP 2490 H=2490:00 +CPU= 45.432
  02:54:19 STEP 2491 H=2491:00 +CPU= 28.600
  02:54:49 STEP 2492 H=2492:00 +CPU= 30.219
  02:55:33 STEP 2493 H=2493:00 +CPU= 43.648
  02:56:03 STEP 2494 H=2494:00 +CPU= 29.298
  02:56:33 STEP 2495 H=2495:00 +CPU= 30.052
  02:57:25 STEP 2496 H=2496:00 +CPU= 51.941
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhpi1+002496
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhpi1+002496
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhpi1+002496
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMGGhpi1+002472
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMSHhpi1+002472
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMUAhpi1+002472
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMGGhpi1+002484
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMSHhpi1+002484
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMUAhpi1+002484
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_2578_2021050100_123_947_12165667_0_r335842750_103.zip
Uploading the intermediate file: upload_file_103.zip
  02:57:54 STEP 2497 H=2497:00 +CPU= 28.447
Uploading trickle at timestep: 8985600
  02:58:25 STEP 2498 H=2498:00 +CPU= 30.762
  02:59:08 STEP 2499 H=2499:00 +CPU= 43.787
  02:59:38 STEP 2500 H=2500:00 +CPU= 29.109
  03:00:06 STEP 2501 H=2501:00 +CPU= 28.791
  03:00:52 STEP 2502 H=2502:00 +CPU= 45.316
  03:01:21 STEP 2503 H=2503:00 +CPU= 28.557
  03:01:51 STEP 2504 H=2504:00 +CPU= 30.130
  03:02:35 STEP 2505 H=2505:00 +CPU= 43.714
  03:03:04 STEP 2506 H=2506:00 +CPU= 29.809
  03:03:33 STEP 2507 H=2507:00 +CPU= 28.817
  03:04:23 STEP 2508 H=2508:00 +CPU= 49.438
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhpi1+002508
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhpi1+002508
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhpi1+002508
  03:04:51 STEP 2509 H=2509:00 +CPU= 28.365
  03:05:22 STEP 2510 H=2510:00 +CPU= 30.431
  03:06:06 STEP 2511 H=2511:00 +CPU= 44.058
  03:06:37 STEP 2512 H=2512:00 +CPU= 30.086
  03:07:06 STEP 2513 H=2513:00 +CPU= 29.513
  03:07:51 STEP 2514 H=2514:00 +CPU= 45.091
  03:08:21 STEP 2515 H=2515:00 +CPU= 29.295
  03:08:51 STEP 2516 H=2516:00 +CPU= 29.717
  03:09:34 STEP 2517 H=2517:00 +CPU= 43.357
  03:10:05 STEP 2518 H=2518:00 +CPU= 30.191
  03:10:35 STEP 2519 H=2519:00 +CPU= 30.414
  03:11:28 STEP 2520 H=2520:00 +CPU= 52.354
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhpi1+002520
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhpi1+002520
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhpi1+002520
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMGGhpi1+002496
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMSHhpi1+002496
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMUAhpi1+002496
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMGGhpi1+002508
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMSHhpi1+002508
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMUAhpi1+002508
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_2578_2021050100_123_947_12165667_0_r335842750_104.zip
Uploading the intermediate file: upload_file_104.zip
Uploading trickle at timestep: 9072000
  03:11:58 STEP 2521 H=2521:00 +CPU= 29.613
  03:12:28 STEP 2522 H=2522:00 +CPU= 29.810
  03:13:12 STEP 2523 H=2523:00 +CPU= 43.873
  03:13:41 STEP 2524 H=2524:00 +CPU= 29.523
  03:14:10 STEP 2525 H=2525:00 +CPU= 29.074
  03:14:55 STEP 2526 H=2526:00 +CPU= 44.936
  03:15:24 STEP 2527 H=2527:00 +CPU= 28.564
  03:15:54 STEP 2528 H=2528:00 +CPU= 29.851
  03:16:37 STEP 2529 H=2529:00 +CPU= 42.873
  03:17:07 STEP 2530 H=2530:00 +CPU= 30.228
  03:17:37 STEP 2531 H=2531:00 +CPU= 29.596
  03:18:26 STEP 2532 H=2532:00 +CPU= 49.416
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhpi1+002532
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhpi1+002532
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhpi1+002532
  03:18:55 STEP 2533 H=2533:00 +CPU= 28.312
  03:19:24 STEP 2534 H=2534:00 +CPU= 29.470
  03:20:08 STEP 2535 H=2535:00 +CPU= 43.521
  03:20:38 STEP 2536 H=2536:00 +CPU= 29.936
  03:21:08 STEP 2537 H=2537:00 +CPU= 29.458
  03:21:53 STEP 2538 H=2538:00 +CPU= 44.585
  03:22:22 STEP 2539 H=2539:00 +CPU= 29.160
  03:22:53 STEP 2540 H=2540:00 +CPU= 30.830
  03:23:37 STEP 2541 H=2541:00 +CPU= 43.816
  03:24:07 STEP 2542 H=2542:00 +CPU= 29.521
  03:24:37 STEP 2543 H=2543:00 +CPU= 30.186
  03:25:30 STEP 2544 H=2544:00 +CPU= 52.485
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhpi1+002544
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhpi1+002544
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhpi1+002544
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMGGhpi1+002520
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMSHhpi1+002520
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMUAhpi1+002520
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMGGhpi1+002532
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMSHhpi1+002532
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMUAhpi1+002532
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_2578_2021050100_123_947_12165667_0_r335842750_105.zip
Uploading the intermediate file: upload_file_105.zip
Uploading trickle at timestep: 9158400
  03:25:59 STEP 2545 H=2545:00 +CPU= 29.017
  03:26:30 STEP 2546 H=2546:00 +CPU= 30.497
  03:27:14 STEP 2547 H=2547:00 +CPU= 44.246
  03:27:44 STEP 2548 H=2548:00 +CPU= 30.295
  03:28:14 STEP 2549 H=2549:00 +CPU= 29.198
  03:28:58 STEP 2550 H=2550:00 +CPU= 44.530
  03:29:27 STEP 2551 H=2551:00 +CPU= 28.957
  03:29:57 STEP 2552 H=2552:00 +CPU= 30.024
  03:30:42 STEP 2553 H=2553:00 +CPU= 44.370
  03:31:12 STEP 2554 H=2554:00 +CPU= 30.506
  03:31:42 STEP 2555 H=2555:00 +CPU= 29.094
  03:32:31 STEP 2556 H=2556:00 +CPU= 49.629
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhpi1+002556
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhpi1+002556
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhpi1+002556
  03:33:01 STEP 2557 H=2557:00 +CPU= 30.062
  03:33:32 STEP 2558 H=2558:00 +CPU= 30.065
  03:34:15 STEP 2559 H=2559:00 +CPU= 43.126
  03:34:46 STEP 2560 H=2560:00 +CPU= 30.566
  03:35:15 STEP 2561 H=2561:00 +CPU= 29.151
  03:35:59 STEP 2562 H=2562:00 +CPU= 44.241
  03:36:29 STEP 2563 H=2563:00 +CPU= 29.753
  03:36:59 STEP 2564 H=2564:00 +CPU= 30.086
  03:37:44 STEP 2565 H=2565:00 +CPU= 44.112
  03:38:14 STEP 2566 H=2566:00 +CPU= 29.576
  03:38:44 STEP 2567 H=2567:00 +CPU= 30.036
  03:39:37 STEP 2568 H=2568:00 +CPU= 53.064
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhpi1+002568
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhpi1+002568
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhpi1+002568
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMGGhpi1+002544
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMSHhpi1+002544
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMUAhpi1+002544
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMGGhpi1+002556
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMSHhpi1+002556
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMUAhpi1+002556
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_2578_2021050100_123_947_12165667_0_r335842750_106.zip
Uploading the intermediate file: upload_file_106.zip
Uploading trickle at timestep: 9244800
  03:40:07 STEP 2569 H=2569:00 +CPU= 29.387
  03:40:38 STEP 2570 H=2570:00 +CPU= 30.780
  03:41:21 STEP 2571 H=2571:00 +CPU= 43.530
  03:41:51 STEP 2572 H=2572:00 +CPU= 29.820
  03:42:20 STEP 2573 H=2573:00 +CPU= 29.268
  03:43:06 STEP 2574 H=2574:00 +CPU= 45.406
  03:43:35 STEP 2575 H=2575:00 +CPU= 28.663
  03:44:05 STEP 2576 H=2576:00 +CPU= 29.849
  03:44:49 STEP 2577 H=2577:00 +CPU= 44.064
 SMILAG TRAJECTORY UNDERGROUND      1 TIMES.
  POINT=            8  MAX ETADOT VERTICAL VEL.=   2.481198290700497E-007
  LON  =    175.000000000000       degrees
  LAT  =   -31.9624433520880       degrees
  03:45:19 STEP 2578 H=2578:00 +CPU= 30.115
  03:45:48 STEP 2579 H=2579:00 +CPU= 29.294
  03:46:38 STEP 2580 H=2580:00 +CPU= 49.271
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhpi1+002580
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhpi1+002580
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhpi1+002580
  03:47:07 STEP 2581 H=2581:00 +CPU= 28.659
  03:47:37 STEP 2582 H=2582:00 +CPU= 30.089
 SMILAG TRAJECTORY UNDERGROUND      1 TIMES.
  POINT=           10  MAX ETADOT VERTICAL VEL.=   1.810225445979151E-007
  LON  =    172.500000000000       degrees
  LAT  =   -34.2054184990488       degrees
  03:48:21 STEP 2583 H=2583:00 +CPU= 43.782
  03:48:52 STEP 2584 H=2584:00 +CPU= 30.635
  03:49:20 STEP 2585 H=2585:00 +CPU= 28.494
  03:50:06 STEP 2586 H=2586:00 +CPU= 45.286
  03:50:34 STEP 2587 H=2587:00 +CPU= 28.489
  03:51:05 STEP 2588 H=2588:00 +CPU= 29.934
  03:51:48 STEP 2589 H=2589:00 +CPU= 43.182
  03:52:18 STEP 2590 H=2590:00 +CPU= 29.789
  03:52:49 STEP 2591 H=2591:00 +CPU= 30.723
  03:53:41 STEP 2592 H=2592:00 +CPU= 52.058
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhpi1+002592
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhpi1+002592
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhpi1+002592
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMGGhpi1+002568
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMSHhpi1+002568
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMUAhpi1+002568
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMGGhpi1+002580
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMSHhpi1+002580
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMUAhpi1+002580
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_2578_2021050100_123_947_12165667_0_r335842750_107.zip
Uploading the intermediate file: upload_file_107.zip
  03:54:11 STEP 2593 H=2593:00 +CPU= 29.387
Uploading trickle at timestep: 9331200
  03:54:41 STEP 2594 H=2594:00 +CPU= 29.866
  03:55:25 STEP 2595 H=2595:00 +CPU= 44.068
  03:55:55 STEP 2596 H=2596:00 +CPU= 29.884
  03:56:24 STEP 2597 H=2597:00 +CPU= 29.393
  03:57:10 STEP 2598 H=2598:00 +CPU= 45.398
  03:57:39 STEP 2599 H=2599:00 +CPU= 29.478
  03:58:10 STEP 2600 H=2600:00 +CPU= 30.597
  03:58:54 STEP 2601 H=2601:00 +CPU= 44.022
  03:59:24 STEP 2602 H=2602:00 +CPU= 30.138
  03:59:53 STEP 2603 H=2603:00 +CPU= 28.735
  04:00:43 STEP 2604 H=2604:00 +CPU= 49.493
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhpi1+002604
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhpi1+002604
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhpi1+002604
  04:01:13 STEP 2605 H=2605:00 +CPU= 29.519
  04:01:42 STEP 2606 H=2606:00 +CPU= 29.674
  04:02:27 STEP 2607 H=2607:00 +CPU= 44.201
  04:02:58 STEP 2608 H=2608:00 +CPU= 30.747
  04:03:27 STEP 2609 H=2609:00 +CPU= 28.999
  04:04:12 STEP 2610 H=2610:00 +CPU= 44.934
  04:04:41 STEP 2611 H=2611:00 +CPU= 29.407
  04:05:12 STEP 2612 H=2612:00 +CPU= 30.222
  04:05:56 STEP 2613 H=2613:00 +CPU= 44.417
  04:06:26 STEP 2614 H=2614:00 +CPU= 29.886
  04:06:57 STEP 2615 H=2615:00 +CPU= 30.782
  04:07:50 STEP 2616 H=2616:00 +CPU= 52.118
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhpi1+002616
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhpi1+002616
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhpi1+002616
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMGGhpi1+002592
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMSHhpi1+002592
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMUAhpi1+002592
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMGGhpi1+002604
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMSHhpi1+002604
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMUAhpi1+002604
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_2578_2021050100_123_947_12165667_0_r335842750_108.zip
Uploading the intermediate file: upload_file_108.zip
Uploading trickle at timestep: 9417600
  04:08:18 STEP 2617 H=2617:00 +CPU= 28.601
  04:08:49 STEP 2618 H=2618:00 +CPU= 30.223
  04:09:32 STEP 2619 H=2619:00 +CPU= 43.273
  04:10:02 STEP 2620 H=2620:00 +CPU= 30.398
  04:10:32 STEP 2621 H=2621:00 +CPU= 29.184
  04:11:17 STEP 2622 H=2622:00 +CPU= 44.814
  04:11:46 STEP 2623 H=2623:00 +CPU= 29.119
  04:12:16 STEP 2624 H=2624:00 +CPU= 29.995
  04:13:00 STEP 2625 H=2625:00 +CPU= 43.882
  04:13:30 STEP 2626 H=2626:00 +CPU= 29.910
  04:13:59 STEP 2627 H=2627:00 +CPU= 29.081
  04:14:48 STEP 2628 H=2628:00 +CPU= 48.694
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhpi1+002628
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhpi1+002628
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhpi1+002628
  04:15:17 STEP 2629 H=2629:00 +CPU= 29.470
  04:15:48 STEP 2630 H=2630:00 +CPU= 30.105
  04:16:31 STEP 2631 H=2631:00 +CPU= 43.042
  04:17:01 STEP 2632 H=2632:00 +CPU= 29.640
  04:17:30 STEP 2633 H=2633:00 +CPU= 28.531
  04:18:14 STEP 2634 H=2634:00 +CPU= 44.497
  04:18:43 STEP 2635 H=2635:00 +CPU= 29.124
  04:19:13 STEP 2636 H=2636:00 +CPU= 29.576
  04:19:57 STEP 2637 H=2637:00 +CPU= 43.263
  04:20:27 STEP 2638 H=2638:00 +CPU= 29.957
  04:20:57 STEP 2639 H=2639:00 +CPU= 30.318
  04:21:51 STEP 2640 H=2640:00 +CPU= 53.214
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhpi1+002640
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhpi1+002640
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhpi1+002640
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMGGhpi1+002616
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMSHhpi1+002616
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMUAhpi1+002616
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMGGhpi1+002628
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMSHhpi1+002628
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMUAhpi1+002628
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_2578_2021050100_123_947_12165667_0_r335842750_109.zip
Uploading the intermediate file: upload_file_109.zip
Uploading trickle at timestep: 9504000
  04:22:20 STEP 2641 H=2641:00 +CPU= 29.276
  04:22:50 STEP 2642 H=2642:00 +CPU= 29.943
  04:23:34 STEP 2643 H=2643:00 +CPU= 43.867
  04:24:04 STEP 2644 H=2644:00 +CPU= 29.949
  04:24:34 STEP 2645 H=2645:00 +CPU= 29.354
  04:25:18 STEP 2646 H=2646:00 +CPU= 44.443
  04:25:47 STEP 2647 H=2647:00 +CPU= 29.152
  04:26:18 STEP 2648 H=2648:00 +CPU= 30.247
  04:27:01 STEP 2649 H=2649:00 +CPU= 43.314
  04:27:31 STEP 2650 H=2650:00 +CPU= 29.955
  04:28:00 STEP 2651 H=2651:00 +CPU= 29.052
  04:28:49 STEP 2652 H=2652:00 +CPU= 49.033
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhpi1+002652
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhpi1+002652
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhpi1+002652
  04:29:18 STEP 2653 H=2653:00 +CPU= 28.771
  04:29:49 STEP 2654 H=2654:00 +CPU= 30.067
  04:30:32 STEP 2655 H=2655:00 +CPU= 43.397
  04:31:02 STEP 2656 H=2656:00 +CPU= 29.609
  04:31:31 STEP 2657 H=2657:00 +CPU= 28.791
  04:32:16 STEP 2658 H=2658:00 +CPU= 44.701
  04:32:45 STEP 2659 H=2659:00 +CPU= 29.000
  04:33:15 STEP 2660 H=2660:00 +CPU= 29.700
  04:34:00 STEP 2661 H=2661:00 +CPU= 44.727
  04:34:30 STEP 2662 H=2662:00 +CPU= 30.393
  04:35:01 STEP 2663 H=2663:00 +CPU= 30.759
  04:35:54 STEP 2664 H=2664:00 +CPU= 52.171
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhpi1+002664
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhpi1+002664
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhpi1+002664
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMGGhpi1+002640
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMSHhpi1+002640
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMUAhpi1+002640
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMGGhpi1+002652
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMSHhpi1+002652
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMUAhpi1+002652
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_2578_2021050100_123_947_12165667_0_r335842750_110.zip
Uploading the intermediate file: upload_file_110.zip
Uploading trickle at timestep: 9590400
  04:36:23 STEP 2665 H=2665:00 +CPU= 29.238
  04:36:53 STEP 2666 H=2666:00 +CPU= 30.070
  04:37:36 STEP 2667 H=2667:00 +CPU= 42.667
  04:38:07 STEP 2668 H=2668:00 +CPU= 30.532
  04:38:36 STEP 2669 H=2669:00 +CPU= 29.365
  04:39:21 STEP 2670 H=2670:00 +CPU= 45.120
  04:39:50 STEP 2671 H=2671:00 +CPU= 28.540
  04:40:20 STEP 2672 H=2672:00 +CPU= 30.019
  04:41:03 STEP 2673 H=2673:00 +CPU= 43.318
  04:41:34 STEP 2674 H=2674:00 +CPU= 30.306
  04:42:03 STEP 2675 H=2675:00 +CPU= 28.826
  04:42:52 STEP 2676 H=2676:00 +CPU= 48.992
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhpi1+002676
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhpi1+002676
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhpi1+002676
  04:43:21 STEP 2677 H=2677:00 +CPU= 29.305
  04:43:52 STEP 2678 H=2678:00 +CPU= 30.183
  04:44:36 STEP 2679 H=2679:00 +CPU= 43.990
  04:45:06 STEP 2680 H=2680:00 +CPU= 30.282
  04:45:36 STEP 2681 H=2681:00 +CPU= 29.274
  04:46:21 STEP 2682 H=2682:00 +CPU= 45.279
  04:46:51 STEP 2683 H=2683:00 +CPU= 29.915
  04:47:22 STEP 2684 H=2684:00 +CPU= 30.722
  04:48:06 STEP 2685 H=2685:00 +CPU= 43.114
  04:48:36 STEP 2686 H=2686:00 +CPU= 29.888
  04:49:06 STEP 2687 H=2687:00 +CPU= 30.354
  04:49:58 STEP 2688 H=2688:00 +CPU= 51.760
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhpi1+002688
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhpi1+002688
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhpi1+002688
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMGGhpi1+002664
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMSHhpi1+002664
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMUAhpi1+002664
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMGGhpi1+002676
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMSHhpi1+002676
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMUAhpi1+002676
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_2578_2021050100_123_947_12165667_0_r335842750_111.zip
Uploading the intermediate file: upload_file_111.zip
  04:50:28 STEP 2689 H=2689:00 +CPU= 29.646
Uploading trickle at timestep: 9676800
  04:50:59 STEP 2690 H=2690:00 +CPU= 30.799
  04:51:42 STEP 2691 H=2691:00 +CPU= 43.074
  04:52:12 STEP 2692 H=2692:00 +CPU= 30.043
  04:52:40 STEP 2693 H=2693:00 +CPU= 28.324
  04:53:24 STEP 2694 H=2694:00 +CPU= 44.019
  04:53:54 STEP 2695 H=2695:00 +CPU= 29.293
  04:54:24 STEP 2696 H=2696:00 +CPU= 29.952
  04:55:08 STEP 2697 H=2697:00 +CPU= 43.662
  04:55:38 STEP 2698 H=2698:00 +CPU= 30.197
  04:56:07 STEP 2699 H=2699:00 +CPU= 29.112
  04:56:57 STEP 2700 H=2700:00 +CPU= 49.237
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhpi1+002700
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhpi1+002700
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhpi1+002700
  04:57:26 STEP 2701 H=2701:00 +CPU= 29.499
  04:57:56 STEP 2702 H=2702:00 +CPU= 29.779
  04:58:40 STEP 2703 H=2703:00 +CPU= 43.928
  04:59:11 STEP 2704 H=2704:00 +CPU= 30.074
  04:59:40 STEP 2705 H=2705:00 +CPU= 29.458
  05:00:25 STEP 2706 H=2706:00 +CPU= 44.259
  05:00:54 STEP 2707 H=2707:00 +CPU= 29.285
  05:01:24 STEP 2708 H=2708:00 +CPU= 30.063
  05:02:09 STEP 2709 H=2709:00 +CPU= 44.309
  05:02:39 STEP 2710 H=2710:00 +CPU= 29.769
  05:03:10 STEP 2711 H=2711:00 +CPU= 30.789
  05:04:02 STEP 2712 H=2712:00 +CPU= 51.967
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhpi1+002712
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhpi1+002712
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhpi1+002712
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMGGhpi1+002688
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMSHhpi1+002688
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMUAhpi1+002688
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMGGhpi1+002700
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMSHhpi1+002700
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMUAhpi1+002700
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_2578_2021050100_123_947_12165667_0_r335842750_112.zip
Uploading the intermediate file: upload_file_112.zip
Uploading trickle at timestep: 9763200
  05:04:32 STEP 2713 H=2713:00 +CPU= 29.502
  05:05:03 STEP 2714 H=2714:00 +CPU= 30.916
  05:05:47 STEP 2715 H=2715:00 +CPU= 44.252
  05:06:17 STEP 2716 H=2716:00 +CPU= 29.958
  05:06:46 STEP 2717 H=2717:00 +CPU= 29.160
  05:07:30 STEP 2718 H=2718:00 +CPU= 43.826
  05:07:59 STEP 2719 H=2719:00 +CPU= 29.000
  05:08:30 STEP 2720 H=2720:00 +CPU= 30.416
  05:09:14 STEP 2721 H=2721:00 +CPU= 44.184
  05:09:44 STEP 2722 H=2722:00 +CPU= 30.008
  05:10:14 STEP 2723 H=2723:00 +CPU= 29.576
  05:11:02 STEP 2724 H=2724:00 +CPU= 48.734
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhpi1+002724
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhpi1+002724
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhpi1+002724
  05:11:32 STEP 2725 H=2725:00 +CPU= 28.853
  05:12:02 STEP 2726 H=2726:00 +CPU= 30.207
  05:12:44 STEP 2727 H=2727:00 +CPU= 42.298
  05:13:15 STEP 2728 H=2728:00 +CPU= 30.100
  05:13:45 STEP 2729 H=2729:00 +CPU= 30.000
  05:14:29 STEP 2730 H=2730:00 +CPU= 44.577
  05:14:58 STEP 2731 H=2731:00 +CPU= 28.993
  05:15:28 STEP 2732 H=2732:00 +CPU= 29.604
  05:16:12 STEP 2733 H=2733:00 +CPU= 43.086
  05:16:41 STEP 2734 H=2734:00 +CPU= 29.242
  05:17:12 STEP 2735 H=2735:00 +CPU= 30.847
  05:18:04 STEP 2736 H=2736:00 +CPU= 51.592
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhpi1+002736
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhpi1+002736
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhpi1+002736
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMGGhpi1+002712
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMSHhpi1+002712
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMUAhpi1+002712
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMGGhpi1+002724
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMSHhpi1+002724
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMUAhpi1+002724
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_2578_2021050100_123_947_12165667_0_r335842750_113.zip
Uploading the intermediate file: upload_file_113.zip
Uploading trickle at timestep: 9849600
  05:18:33 STEP 2737 H=2737:00 +CPU= 29.365
  05:19:05 STEP 2738 H=2738:00 +CPU= 30.964
  05:19:48 STEP 2739 H=2739:00 +CPU= 43.237
  05:20:17 STEP 2740 H=2740:00 +CPU= 29.590
  05:20:47 STEP 2741 H=2741:00 +CPU= 29.909
  05:21:32 STEP 2742 H=2742:00 +CPU= 44.844
  05:22:01 STEP 2743 H=2743:00 +CPU= 28.629
  05:22:31 STEP 2744 H=2744:00 +CPU= 29.771
  05:23:14 STEP 2745 H=2745:00 +CPU= 43.117
  05:23:44 STEP 2746 H=2746:00 +CPU= 29.742
  05:24:14 STEP 2747 H=2747:00 +CPU= 29.895
  05:25:03 STEP 2748 H=2748:00 +CPU= 48.424
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhpi1+002748
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhpi1+002748
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhpi1+002748
  05:25:31 STEP 2749 H=2749:00 +CPU= 28.744
  05:26:02 STEP 2750 H=2750:00 +CPU= 30.542
  05:26:45 STEP 2751 H=2751:00 +CPU= 43.049
  05:27:15 STEP 2752 H=2752:00 +CPU= 29.996
  05:27:45 STEP 2753 H=2753:00 +CPU= 29.473
  05:28:29 STEP 2754 H=2754:00 +CPU= 43.643
  05:28:57 STEP 2755 H=2755:00 +CPU= 28.460
  05:29:28 STEP 2756 H=2756:00 +CPU= 30.064
  05:30:12 STEP 2757 H=2757:00 +CPU= 43.946
  05:30:42 STEP 2758 H=2758:00 +CPU= 30.195
  05:31:13 STEP 2759 H=2759:00 +CPU= 30.225
  05:32:05 STEP 2760 H=2760:00 +CPU= 52.186
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhpi1+002760
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhpi1+002760
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhpi1+002760
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMGGhpi1+002736
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMSHhpi1+002736
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMUAhpi1+002736
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMGGhpi1+002748
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMSHhpi1+002748
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMUAhpi1+002748
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_2578_2021050100_123_947_12165667_0_r335842750_114.zip
Uploading the intermediate file: upload_file_114.zip
Uploading trickle at timestep: 9936000
  05:32:34 STEP 2761 H=2761:00 +CPU= 28.879
  05:33:04 STEP 2762 H=2762:00 +CPU= 30.234
  05:33:48 STEP 2763 H=2763:00 +CPU= 43.609
  05:34:17 STEP 2764 H=2764:00 +CPU= 29.147
  05:34:47 STEP 2765 H=2765:00 +CPU= 29.229
  05:35:31 STEP 2766 H=2766:00 +CPU= 43.833
  05:35:59 STEP 2767 H=2767:00 +CPU= 28.709
  05:36:30 STEP 2768 H=2768:00 +CPU= 30.194
  05:37:13 STEP 2769 H=2769:00 +CPU= 43.626
  05:37:43 STEP 2770 H=2770:00 +CPU= 29.612
  05:38:12 STEP 2771 H=2771:00 +CPU= 29.366
  05:39:01 STEP 2772 H=2772:00 +CPU= 48.790
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhpi1+002772
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhpi1+002772
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhpi1+002772
  05:39:30 STEP 2773 H=2773:00 +CPU= 28.666
  05:40:00 STEP 2774 H=2774:00 +CPU= 30.067
  05:40:43 STEP 2775 H=2775:00 +CPU= 42.904
  05:41:13 STEP 2776 H=2776:00 +CPU= 29.382
  05:41:42 STEP 2777 H=2777:00 +CPU= 29.504
  05:42:27 STEP 2778 H=2778:00 +CPU= 44.338
  05:42:56 STEP 2779 H=2779:00 +CPU= 29.142
  05:43:27 STEP 2780 H=2780:00 +CPU= 30.306
  05:44:11 STEP 2781 H=2781:00 +CPU= 43.822
  05:44:41 STEP 2782 H=2782:00 +CPU= 30.096
  05:45:11 STEP 2783 H=2783:00 +CPU= 29.863
  05:46:03 STEP 2784 H=2784:00 +CPU= 51.227
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhpi1+002784
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhpi1+002784
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhpi1+002784
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMGGhpi1+002760
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMSHhpi1+002760
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMUAhpi1+002760
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMGGhpi1+002772
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMSHhpi1+002772
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMUAhpi1+002772
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_2578_2021050100_123_947_12165667_0_r335842750_115.zip
Uploading the intermediate file: upload_file_115.zip
  05:46:32 STEP 2785 H=2785:00 +CPU= 28.862
Uploading trickle at timestep: 10022400
  05:47:02 STEP 2786 H=2786:00 +CPU= 29.702
  05:47:46 STEP 2787 H=2787:00 +CPU= 43.795
  05:48:15 STEP 2788 H=2788:00 +CPU= 29.348
  05:48:44 STEP 2789 H=2789:00 +CPU= 29.079
  05:49:28 STEP 2790 H=2790:00 +CPU= 43.355
  05:49:57 STEP 2791 H=2791:00 +CPU= 29.603
  05:50:27 STEP 2792 H=2792:00 +CPU= 30.238
  05:51:12 STEP 2793 H=2793:00 +CPU= 44.055
  05:51:42 STEP 2794 H=2794:00 +CPU= 29.844
  05:52:11 STEP 2795 H=2795:00 +CPU= 29.236
  05:52:58 STEP 2796 H=2796:00 +CPU= 47.249
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhpi1+002796
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhpi1+002796
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhpi1+002796
  05:53:27 STEP 2797 H=2797:00 +CPU= 28.871
  05:53:58 STEP 2798 H=2798:00 +CPU= 30.295
  05:54:41 STEP 2799 H=2799:00 +CPU= 43.529
  05:55:11 STEP 2800 H=2800:00 +CPU= 29.750
  05:55:40 STEP 2801 H=2801:00 +CPU= 28.868
  05:56:25 STEP 2802 H=2802:00 +CPU= 44.193
  05:56:54 STEP 2803 H=2803:00 +CPU= 28.824
  05:57:24 STEP 2804 H=2804:00 +CPU= 29.909
  05:58:07 STEP 2805 H=2805:00 +CPU= 43.372
  05:58:37 STEP 2806 H=2806:00 +CPU= 29.584
  05:59:07 STEP 2807 H=2807:00 +CPU= 30.240
  05:59:59 STEP 2808 H=2808:00 +CPU= 51.639
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhpi1+002808
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhpi1+002808
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhpi1+002808
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMGGhpi1+002784
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMSHhpi1+002784
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMUAhpi1+002784
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMGGhpi1+002796
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMSHhpi1+002796
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMUAhpi1+002796
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_2578_2021050100_123_947_12165667_0_r335842750_116.zip
Uploading the intermediate file: upload_file_116.zip
  06:00:28 STEP 2809 H=2809:00 +CPU= 29.283
Uploading trickle at timestep: 10108800
  06:00:58 STEP 2810 H=2810:00 +CPU= 29.941
  06:01:42 STEP 2811 H=2811:00 +CPU= 43.408
  06:02:12 STEP 2812 H=2812:00 +CPU= 30.002
  06:02:41 STEP 2813 H=2813:00 +CPU= 28.772
  06:03:26 STEP 2814 H=2814:00 +CPU= 45.271
  06:03:55 STEP 2815 H=2815:00 +CPU= 28.636
  06:04:25 STEP 2816 H=2816:00 +CPU= 29.803
  06:05:09 STEP 2817 H=2817:00 +CPU= 44.085
  06:05:39 STEP 2818 H=2818:00 +CPU= 29.759
  06:06:09 STEP 2819 H=2819:00 +CPU= 29.769
  06:06:57 STEP 2820 H=2820:00 +CPU= 48.149
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhpi1+002820
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhpi1+002820
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhpi1+002820
  06:07:26 STEP 2821 H=2821:00 +CPU= 29.082
  06:07:56 STEP 2822 H=2822:00 +CPU= 30.170
  06:08:40 STEP 2823 H=2823:00 +CPU= 43.233
  06:09:10 STEP 2824 H=2824:00 +CPU= 30.327
  06:09:40 STEP 2825 H=2825:00 +CPU= 29.488
  06:10:25 STEP 2826 H=2826:00 +CPU= 44.897
  06:10:54 STEP 2827 H=2827:00 +CPU= 28.947
  06:11:24 STEP 2828 H=2828:00 +CPU= 30.079
  06:12:09 STEP 2829 H=2829:00 +CPU= 44.196
  06:12:38 STEP 2830 H=2830:00 +CPU= 29.297
  06:13:09 STEP 2831 H=2831:00 +CPU= 30.494
  06:14:01 STEP 2832 H=2832:00 +CPU= 51.541
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhpi1+002832
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhpi1+002832
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhpi1+002832
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMGGhpi1+002808
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMSHhpi1+002808
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMUAhpi1+002808
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMGGhpi1+002820
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMSHhpi1+002820
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMUAhpi1+002820
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_2578_2021050100_123_947_12165667_0_r335842750_117.zip
Uploading the intermediate file: upload_file_117.zip
Uploading trickle at timestep: 10195200
  06:14:30 STEP 2833 H=2833:00 +CPU= 28.938
  06:15:00 STEP 2834 H=2834:00 +CPU= 29.538
  06:15:43 STEP 2835 H=2835:00 +CPU= 43.122
  06:16:13 STEP 2836 H=2836:00 +CPU= 29.482
  06:16:42 STEP 2837 H=2837:00 +CPU= 29.209
  06:17:26 STEP 2838 H=2838:00 +CPU= 44.269
  06:17:55 STEP 2839 H=2839:00 +CPU= 29.003
  06:18:25 STEP 2840 H=2840:00 +CPU= 29.935
  06:19:09 STEP 2841 H=2841:00 +CPU= 43.249
  06:19:38 STEP 2842 H=2842:00 +CPU= 29.557
  06:20:07 STEP 2843 H=2843:00 +CPU= 28.542
  06:20:56 STEP 2844 H=2844:00 +CPU= 49.501
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhpi1+002844
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhpi1+002844
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhpi1+002844
  06:21:25 STEP 2845 H=2845:00 +CPU= 28.653
  06:21:55 STEP 2846 H=2846:00 +CPU= 29.887
  06:22:40 STEP 2847 H=2847:00 +CPU= 44.384
  06:23:10 STEP 2848 H=2848:00 +CPU= 29.555
  06:23:39 STEP 2849 H=2849:00 +CPU= 29.170
  06:24:23 STEP 2850 H=2850:00 +CPU= 44.209
  06:24:52 STEP 2851 H=2851:00 +CPU= 28.484
  06:25:22 STEP 2852 H=2852:00 +CPU= 29.980
  06:26:05 STEP 2853 H=2853:00 +CPU= 43.282
  06:26:35 STEP 2854 H=2854:00 +CPU= 29.239
  06:27:05 STEP 2855 H=2855:00 +CPU= 30.069
  06:27:47 STEP 2856 H=2856:00 +CPU= 41.708
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhpi1+002856
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhpi1+002856
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhpi1+002856
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMGGhpi1+002832
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMSHhpi1+002832
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMUAhpi1+002832
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMGGhpi1+002844
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMSHhpi1+002844
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMUAhpi1+002844
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_2578_2021050100_123_947_12165667_0_r335842750_118.zip
Uploading the intermediate file: upload_file_118.zip
  06:28:05 STEP 2857 H=2857:00 +CPU= 18.445
Uploading trickle at timestep: 10281600
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.01_x86_64-pc-linux-gnu
(argv1) start_date: 2021050100
(argv2) exptid: hpi1
(argv3) unique_member_id: 2578
(argv4) batchid: 947
(argv5) wuid: 12165667
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/0
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.01
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.01_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/0/oifs_43r3_ps_app_1.01_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/0/oifs_43r3_ps_app_1.01_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_948b22b49ae8f9995388382513f17cab to: /var/lib/boinc-client/slots/0/oifs_43r3_ps_2578_2021050100_123_947_12165667.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/0/oifs_43r3_ps_2578_2021050100_123_947_12165667.zip
ic_ancil_file: ic_ancil_12165667
ifsdata_file: ifsdata_12165667
climate_data_file: clim_data_12165667
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
Copying IC ancils from: ../../projects/climateprediction.net/jf_9f2f9b99833c1e4c3da66fa6aa113e75 to: /var/lib/boinc-client/slots/0/ic_ancil_12165667.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/0/ic_ancil_12165667.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_8cbbda7835cc858836f846d3cfadfb23 to: /var/lib/boinc-client/slots/0/ifsdata/ifsdata_12165667.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/0/ifsdata/ifsdata_12165667.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/0/159l_2/clim_data_12165667.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/0/159l_2/clim_data_12165667.zip
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_2578_2021050100_123_947_12165667_0_r335842750
The child process has been launched with process id: 134145
Executing the command: /var/lib/boinc-client/slots/0/./master.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:Wild-Dog:1:1:134145:134145] [20221130:141657:1669839417:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:Wild-Dog:1:1:134145:134145] [20221130:141657:1669839417:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:Wild-Dog:1:1:134145:134145] [20221130:141657:1669839417:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:Wild-Dog:1:1:134145:134145] [20221130:141657:1669839417:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:Wild-Dog:1:1:134145:134145] [20221130:141657:1669839417:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:Wild-Dog:1:1:134145:134145] [20221130:141657:1669839417:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:Wild-Dog:1:1:134145:134145] [20221130:141657:1669839417:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:Wild-Dog:1:1:134145:134145] [20221130:141657:1669839417:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:Wild-Dog:1:1:134145:134145] [20221130:141657:1669839417:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:Wild-Dog:1:1:134145:134145] [20221130:141657:1669839417:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:Wild-Dog:1:1:134145:134145] [20221130:141657:1669839417:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:Wild-Dog:1:1:134145:134145] [20221130:141657:1669839417:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:Wild-Dog:1:1:134145:134145] [20221130:141657:1669839417:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:Wild-Dog:1:1:134145:134145] [20221130:141657:1669839417:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:Wild-Dog:1:1:134145:134145] [20221130:141657:1669839417:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:Wild-Dog:1:1:134145:134145] [20221130:141657:1669839417:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Wild-Dog:1:1:134145:134145] [20221130:141657:1669839417:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:Wild-Dog:1:1:134145:134145] [20221130:141657:1669839417:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Wild-Dog:1:1:134145:134145] [20221130:141657:1669839417:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:Wild-Dog:1:1:134145:134145] [20221130:141657:1669839417:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:Wild-Dog:1:1:134145:134145] [20221130:141657:1669839417:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Wild-Dog:1:1:134145:134145] [20221130:141657:1669839417:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Wild-Dog:1:1:134145:134145] [20221130:141657:1669839417:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:Wild-Dog:1:1:134145:134145] [20221130:141657:1669839417:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Wild-Dog:1:1:134145:134145] [20221130:141657:1669839417:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Wild-Dog:1:1:134145:134145] [20221130:141657:1669839417:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Wild-Dog:1:1:134145:134145] [20221130:141657:1669839417:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:Wild-Dog:1:1:134145:134145] [20221130:141657:1669839417:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:Wild-Dog:1:1:134145:134145] [20221130:141657:1669839417:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:Wild-Dog:1:1:134145:134145] [20221130:141657:1669839417:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  14:17:37 STEP 2856 H=2856:00 +CPU= 37.121
  14:17:58 STEP 2857 H=2857:00 +CPU= 21.027
  14:18:20 STEP 2858 H=2858:00 +CPU= 21.618
  14:18:52 STEP 2859 H=2859:00 +CPU= 32.068
  14:19:13 STEP 2860 H=2860:00 +CPU= 20.841
  14:19:33 STEP 2861 H=2861:00 +CPU= 20.274
  14:20:05 STEP 2862 H=2862:00 +CPU= 31.976
  14:20:26 STEP 2863 H=2863:00 +CPU= 20.408
  14:20:47 STEP 2864 H=2864:00 +CPU= 21.028
  14:21:18 STEP 2865 H=2865:00 +CPU= 31.388
  14:21:40 STEP 2866 H=2866:00 +CPU= 21.821
  14:22:02 STEP 2867 H=2867:00 +CPU= 21.355
  14:22:42 STEP 2868 H=2868:00 +CPU= 40.610
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhpi1+002868
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhpi1+002868
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhpi1+002868
  14:23:09 STEP 2869 H=2869:00 +CPU= 26.071
  14:23:35 STEP 2870 H=2870:00 +CPU= 26.162
  14:24:14 STEP 2871 H=2871:00 +CPU= 38.666
  14:24:42 STEP 2872 H=2872:00 +CPU= 28.256
  14:25:10 STEP 2873 H=2873:00 +CPU= 27.656
  14:25:53 STEP 2874 H=2874:00 +CPU= 42.956
  14:26:20 STEP 2875 H=2875:00 +CPU= 27.017
  14:26:49 STEP 2876 H=2876:00 +CPU= 29.036
  14:27:30 STEP 2877 H=2877:00 +CPU= 41.629
  14:27:59 STEP 2878 H=2878:00 +CPU= 28.611
  14:28:28 STEP 2879 H=2879:00 +CPU= 28.553
  14:29:19 STEP 2880 H=2880:00 +CPU= 50.852
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhpi1+002880
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhpi1+002880
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhpi1+002880
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMGGhpi1+002856
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMSHhpi1+002856
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMUAhpi1+002856
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMGGhpi1+002868
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMSHhpi1+002868
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMUAhpi1+002868
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_2578_2021050100_123_947_12165667_0_r335842750_1.zip
Uploading the intermediate file: upload_file_1.zip
  14:29:47 STEP 2881 H=2881:00 +CPU= 28.535
Uploading trickle at timestep: 10368000
  14:30:18 STEP 2882 H=2882:00 +CPU= 30.751
  14:31:02 STEP 2883 H=2883:00 +CPU= 43.320
  14:31:31 STEP 2884 H=2884:00 +CPU= 29.723
  14:32:01 STEP 2885 H=2885:00 +CPU= 29.606
  14:32:46 STEP 2886 H=2886:00 +CPU= 45.001
  14:33:15 STEP 2887 H=2887:00 +CPU= 28.478
  14:33:46 STEP 2888 H=2888:00 +CPU= 31.207
  14:34:29 STEP 2889 H=2889:00 +CPU= 43.029
  14:34:59 STEP 2890 H=2890:00 +CPU= 30.276
  14:35:28 STEP 2891 H=2891:00 +CPU= 29.065
  14:36:18 STEP 2892 H=2892:00 +CPU= 48.535
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhpi1+002892
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhpi1+002892
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhpi1+002892
  14:36:47 STEP 2893 H=2893:00 +CPU= 28.755
  14:37:17 STEP 2894 H=2894:00 +CPU= 30.080
  14:38:01 STEP 2895 H=2895:00 +CPU= 43.476
  14:38:31 STEP 2896 H=2896:00 +CPU= 29.674
  14:39:00 STEP 2897 H=2897:00 +CPU= 29.288
  14:39:46 STEP 2898 H=2898:00 +CPU= 45.545
  14:40:15 STEP 2899 H=2899:00 +CPU= 28.819
  14:40:45 STEP 2900 H=2900:00 +CPU= 30.080
  14:41:29 STEP 2901 H=2901:00 +CPU= 43.500
  14:41:59 STEP 2902 H=2902:00 +CPU= 30.020
  14:42:29 STEP 2903 H=2903:00 +CPU= 29.823
  14:43:21 STEP 2904 H=2904:00 +CPU= 51.446
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhpi1+002904
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhpi1+002904
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhpi1+002904
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMGGhpi1+002880
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMSHhpi1+002880
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMUAhpi1+002880
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMGGhpi1+002892
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMSHhpi1+002892
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMUAhpi1+002892
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_2578_2021050100_123_947_12165667_0_r335842750_2.zip
Uploading the intermediate file: upload_file_2.zip
Uploading trickle at timestep: 10454400
  14:43:50 STEP 2905 H=2905:00 +CPU= 29.072
  14:44:20 STEP 2906 H=2906:00 +CPU= 29.949
  14:45:04 STEP 2907 H=2907:00 +CPU= 43.995
  14:45:34 STEP 2908 H=2908:00 +CPU= 29.447
  14:46:04 STEP 2909 H=2909:00 +CPU= 29.638
  14:46:48 STEP 2910 H=2910:00 +CPU= 44.456
  14:47:18 STEP 2911 H=2911:00 +CPU= 29.433
  14:47:49 STEP 2912 H=2912:00 +CPU= 30.644
  14:48:32 STEP 2913 H=2913:00 +CPU= 43.585
  14:49:02 STEP 2914 H=2914:00 +CPU= 29.725
  14:49:31 STEP 2915 H=2915:00 +CPU= 28.961
  14:50:20 STEP 2916 H=2916:00 +CPU= 48.239
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhpi1+002916
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhpi1+002916
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhpi1+002916
  14:50:50 STEP 2917 H=2917:00 +CPU= 30.116
  14:51:20 STEP 2918 H=2918:00 +CPU= 29.982
  14:52:05 STEP 2919 H=2919:00 +CPU= 43.948
  14:52:34 STEP 2920 H=2920:00 +CPU= 29.778
  14:53:04 STEP 2921 H=2921:00 +CPU= 29.048
  14:53:48 STEP 2922 H=2922:00 +CPU= 44.512
  14:54:18 STEP 2923 H=2923:00 +CPU= 29.254
  14:54:48 STEP 2924 H=2924:00 +CPU= 29.995
  14:55:32 STEP 2925 H=2925:00 +CPU= 43.970
  14:56:02 STEP 2926 H=2926:00 +CPU= 29.868
  14:56:33 STEP 2927 H=2927:00 +CPU= 30.194
  14:57:25 STEP 2928 H=2928:00 +CPU= 52.139
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhpi1+002928
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhpi1+002928
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhpi1+002928
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMGGhpi1+002904
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMSHhpi1+002904
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMUAhpi1+002904
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMGGhpi1+002916
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMSHhpi1+002916
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMUAhpi1+002916
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_2578_2021050100_123_947_12165667_0_r335842750_3.zip
Uploading the intermediate file: upload_file_3.zip
Uploading trickle at timestep: 10540800
  14:57:54 STEP 2929 H=2929:00 +CPU= 28.729
  14:58:24 STEP 2930 H=2930:00 +CPU= 29.999
  14:59:08 STEP 2931 H=2931:00 +CPU= 43.444
  14:59:37 STEP 2932 H=2932:00 +CPU= 29.387
  15:00:06 STEP 2933 H=2933:00 +CPU= 29.197
  15:00:51 STEP 2934 H=2934:00 +CPU= 44.936
  15:01:20 STEP 2935 H=2935:00 +CPU= 28.736
  15:01:50 STEP 2936 H=2936:00 +CPU= 29.915
  15:02:34 STEP 2937 H=2937:00 +CPU= 43.518
  15:03:04 STEP 2938 H=2938:00 +CPU= 29.781
  15:03:33 STEP 2939 H=2939:00 +CPU= 28.960
  15:04:23 STEP 2940 H=2940:00 +CPU= 49.344
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhpi1+002940
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhpi1+002940
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhpi1+002940
  15:04:53 STEP 2941 H=2941:00 +CPU= 29.318
  15:05:23 STEP 2942 H=2942:00 +CPU= 30.021
  15:06:08 STEP 2943 H=2943:00 +CPU= 44.414
  15:06:38 STEP 2944 H=2944:00 +CPU= 29.917
  15:07:07 STEP 2945 H=2945:00 +CPU= 29.232
  15:07:53 STEP 2946 H=2946:00 +CPU= 45.383
  15:08:22 STEP 2947 H=2947:00 +CPU= 28.746
  15:08:51 STEP 2948 H=2948:00 +CPU= 29.694
  15:09:36 STEP 2949 H=2949:00 +CPU= 44.844
  15:10:06 STEP 2950 H=2950:00 +CPU= 30.050
  15:10:37 STEP 2951 H=2951:00 +CPU= 30.424
  15:11:28 STEP 2952 H=2952:00 +CPU= 50.447
The child process terminated with status: 0
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhpi1+002952
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhpi1+002952
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhpi1+002952
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMSHhpi1+002940
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMSHhpi1+002928
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMGGhpi1+002928
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMUAhpi1+002928
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMSHhpi1+002952
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMGGhpi1+002952
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMUAhpi1+002952
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMGGhpi1+002940
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12165667/ICMUAhpi1+002940
Zipping up the final file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_2578_2021050100_123_947_12165667_0_r335842750_4.zip
Uploading the final file: upload_file_4.zip
Uploading trickle at timestep: 10623600
15:15:21 (134140): called boinc_finish(0)

</stderr_txt>
<message>
upload failure: <file_xfer_error>
  <file_name>oifs_43r3_ps_2578_2021050100_123_947_12165667_0_r335842750_119.zip</file_name>
  <error_code>-161 (not found)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>oifs_43r3_ps_2578_2021050100_123_947_12165667_0_r335842750_120.zip</file_name>
  <error_code>-161 (not found)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>oifs_43r3_ps_2578_2021050100_123_947_12165667_0_r335842750_121.zip</file_name>
  <error_code>-161 (not found)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>oifs_43r3_ps_2578_2021050100_123_947_12165667_0_r335842750_122.zip</file_name>
  <error_code>-161 (not found)</error_code>
</file_xfer_error>
</message>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 May 2023 15:16:33 1521562 22247691 oifs_43r3_ps_2578_2021050100_123_947_12165667_0 10,623,600 3,256 0.0003
09 May 2023 15:16:33 1521562 22247691 oifs_43r3_ps_2578_2021050100_123_947_12165667_0 10,540,800 2,421 0.0002
09 May 2023 15:16:33 1521562 22247691 oifs_43r3_ps_2578_2021050100_123_947_12165667_0 10,454,400 1,578 0.0002
09 May 2023 15:16:33 1521562 22247691 oifs_43r3_ps_2578_2021050100_123_947_12165667_0 10,368,000 747 0.0001
09 May 2023 15:12:44 1521562 22247691 oifs_43r3_ps_2578_2021050100_123_947_12165667_0 10,281,600 78,816 0.0077
09 May 2023 15:12:44 1521562 22247691 oifs_43r3_ps_2578_2021050100_123_947_12165667_0 10,195,200 77,993 0.0076
09 May 2023 15:12:44 1521562 22247691 oifs_43r3_ps_2578_2021050100_123_947_12165667_0 10,108,800 77,160 0.0076
09 May 2023 15:12:44 1521562 22247691 oifs_43r3_ps_2578_2021050100_123_947_12165667_0 10,022,400 76,325 0.0076
09 May 2023 15:12:17 1521562 22247691 oifs_43r3_ps_2578_2021050100_123_947_12165667_0 9,936,000 75,483 0.0076
09 May 2023 15:12:17 1521562 22247691 oifs_43r3_ps_2578_2021050100_123_947_12165667_0 9,849,600 74,647 0.0076
09 May 2023 15:12:17 1521562 22247691 oifs_43r3_ps_2578_2021050100_123_947_12165667_0 9,763,200 73,805 0.0076
09 May 2023 15:12:17 1521562 22247691 oifs_43r3_ps_2578_2021050100_123_947_12165667_0 9,676,800 72,973 0.0075
09 May 2023 15:11:50 1521562 22247691 oifs_43r3_ps_2578_2021050100_123_947_12165667_0 9,590,400 72,129 0.0075
09 May 2023 15:11:50 1521562 22247691 oifs_43r3_ps_2578_2021050100_123_947_12165667_0 9,504,000 71,283 0.0075
09 May 2023 15:11:50 1521562 22247691 oifs_43r3_ps_2578_2021050100_123_947_12165667_0 9,417,600 70,452 0.0075
09 May 2023 15:11:50 1521562 22247691 oifs_43r3_ps_2578_2021050100_123_947_12165667_0 9,331,200 69,609 0.0075
09 May 2023 15:11:23 1521562 22247691 oifs_43r3_ps_2578_2021050100_123_947_12165667_0 9,244,800 68,765 0.0074
09 May 2023 15:11:23 1521562 22247691 oifs_43r3_ps_2578_2021050100_123_947_12165667_0 9,158,400 67,921 0.0074
09 May 2023 15:11:23 1521562 22247691 oifs_43r3_ps_2578_2021050100_123_947_12165667_0 9,072,000 67,078 0.0074
09 May 2023 15:11:23 1521562 22247691 oifs_43r3_ps_2578_2021050100_123_947_12165667_0 8,985,600 66,245 0.0074


©2024 climateprediction.net