climateprediction.net home page
Task 22303042

Task 22303042

Name oifs_43r3_ps_0890_2013050100_123_982_12199534_2
Workunit 12199534
Created 24 Jan 2023, 18:12:40 UTC
Sent 24 Jan 2023, 18:40:12 UTC
Report deadline 25 Mar 2023, 18:40:12 UTC
Received 27 Jan 2023, 17:37:50 UTC
Server state Over
Outcome Success
Client state Done
Exit status 0 (0x00000000)
Computer ID 1536698
Run time 10 hours 54 min 40 sec
CPU time 10 hours 44 min 37 sec
Validate state Valid
Credit 14,873.04
Device peak FLOPS 8.56 GFLOPS
Application version OpenIFS 43r3 Perturbed Surface v1.05
x86_64-pc-linux-gnu
Peak working set size 4,419.00 MB
Peak swap size 4,978.07 MB
Peak disk usage 4,449.94 MB
Stderr
<core_client_version>7.20.2</core_client_version>
<![CDATA[
<stderr_txt>
:00 +CPU= 11.110
  14:31:30 STEP 2436 H=2436:00 +CPU= 19.418
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGhq0f+002436
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHhq0f+002436
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAhq0f+002436
  14:31:41 STEP 2437 H=2437:00 +CPU= 11.406
  14:31:53 STEP 2438 H=2438:00 +CPU= 11.505
  14:32:10 STEP 2439 H=2439:00 +CPU= 17.672
  14:32:22 STEP 2440 H=2440:00 +CPU= 11.570
  14:32:34 STEP 2441 H=2441:00 +CPU= 11.617
  14:32:52 STEP 2442 H=2442:00 +CPU= 18.114
  14:33:03 STEP 2443 H=2443:00 +CPU= 11.063
  14:33:15 STEP 2444 H=2444:00 +CPU= 11.330
  14:33:33 STEP 2445 H=2445:00 +CPU= 18.180
  14:33:45 STEP 2446 H=2446:00 +CPU= 11.468
  14:33:57 STEP 2447 H=2447:00 +CPU= 11.696
  14:34:19 STEP 2448 H=2448:00 +CPU= 21.746
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGhq0f+002448
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHhq0f+002448
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAhq0f+002448
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMGGhq0f+002424
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMSHhq0f+002424
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMUAhq0f+002424
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMGGhq0f+002436
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMSHhq0f+002436
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMUAhq0f+002436
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0890_2013050100_123_982_12199534_2_r569474425_101.zip
Uploading the intermediate file: upload_file_101.zip
  14:34:30 STEP 2449 H=2449:00 +CPU= 11.574
  14:34:42 STEP 2450 H=2450:00 +CPU= 11.715
  14:35:01 STEP 2451 H=2451:00 +CPU= 18.154
  14:35:13 STEP 2452 H=2452:00 +CPU= 12.024
  14:35:24 STEP 2453 H=2453:00 +CPU= 11.277
  14:35:43 STEP 2454 H=2454:00 +CPU= 18.473
  14:35:55 STEP 2455 H=2455:00 +CPU= 11.563
  14:36:07 STEP 2456 H=2456:00 +CPU= 11.765
  14:36:25 STEP 2457 H=2457:00 +CPU= 17.766
  14:36:37 STEP 2458 H=2458:00 +CPU= 11.737
  14:36:48 STEP 2459 H=2459:00 +CPU= 11.401
  14:37:09 STEP 2460 H=2460:00 +CPU= 20.143
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGhq0f+002460
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHhq0f+002460
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAhq0f+002460
  14:37:21 STEP 2461 H=2461:00 +CPU= 11.588
  14:37:33 STEP 2462 H=2462:00 +CPU= 11.987
  14:37:51 STEP 2463 H=2463:00 +CPU= 18.203
  14:38:04 STEP 2464 H=2464:00 +CPU= 12.192
  14:38:15 STEP 2465 H=2465:00 +CPU= 11.580
  14:38:33 STEP 2466 H=2466:00 +CPU= 17.722
  14:38:45 STEP 2467 H=2467:00 +CPU= 11.378
  14:38:57 STEP 2468 H=2468:00 +CPU= 11.503
  14:39:15 STEP 2469 H=2469:00 +CPU= 18.146
  14:39:27 STEP 2470 H=2470:00 +CPU= 11.926
  14:39:39 STEP 2471 H=2471:00 +CPU= 11.296
  14:40:01 STEP 2472 H=2472:00 +CPU= 21.567
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGhq0f+002472
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHhq0f+002472
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAhq0f+002472
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMGGhq0f+002448
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMSHhq0f+002448
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMUAhq0f+002448
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMGGhq0f+002460
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMSHhq0f+002460
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMUAhq0f+002460
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0890_2013050100_123_982_12199534_2_r569474425_102.zip
Uploading the intermediate file: upload_file_102.zip
  14:40:12 STEP 2473 H=2473:00 +CPU= 11.351
  14:40:24 STEP 2474 H=2474:00 +CPU= 11.972
  14:40:43 STEP 2475 H=2475:00 +CPU= 18.363
  14:40:55 STEP 2476 H=2476:00 +CPU= 11.724
  14:41:06 STEP 2477 H=2477:00 +CPU= 11.439
  14:41:25 STEP 2478 H=2478:00 +CPU= 18.469
  14:41:36 STEP 2479 H=2479:00 +CPU= 11.419
  14:41:48 STEP 2480 H=2480:00 +CPU= 11.805
  14:42:07 STEP 2481 H=2481:00 +CPU= 18.383
  14:42:19 STEP 2482 H=2482:00 +CPU= 12.026
  14:42:31 STEP 2483 H=2483:00 +CPU= 11.809
  14:42:51 STEP 2484 H=2484:00 +CPU= 20.171
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGhq0f+002484
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHhq0f+002484
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAhq0f+002484
  14:43:03 STEP 2485 H=2485:00 +CPU= 11.527
  14:43:14 STEP 2486 H=2486:00 +CPU= 11.613
  14:43:32 STEP 2487 H=2487:00 +CPU= 17.766
  14:43:44 STEP 2488 H=2488:00 +CPU= 11.335
  14:43:55 STEP 2489 H=2489:00 +CPU= 10.973
  14:44:13 STEP 2490 H=2490:00 +CPU= 17.812
  14:44:24 STEP 2491 H=2491:00 +CPU= 10.751
  14:44:35 STEP 2492 H=2492:00 +CPU= 11.628
  14:44:53 STEP 2493 H=2493:00 +CPU= 17.840
  14:45:04 STEP 2494 H=2494:00 +CPU= 10.912
  14:45:17 STEP 2495 H=2495:00 +CPU= 11.876
  14:45:38 STEP 2496 H=2496:00 +CPU= 21.512
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGhq0f+002496
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHhq0f+002496
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAhq0f+002496
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMGGhq0f+002472
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMSHhq0f+002472
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMUAhq0f+002472
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMGGhq0f+002484
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMSHhq0f+002484
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMUAhq0f+002484
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0890_2013050100_123_982_12199534_2_r569474425_103.zip
Uploading the intermediate file: upload_file_103.zip
  14:45:50 STEP 2497 H=2497:00 +CPU= 11.573
  14:46:02 STEP 2498 H=2498:00 +CPU= 11.769
  14:46:20 STEP 2499 H=2499:00 +CPU= 17.829
  14:46:32 STEP 2500 H=2500:00 +CPU= 12.266
  14:46:43 STEP 2501 H=2501:00 +CPU= 11.336
  14:47:01 STEP 2502 H=2502:00 +CPU= 17.661
  14:47:13 STEP 2503 H=2503:00 +CPU= 11.278
  14:47:25 STEP 2504 H=2504:00 +CPU= 11.758
  14:47:42 STEP 2505 H=2505:00 +CPU= 17.491
  14:47:54 STEP 2506 H=2506:00 +CPU= 11.563
  14:48:05 STEP 2507 H=2507:00 +CPU= 11.424
  14:48:26 STEP 2508 H=2508:00 +CPU= 19.946
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGhq0f+002508
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHhq0f+002508
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAhq0f+002508
  14:48:37 STEP 2509 H=2509:00 +CPU= 11.326
  14:48:49 STEP 2510 H=2510:00 +CPU= 11.704
  14:49:07 STEP 2511 H=2511:00 +CPU= 17.622
  14:49:18 STEP 2512 H=2512:00 +CPU= 11.395
  14:49:29 STEP 2513 H=2513:00 +CPU= 11.210
  14:49:48 STEP 2514 H=2514:00 +CPU= 18.233
  14:49:59 STEP 2515 H=2515:00 +CPU= 11.249
  14:50:11 STEP 2516 H=2516:00 +CPU= 11.259
  14:50:28 STEP 2517 H=2517:00 +CPU= 17.274
  14:50:39 STEP 2518 H=2518:00 +CPU= 11.246
  14:50:51 STEP 2519 H=2519:00 +CPU= 11.811
  14:51:13 STEP 2520 H=2520:00 +CPU= 21.118
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGhq0f+002520
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHhq0f+002520
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAhq0f+002520
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMGGhq0f+002496
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMSHhq0f+002496
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMUAhq0f+002496
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMGGhq0f+002508
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMSHhq0f+002508
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMUAhq0f+002508
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0890_2013050100_123_982_12199534_2_r569474425_104.zip
Uploading the intermediate file: upload_file_104.zip
  14:51:24 STEP 2521 H=2521:00 +CPU= 11.087
  14:51:36 STEP 2522 H=2522:00 +CPU= 11.636
  14:51:54 STEP 2523 H=2523:00 +CPU= 17.555
  14:52:05 STEP 2524 H=2524:00 +CPU= 11.197
  14:52:16 STEP 2525 H=2525:00 +CPU= 11.100
  14:52:34 STEP 2526 H=2526:00 +CPU= 17.937
  14:52:46 STEP 2527 H=2527:00 +CPU= 11.737
  14:52:58 STEP 2528 H=2528:00 +CPU= 12.030
  14:53:16 STEP 2529 H=2529:00 +CPU= 17.777
  14:53:28 STEP 2530 H=2530:00 +CPU= 11.714
  14:53:40 STEP 2531 H=2531:00 +CPU= 11.488
  14:53:59 STEP 2532 H=2532:00 +CPU= 19.307
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGhq0f+002532
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHhq0f+002532
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAhq0f+002532
  14:54:11 STEP 2533 H=2533:00 +CPU= 11.362
  14:54:22 STEP 2534 H=2534:00 +CPU= 11.644
  14:54:40 STEP 2535 H=2535:00 +CPU= 17.569
  14:54:52 STEP 2536 H=2536:00 +CPU= 11.459
  14:55:03 STEP 2537 H=2537:00 +CPU= 11.295
  14:55:21 STEP 2538 H=2538:00 +CPU= 17.751
  14:55:32 STEP 2539 H=2539:00 +CPU= 10.832
  14:55:43 STEP 2540 H=2540:00 +CPU= 11.115
  14:56:01 STEP 2541 H=2541:00 +CPU= 17.308
  14:56:12 STEP 2542 H=2542:00 +CPU= 11.436
  14:56:24 STEP 2543 H=2543:00 +CPU= 11.678
  14:56:46 STEP 2544 H=2544:00 +CPU= 21.503
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGhq0f+002544
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHhq0f+002544
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAhq0f+002544
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMGGhq0f+002520
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMSHhq0f+002520
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMUAhq0f+002520
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMGGhq0f+002532
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMSHhq0f+002532
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMUAhq0f+002532
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0890_2013050100_123_982_12199534_2_r569474425_105.zip
Uploading the intermediate file: upload_file_105.zip
  14:56:57 STEP 2545 H=2545:00 +CPU= 11.270
  14:57:09 STEP 2546 H=2546:00 +CPU= 11.619
Uploading trickle at timestep: 9158400
  14:57:27 STEP 2547 H=2547:00 +CPU= 17.356
  14:57:38 STEP 2548 H=2548:00 +CPU= 11.465
  14:57:50 STEP 2549 H=2549:00 +CPU= 11.149
  14:58:08 STEP 2550 H=2550:00 +CPU= 18.086
  14:58:19 STEP 2551 H=2551:00 +CPU= 11.177
  14:58:30 STEP 2552 H=2552:00 +CPU= 11.236
  14:58:48 STEP 2553 H=2553:00 +CPU= 17.339
  14:59:00 STEP 2554 H=2554:00 +CPU= 11.625
  14:59:11 STEP 2555 H=2555:00 +CPU= 11.252
  14:59:32 STEP 2556 H=2556:00 +CPU= 20.240
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGhq0f+002556
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHhq0f+002556
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAhq0f+002556
  14:59:44 STEP 2557 H=2557:00 +CPU= 11.666
  14:59:56 STEP 2558 H=2558:00 +CPU= 11.845
  15:00:14 STEP 2559 H=2559:00 +CPU= 18.011
  15:00:26 STEP 2560 H=2560:00 +CPU= 11.870
  15:00:38 STEP 2561 H=2561:00 +CPU= 11.672
  15:00:57 STEP 2562 H=2562:00 +CPU= 18.706
  15:01:09 STEP 2563 H=2563:00 +CPU= 11.473
  15:01:20 STEP 2564 H=2564:00 +CPU= 11.668
  15:01:38 STEP 2565 H=2565:00 +CPU= 17.775
  15:01:50 STEP 2566 H=2566:00 +CPU= 11.686
  15:02:02 STEP 2567 H=2567:00 +CPU= 11.544
  15:02:23 STEP 2568 H=2568:00 +CPU= 21.169
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGhq0f+002568
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHhq0f+002568
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAhq0f+002568
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMGGhq0f+002544
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMSHhq0f+002544
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMUAhq0f+002544
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMGGhq0f+002556
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMSHhq0f+002556
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMUAhq0f+002556
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0890_2013050100_123_982_12199534_2_r569474425_106.zip
Uploading the intermediate file: upload_file_106.zip
  15:02:34 STEP 2569 H=2569:00 +CPU= 10.916
  15:02:46 STEP 2570 H=2570:00 +CPU= 11.240
  15:03:04 STEP 2571 H=2571:00 +CPU= 17.863
  15:03:15 STEP 2572 H=2572:00 +CPU= 11.585
  15:03:27 STEP 2573 H=2573:00 +CPU= 11.206
  15:03:45 STEP 2574 H=2574:00 +CPU= 18.233
  15:03:57 STEP 2575 H=2575:00 +CPU= 11.527
  15:04:09 STEP 2576 H=2576:00 +CPU= 11.987
  15:04:27 STEP 2577 H=2577:00 +CPU= 17.930
  15:04:39 STEP 2578 H=2578:00 +CPU= 11.468
  15:04:50 STEP 2579 H=2579:00 +CPU= 11.406
  15:05:09 STEP 2580 H=2580:00 +CPU= 19.173
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGhq0f+002580
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHhq0f+002580
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAhq0f+002580
  15:05:21 STEP 2581 H=2581:00 +CPU= 11.199
  15:05:32 STEP 2582 H=2582:00 +CPU= 11.353
  15:05:50 STEP 2583 H=2583:00 +CPU= 17.404
  15:06:02 STEP 2584 H=2584:00 +CPU= 11.672
  15:06:13 STEP 2585 H=2585:00 +CPU= 11.268
  15:06:31 STEP 2586 H=2586:00 +CPU= 18.121
  15:06:43 STEP 2587 H=2587:00 +CPU= 11.391
  15:06:55 STEP 2588 H=2588:00 +CPU= 12.068
  15:07:13 STEP 2589 H=2589:00 +CPU= 17.760
  15:07:25 STEP 2590 H=2590:00 +CPU= 11.601
  15:07:37 STEP 2591 H=2591:00 +CPU= 11.614
  15:07:58 STEP 2592 H=2592:00 +CPU= 21.073
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGhq0f+002592
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHhq0f+002592
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAhq0f+002592
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMGGhq0f+002568
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMSHhq0f+002568
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMUAhq0f+002568
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMGGhq0f+002580
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMSHhq0f+002580
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMUAhq0f+002580
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0890_2013050100_123_982_12199534_2_r569474425_107.zip
Uploading the intermediate file: upload_file_107.zip
  15:08:10 STEP 2593 H=2593:00 +CPU= 11.435
  15:08:22 STEP 2594 H=2594:00 +CPU= 11.663
  15:08:40 STEP 2595 H=2595:00 +CPU= 17.592
  15:08:52 STEP 2596 H=2596:00 +CPU= 11.657
  15:09:03 STEP 2597 H=2597:00 +CPU= 11.563
  15:09:21 STEP 2598 H=2598:00 +CPU= 17.741
  15:09:33 STEP 2599 H=2599:00 +CPU= 11.420
  15:09:45 STEP 2600 H=2600:00 +CPU= 11.720
  15:10:03 STEP 2601 H=2601:00 +CPU= 18.346
  15:10:15 STEP 2602 H=2602:00 +CPU= 11.429
  15:10:27 STEP 2603 H=2603:00 +CPU= 11.486
  15:10:47 STEP 2604 H=2604:00 +CPU= 20.105
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGhq0f+002604
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHhq0f+002604
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAhq0f+002604
  15:10:59 STEP 2605 H=2605:00 +CPU= 11.432
  15:11:10 STEP 2606 H=2606:00 +CPU= 11.739
  15:11:28 STEP 2607 H=2607:00 +CPU= 17.692
  15:11:40 STEP 2608 H=2608:00 +CPU= 11.658
  15:11:52 STEP 2609 H=2609:00 +CPU= 11.707
  15:12:11 STEP 2610 H=2610:00 +CPU= 18.503
  15:12:22 STEP 2611 H=2611:00 +CPU= 11.311
  15:12:34 STEP 2612 H=2612:00 +CPU= 11.690
  15:12:52 STEP 2613 H=2613:00 +CPU= 18.368
  15:13:04 STEP 2614 H=2614:00 +CPU= 11.888
  15:13:16 STEP 2615 H=2615:00 +CPU= 11.704
  15:13:38 STEP 2616 H=2616:00 +CPU= 21.381
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGhq0f+002616
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHhq0f+002616
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAhq0f+002616
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMGGhq0f+002592
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMSHhq0f+002592
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMUAhq0f+002592
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMGGhq0f+002604
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMSHhq0f+002604
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMUAhq0f+002604
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0890_2013050100_123_982_12199534_2_r569474425_108.zip
Uploading the intermediate file: upload_file_108.zip
  15:13:49 STEP 2617 H=2617:00 +CPU= 11.435
  15:14:01 STEP 2618 H=2618:00 +CPU= 11.834
  15:14:20 STEP 2619 H=2619:00 +CPU= 18.379
  15:14:32 STEP 2620 H=2620:00 +CPU= 11.993
  15:14:44 STEP 2621 H=2621:00 +CPU= 11.340
  15:15:03 STEP 2622 H=2622:00 +CPU= 18.409
  15:15:14 STEP 2623 H=2623:00 +CPU= 11.529
  15:15:26 STEP 2624 H=2624:00 +CPU= 11.646
  15:15:44 STEP 2625 H=2625:00 +CPU= 18.243
  15:15:57 STEP 2626 H=2626:00 +CPU= 12.047
  15:16:08 STEP 2627 H=2627:00 +CPU= 11.326
  15:16:28 STEP 2628 H=2628:00 +CPU= 20.086
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGhq0f+002628
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHhq0f+002628
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAhq0f+002628
  15:16:40 STEP 2629 H=2629:00 +CPU= 11.449
  15:16:52 STEP 2630 H=2630:00 +CPU= 12.175
  15:17:10 STEP 2631 H=2631:00 +CPU= 17.719
  15:17:22 STEP 2632 H=2632:00 +CPU= 11.370
  15:17:33 STEP 2633 H=2633:00 +CPU= 11.401
  15:17:51 STEP 2634 H=2634:00 +CPU= 18.152
  15:18:03 STEP 2635 H=2635:00 +CPU= 11.262
  15:18:15 STEP 2636 H=2636:00 +CPU= 11.996
  15:18:33 STEP 2637 H=2637:00 +CPU= 17.783
  15:18:45 STEP 2638 H=2638:00 +CPU= 11.965
  15:18:57 STEP 2639 H=2639:00 +CPU= 11.775
  15:19:19 STEP 2640 H=2640:00 +CPU= 21.910
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGhq0f+002640
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHhq0f+002640
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAhq0f+002640
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMGGhq0f+002616
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMSHhq0f+002616
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMUAhq0f+002616
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMGGhq0f+002628
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMSHhq0f+002628
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMUAhq0f+002628
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0890_2013050100_123_982_12199534_2_r569474425_109.zip
Uploading the intermediate file: upload_file_109.zip
  15:19:31 STEP 2641 H=2641:00 +CPU= 11.524
  15:19:43 STEP 2642 H=2642:00 +CPU= 11.766
  15:20:02 STEP 2643 H=2643:00 +CPU= 18.486
  15:20:14 STEP 2644 H=2644:00 +CPU= 12.070
  15:20:26 STEP 2645 H=2645:00 +CPU= 11.533
  15:20:45 STEP 2646 H=2646:00 +CPU= 18.551
  15:20:56 STEP 2647 H=2647:00 +CPU= 11.532
  15:21:08 STEP 2648 H=2648:00 +CPU= 11.876
  15:21:27 STEP 2649 H=2649:00 +CPU= 18.035
  15:21:38 STEP 2650 H=2650:00 +CPU= 11.402
  15:21:49 STEP 2651 H=2651:00 +CPU= 11.289
  15:22:09 STEP 2652 H=2652:00 +CPU= 19.686
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGhq0f+002652
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHhq0f+002652
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAhq0f+002652
  15:22:21 STEP 2653 H=2653:00 +CPU= 11.281
  15:22:33 STEP 2654 H=2654:00 +CPU= 11.899
  15:22:51 STEP 2655 H=2655:00 +CPU= 18.022
  15:23:03 STEP 2656 H=2656:00 +CPU= 12.086
  15:23:15 STEP 2657 H=2657:00 +CPU= 12.007
  15:23:33 STEP 2658 H=2658:00 +CPU= 18.138
  15:23:44 STEP 2659 H=2659:00 +CPU= 11.030
  15:23:56 STEP 2660 H=2660:00 +CPU= 11.493
  15:24:14 STEP 2661 H=2661:00 +CPU= 17.840
  15:24:26 STEP 2662 H=2662:00 +CPU= 11.770
  15:24:38 STEP 2663 H=2663:00 +CPU= 11.879
  15:25:00 STEP 2664 H=2664:00 +CPU= 21.496
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGhq0f+002664
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHhq0f+002664
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAhq0f+002664
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMGGhq0f+002640
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMSHhq0f+002640
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMUAhq0f+002640
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMGGhq0f+002652
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMSHhq0f+002652
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMUAhq0f+002652
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0890_2013050100_123_982_12199534_2_r569474425_110.zip
Uploading the intermediate file: upload_file_110.zip
  15:25:12 STEP 2665 H=2665:00 +CPU= 11.478
  15:25:24 STEP 2666 H=2666:00 +CPU= 11.922
  15:25:42 STEP 2667 H=2667:00 +CPU= 17.892
  15:25:54 STEP 2668 H=2668:00 +CPU= 11.988
  15:26:06 STEP 2669 H=2669:00 +CPU= 11.614
  15:26:25 STEP 2670 H=2670:00 +CPU= 18.690
  15:26:36 STEP 2671 H=2671:00 +CPU= 11.530
  15:26:48 STEP 2672 H=2672:00 +CPU= 11.758
  15:27:07 STEP 2673 H=2673:00 +CPU= 17.925
  15:27:19 STEP 2674 H=2674:00 +CPU= 12.034
  15:27:31 STEP 2675 H=2675:00 +CPU= 11.556
  15:27:51 STEP 2676 H=2676:00 +CPU= 20.472
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGhq0f+002676
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHhq0f+002676
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAhq0f+002676
  15:28:03 STEP 2677 H=2677:00 +CPU= 11.496
  15:28:15 STEP 2678 H=2678:00 +CPU= 11.937
  15:28:34 STEP 2679 H=2679:00 +CPU= 18.249
  15:28:46 STEP 2680 H=2680:00 +CPU= 11.908
  15:28:58 STEP 2681 H=2681:00 +CPU= 11.708
  15:29:17 STEP 2682 H=2682:00 +CPU= 18.182
  15:29:28 STEP 2683 H=2683:00 +CPU= 11.533
  15:29:40 STEP 2684 H=2684:00 +CPU= 11.683
  15:29:58 STEP 2685 H=2685:00 +CPU= 18.077
  15:30:10 STEP 2686 H=2686:00 +CPU= 11.865
  15:30:22 STEP 2687 H=2687:00 +CPU= 11.858
  15:30:45 STEP 2688 H=2688:00 +CPU= 22.012
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGhq0f+002688
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHhq0f+002688
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAhq0f+002688
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMGGhq0f+002664
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMSHhq0f+002664
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMUAhq0f+002664
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMGGhq0f+002676
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMSHhq0f+002676
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMUAhq0f+002676
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0890_2013050100_123_982_12199534_2_r569474425_111.zip
Uploading the intermediate file: upload_file_111.zip
  15:30:57 STEP 2689 H=2689:00 +CPU= 11.487
  15:31:08 STEP 2690 H=2690:00 +CPU= 11.627
  15:31:27 STEP 2691 H=2691:00 +CPU= 18.019
  15:31:40 STEP 2692 H=2692:00 +CPU= 12.190
  15:31:53 STEP 2693 H=2693:00 +CPU= 12.021
  15:32:13 STEP 2694 H=2694:00 +CPU= 19.411
  15:32:27 STEP 2695 H=2695:00 +CPU= 12.148
  15:32:40 STEP 2696 H=2696:00 +CPU= 12.554
  15:33:00 STEP 2697 H=2697:00 +CPU= 19.059
  15:33:13 STEP 2698 H=2698:00 +CPU= 12.391
  15:33:26 STEP 2699 H=2699:00 +CPU= 12.135
  15:33:48 STEP 2700 H=2700:00 +CPU= 21.377
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGhq0f+002700
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHhq0f+002700
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAhq0f+002700
  15:34:01 STEP 2701 H=2701:00 +CPU= 11.842
  15:34:14 STEP 2702 H=2702:00 +CPU= 12.311
  15:34:33 STEP 2703 H=2703:00 +CPU= 18.446
  15:34:45 STEP 2704 H=2704:00 +CPU= 12.230
  15:34:57 STEP 2705 H=2705:00 +CPU= 11.608
  15:35:15 STEP 2706 H=2706:00 +CPU= 18.261
  15:35:27 STEP 2707 H=2707:00 +CPU= 11.627
  15:35:39 STEP 2708 H=2708:00 +CPU= 11.982
  15:35:57 STEP 2709 H=2709:00 +CPU= 17.906
  15:36:10 STEP 2710 H=2710:00 +CPU= 12.465
  15:36:21 STEP 2711 H=2711:00 +CPU= 11.645
  15:36:44 STEP 2712 H=2712:00 +CPU= 21.903
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGhq0f+002712
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHhq0f+002712
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAhq0f+002712
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMGGhq0f+002688
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMSHhq0f+002688
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMUAhq0f+002688
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMGGhq0f+002700
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMSHhq0f+002700
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMUAhq0f+002700
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0890_2013050100_123_982_12199534_2_r569474425_112.zip
Uploading the intermediate file: upload_file_112.zip
  15:36:55 STEP 2713 H=2713:00 +CPU= 11.640
  15:37:07 STEP 2714 H=2714:00 +CPU= 11.750
  15:37:26 STEP 2715 H=2715:00 +CPU= 18.469
  15:37:38 STEP 2716 H=2716:00 +CPU= 12.479
  15:37:50 STEP 2717 H=2717:00 +CPU= 11.786
  15:38:09 STEP 2718 H=2718:00 +CPU= 18.399
  15:38:21 STEP 2719 H=2719:00 +CPU= 12.064
  15:38:33 STEP 2720 H=2720:00 +CPU= 11.542
  15:38:51 STEP 2721 H=2721:00 +CPU= 17.932
  15:39:02 STEP 2722 H=2722:00 +CPU= 11.533
  15:39:14 STEP 2723 H=2723:00 +CPU= 11.373
  15:39:33 STEP 2724 H=2724:00 +CPU= 19.408
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGhq0f+002724
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHhq0f+002724
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAhq0f+002724
  15:39:45 STEP 2725 H=2725:00 +CPU= 11.488
  15:39:57 STEP 2726 H=2726:00 +CPU= 11.687
  15:40:15 STEP 2727 H=2727:00 +CPU= 18.224
  15:40:28 STEP 2728 H=2728:00 +CPU= 12.130
  15:40:39 STEP 2729 H=2729:00 +CPU= 11.357
  15:40:58 STEP 2730 H=2730:00 +CPU= 18.541
  15:41:09 STEP 2731 H=2731:00 +CPU= 11.218
  15:41:21 STEP 2732 H=2732:00 +CPU= 11.932
  15:41:39 STEP 2733 H=2733:00 +CPU= 17.543
  15:41:50 STEP 2734 H=2734:00 +CPU= 11.309
  15:42:02 STEP 2735 H=2735:00 +CPU= 12.025
  15:42:24 STEP 2736 H=2736:00 +CPU= 21.706
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGhq0f+002736
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHhq0f+002736
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAhq0f+002736
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMGGhq0f+002712
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMSHhq0f+002712
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMUAhq0f+002712
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMGGhq0f+002724
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMSHhq0f+002724
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMUAhq0f+002724
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0890_2013050100_123_982_12199534_2_r569474425_113.zip
Uploading the intermediate file: upload_file_113.zip
  15:42:35 STEP 2737 H=2737:00 +CPU= 11.623
  15:42:47 STEP 2738 H=2738:00 +CPU= 11.639
  15:43:05 STEP 2739 H=2739:00 +CPU= 18.198
  15:43:17 STEP 2740 H=2740:00 +CPU= 11.199
  15:43:28 STEP 2741 H=2741:00 +CPU= 10.977
  15:43:45 STEP 2742 H=2742:00 +CPU= 17.481
  15:43:57 STEP 2743 H=2743:00 +CPU= 11.325
  15:44:08 STEP 2744 H=2744:00 +CPU= 11.721
  15:44:26 STEP 2745 H=2745:00 +CPU= 17.417
  15:44:38 STEP 2746 H=2746:00 +CPU= 11.628
  15:44:49 STEP 2747 H=2747:00 +CPU= 11.224
  15:45:09 STEP 2748 H=2748:00 +CPU= 19.584
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGhq0f+002748
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHhq0f+002748
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAhq0f+002748
  15:45:20 STEP 2749 H=2749:00 +CPU= 11.303
  15:45:32 STEP 2750 H=2750:00 +CPU= 11.695
  15:45:49 STEP 2751 H=2751:00 +CPU= 17.644
  15:46:01 STEP 2752 H=2752:00 +CPU= 11.915
  15:46:13 STEP 2753 H=2753:00 +CPU= 11.694
  15:46:32 STEP 2754 H=2754:00 +CPU= 18.793
  15:46:44 STEP 2755 H=2755:00 +CPU= 11.596
  15:46:56 STEP 2756 H=2756:00 +CPU= 11.615
  15:47:14 STEP 2757 H=2757:00 +CPU= 18.400
  15:47:26 STEP 2758 H=2758:00 +CPU= 11.802
  15:47:38 STEP 2759 H=2759:00 +CPU= 11.876
  15:48:00 STEP 2760 H=2760:00 +CPU= 21.842
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGhq0f+002760
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHhq0f+002760
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAhq0f+002760
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMGGhq0f+002736
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMSHhq0f+002736
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMUAhq0f+002736
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMGGhq0f+002748
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMSHhq0f+002748
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMUAhq0f+002748
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0890_2013050100_123_982_12199534_2_r569474425_114.zip
Uploading the intermediate file: upload_file_114.zip
  15:48:12 STEP 2761 H=2761:00 +CPU= 11.470
  15:48:24 STEP 2762 H=2762:00 +CPU= 12.145
  15:48:42 STEP 2763 H=2763:00 +CPU= 17.928
  15:48:54 STEP 2764 H=2764:00 +CPU= 11.497
  15:49:05 STEP 2765 H=2765:00 +CPU= 10.994
  15:49:23 STEP 2766 H=2766:00 +CPU= 18.411
  15:49:35 STEP 2767 H=2767:00 +CPU= 11.392
  15:49:47 STEP 2768 H=2768:00 +CPU= 11.359
  15:50:04 STEP 2769 H=2769:00 +CPU= 17.821
  15:50:16 STEP 2770 H=2770:00 +CPU= 11.302
  15:50:27 STEP 2771 H=2771:00 +CPU= 11.340
  15:50:47 STEP 2772 H=2772:00 +CPU= 19.970
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGhq0f+002772
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHhq0f+002772
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAhq0f+002772
  15:50:59 STEP 2773 H=2773:00 +CPU= 11.536
  15:51:11 STEP 2774 H=2774:00 +CPU= 12.042
  15:51:30 STEP 2775 H=2775:00 +CPU= 18.040
  15:51:41 STEP 2776 H=2776:00 +CPU= 11.252
  15:51:52 STEP 2777 H=2777:00 +CPU= 10.843
  15:52:10 STEP 2778 H=2778:00 +CPU= 18.012
  15:52:21 STEP 2779 H=2779:00 +CPU= 11.445
  15:52:33 STEP 2780 H=2780:00 +CPU= 11.653
  15:52:51 STEP 2781 H=2781:00 +CPU= 17.922
  15:53:03 STEP 2782 H=2782:00 +CPU= 11.952
  15:53:15 STEP 2783 H=2783:00 +CPU= 11.690
  15:53:37 STEP 2784 H=2784:00 +CPU= 21.869
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGhq0f+002784
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHhq0f+002784
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAhq0f+002784
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMGGhq0f+002760
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMSHhq0f+002760
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMUAhq0f+002760
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMGGhq0f+002772
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMSHhq0f+002772
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMUAhq0f+002772
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0890_2013050100_123_982_12199534_2_r569474425_115.zip
Uploading the intermediate file: upload_file_115.zip
  15:53:49 STEP 2785 H=2785:00 +CPU= 11.763
  15:54:01 STEP 2786 H=2786:00 +CPU= 11.721
Uploading trickle at timestep: 10022400
  15:54:20 STEP 2787 H=2787:00 +CPU= 18.202
  15:54:31 STEP 2788 H=2788:00 +CPU= 11.629
  15:54:43 STEP 2789 H=2789:00 +CPU= 11.212
  15:55:01 STEP 2790 H=2790:00 +CPU= 18.339
  15:55:13 STEP 2791 H=2791:00 +CPU= 11.205
  15:55:24 STEP 2792 H=2792:00 +CPU= 11.647
  15:55:43 STEP 2793 H=2793:00 +CPU= 18.355
  15:55:54 STEP 2794 H=2794:00 +CPU= 11.367
  15:56:06 STEP 2795 H=2795:00 +CPU= 11.091
  15:56:26 STEP 2796 H=2796:00 +CPU= 20.271
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGhq0f+002796
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHhq0f+002796
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAhq0f+002796
  15:56:38 STEP 2797 H=2797:00 +CPU= 11.587
  15:56:50 STEP 2798 H=2798:00 +CPU= 11.600
  15:57:08 STEP 2799 H=2799:00 +CPU= 18.193
  15:57:20 STEP 2800 H=2800:00 +CPU= 11.832
  15:57:31 STEP 2801 H=2801:00 +CPU= 11.339
  15:57:50 STEP 2802 H=2802:00 +CPU= 18.696
  15:58:02 STEP 2803 H=2803:00 +CPU= 11.167
  15:58:14 STEP 2804 H=2804:00 +CPU= 11.759
  15:58:31 STEP 2805 H=2805:00 +CPU= 17.680
  15:58:43 STEP 2806 H=2806:00 +CPU= 11.675
  15:58:55 STEP 2807 H=2807:00 +CPU= 11.832
  15:59:17 STEP 2808 H=2808:00 +CPU= 21.480
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGhq0f+002808
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHhq0f+002808
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAhq0f+002808
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMGGhq0f+002784
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMSHhq0f+002784
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMUAhq0f+002784
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMGGhq0f+002796
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMSHhq0f+002796
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMUAhq0f+002796
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0890_2013050100_123_982_12199534_2_r569474425_116.zip
Uploading the intermediate file: upload_file_116.zip
  15:59:28 STEP 2809 H=2809:00 +CPU= 11.073
  15:59:40 STEP 2810 H=2810:00 +CPU= 11.720
  15:59:58 STEP 2811 H=2811:00 +CPU= 18.149
  16:00:10 STEP 2812 H=2812:00 +CPU= 11.643
  16:00:22 STEP 2813 H=2813:00 +CPU= 11.227
  16:00:41 STEP 2814 H=2814:00 +CPU= 18.699
  16:00:52 STEP 2815 H=2815:00 +CPU= 11.497
  16:01:04 STEP 2816 H=2816:00 +CPU= 11.890
  16:01:22 STEP 2817 H=2817:00 +CPU= 18.165
  16:01:34 STEP 2818 H=2818:00 +CPU= 11.826
  16:01:46 STEP 2819 H=2819:00 +CPU= 11.528
  16:02:06 STEP 2820 H=2820:00 +CPU= 20.112
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGhq0f+002820
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHhq0f+002820
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAhq0f+002820
  16:02:17 STEP 2821 H=2821:00 +CPU= 10.760
  16:02:29 STEP 2822 H=2822:00 +CPU= 11.702
  16:02:47 STEP 2823 H=2823:00 +CPU= 17.602
  16:02:59 STEP 2824 H=2824:00 +CPU= 11.801
  16:03:10 STEP 2825 H=2825:00 +CPU= 11.137
  16:03:28 STEP 2826 H=2826:00 +CPU= 18.145
  16:03:40 STEP 2827 H=2827:00 +CPU= 11.287
  16:03:51 STEP 2828 H=2828:00 +CPU= 11.280
  16:04:09 STEP 2829 H=2829:00 +CPU= 18.176
  16:04:21 STEP 2830 H=2830:00 +CPU= 11.515
  16:04:32 STEP 2831 H=2831:00 +CPU= 11.281
  16:04:54 STEP 2832 H=2832:00 +CPU= 21.452
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGhq0f+002832
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHhq0f+002832
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAhq0f+002832
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMGGhq0f+002808
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMSHhq0f+002808
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMUAhq0f+002808
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMGGhq0f+002820
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMSHhq0f+002820
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMUAhq0f+002820
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0890_2013050100_123_982_12199534_2_r569474425_117.zip
Uploading the intermediate file: upload_file_117.zip
  16:05:06 STEP 2833 H=2833:00 +CPU= 11.430
  16:05:17 STEP 2834 H=2834:00 +CPU= 11.696
  16:05:36 STEP 2835 H=2835:00 +CPU= 17.875
  16:05:47 STEP 2836 H=2836:00 +CPU= 11.799
  16:05:59 STEP 2837 H=2837:00 +CPU= 11.383
  16:06:17 STEP 2838 H=2838:00 +CPU= 17.921
  16:06:28 STEP 2839 H=2839:00 +CPU= 11.229
  16:06:40 STEP 2840 H=2840:00 +CPU= 11.650
  16:06:59 STEP 2841 H=2841:00 +CPU= 17.909
  16:07:10 STEP 2842 H=2842:00 +CPU= 11.770
  16:07:22 STEP 2843 H=2843:00 +CPU= 11.412
  16:07:42 STEP 2844 H=2844:00 +CPU= 20.217
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGhq0f+002844
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHhq0f+002844
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAhq0f+002844
  16:07:54 STEP 2845 H=2845:00 +CPU= 11.344
  16:08:06 STEP 2846 H=2846:00 +CPU= 11.873
  16:08:24 STEP 2847 H=2847:00 +CPU= 17.937
  16:08:36 STEP 2848 H=2848:00 +CPU= 11.684
  16:08:47 STEP 2849 H=2849:00 +CPU= 10.697
  16:09:06 STEP 2850 H=2850:00 +CPU= 18.408
  16:09:17 STEP 2851 H=2851:00 +CPU= 11.539
  16:09:29 STEP 2852 H=2852:00 +CPU= 11.440
  16:09:47 STEP 2853 H=2853:00 +CPU= 17.768
  16:09:58 STEP 2854 H=2854:00 +CPU= 11.559
  16:10:10 STEP 2855 H=2855:00 +CPU= 11.623
  16:10:31 STEP 2856 H=2856:00 +CPU= 20.977
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGhq0f+002856
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHhq0f+002856
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAhq0f+002856
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMGGhq0f+002832
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMSHhq0f+002832
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMUAhq0f+002832
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMGGhq0f+002844
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMSHhq0f+002844
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMUAhq0f+002844
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0890_2013050100_123_982_12199534_2_r569474425_118.zip
Uploading the intermediate file: upload_file_118.zip
  16:10:42 STEP 2857 H=2857:00 +CPU= 11.173
  16:10:54 STEP 2858 H=2858:00 +CPU= 11.484
  16:11:11 STEP 2859 H=2859:00 +CPU= 17.212
  16:11:23 STEP 2860 H=2860:00 +CPU= 11.446
  16:11:34 STEP 2861 H=2861:00 +CPU= 11.348
  16:11:52 STEP 2862 H=2862:00 +CPU= 18.022
  16:12:03 STEP 2863 H=2863:00 +CPU= 10.523
  16:12:14 STEP 2864 H=2864:00 +CPU= 11.290
  16:12:32 STEP 2865 H=2865:00 +CPU= 17.477
  16:12:43 STEP 2866 H=2866:00 +CPU= 11.622
  16:12:55 STEP 2867 H=2867:00 +CPU= 11.239
  16:13:15 STEP 2868 H=2868:00 +CPU= 20.147
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGhq0f+002868
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHhq0f+002868
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAhq0f+002868
  16:13:26 STEP 2869 H=2869:00 +CPU= 11.010
  16:13:38 STEP 2870 H=2870:00 +CPU= 11.594
  16:13:55 STEP 2871 H=2871:00 +CPU= 17.029
  16:14:06 STEP 2872 H=2872:00 +CPU= 11.587
  16:14:18 STEP 2873 H=2873:00 +CPU= 11.203
  16:14:36 STEP 2874 H=2874:00 +CPU= 18.157
  16:14:47 STEP 2875 H=2875:00 +CPU= 11.298
  16:14:59 STEP 2876 H=2876:00 +CPU= 11.489
  16:15:17 STEP 2877 H=2877:00 +CPU= 17.514
  16:15:28 STEP 2878 H=2878:00 +CPU= 11.526
  16:15:40 STEP 2879 H=2879:00 +CPU= 11.584
  16:16:01 STEP 2880 H=2880:00 +CPU= 21.422
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGhq0f+002880
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHhq0f+002880
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAhq0f+002880
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMGGhq0f+002856
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMSHhq0f+002856
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMUAhq0f+002856
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMGGhq0f+002868
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMSHhq0f+002868
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMUAhq0f+002868
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0890_2013050100_123_982_12199534_2_r569474425_119.zip
Uploading the intermediate file: upload_file_119.zip
  16:16:13 STEP 2881 H=2881:00 +CPU= 11.462
  16:16:24 STEP 2882 H=2882:00 +CPU= 11.002
  16:16:41 STEP 2883 H=2883:00 +CPU= 17.092
  16:16:52 STEP 2884 H=2884:00 +CPU= 11.070
  16:17:03 STEP 2885 H=2885:00 +CPU= 10.955
  16:17:21 STEP 2886 H=2886:00 +CPU= 17.752
  16:17:32 STEP 2887 H=2887:00 +CPU= 11.244
  16:17:44 STEP 2888 H=2888:00 +CPU= 11.673
  16:18:02 STEP 2889 H=2889:00 +CPU= 17.722
  16:18:14 STEP 2890 H=2890:00 +CPU= 11.529
  16:18:25 STEP 2891 H=2891:00 +CPU= 11.284
  16:18:45 STEP 2892 H=2892:00 +CPU= 19.528
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGhq0f+002892
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHhq0f+002892
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAhq0f+002892
  16:18:56 STEP 2893 H=2893:00 +CPU= 11.249
  16:19:08 STEP 2894 H=2894:00 +CPU= 11.771
  16:19:25 STEP 2895 H=2895:00 +CPU= 17.358
  16:19:37 STEP 2896 H=2896:00 +CPU= 11.732
  16:19:48 STEP 2897 H=2897:00 +CPU= 10.956
  16:20:06 STEP 2898 H=2898:00 +CPU= 17.878
  16:20:17 STEP 2899 H=2899:00 +CPU= 11.016
  16:20:28 STEP 2900 H=2900:00 +CPU= 11.106
  16:20:46 STEP 2901 H=2901:00 +CPU= 17.671
  16:20:58 STEP 2902 H=2902:00 +CPU= 11.467
  16:21:09 STEP 2903 H=2903:00 +CPU= 11.645
  16:21:31 STEP 2904 H=2904:00 +CPU= 21.227
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGhq0f+002904
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHhq0f+002904
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAhq0f+002904
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMGGhq0f+002880
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMSHhq0f+002880
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMUAhq0f+002880
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMGGhq0f+002892
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMSHhq0f+002892
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMUAhq0f+002892
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0890_2013050100_123_982_12199534_2_r569474425_120.zip
Uploading the intermediate file: upload_file_120.zip
  16:21:43 STEP 2905 H=2905:00 +CPU= 11.104
  16:21:54 STEP 2906 H=2906:00 +CPU= 11.590
  16:22:12 STEP 2907 H=2907:00 +CPU= 17.883
  16:22:24 STEP 2908 H=2908:00 +CPU= 11.731
  16:22:36 STEP 2909 H=2909:00 +CPU= 11.424
  16:22:53 STEP 2910 H=2910:00 +CPU= 17.406
  16:23:05 STEP 2911 H=2911:00 +CPU= 11.387
  16:23:16 STEP 2912 H=2912:00 +CPU= 11.618
  16:23:35 STEP 2913 H=2913:00 +CPU= 17.964
  16:23:47 STEP 2914 H=2914:00 +CPU= 12.092
  16:23:58 STEP 2915 H=2915:00 +CPU= 11.343
  16:24:18 STEP 2916 H=2916:00 +CPU= 19.509
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGhq0f+002916
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHhq0f+002916
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAhq0f+002916
  16:24:29 STEP 2917 H=2917:00 +CPU= 11.477
  16:24:41 STEP 2918 H=2918:00 +CPU= 11.886
  16:24:59 STEP 2919 H=2919:00 +CPU= 17.732
  16:25:10 STEP 2920 H=2920:00 +CPU= 11.134
  16:25:22 STEP 2921 H=2921:00 +CPU= 11.064
  16:25:40 STEP 2922 H=2922:00 +CPU= 17.917
  16:25:51 STEP 2923 H=2923:00 +CPU= 10.788
  16:26:02 STEP 2924 H=2924:00 +CPU= 11.290
  16:26:20 STEP 2925 H=2925:00 +CPU= 17.473
  16:26:31 STEP 2926 H=2926:00 +CPU= 11.729
  16:26:43 STEP 2927 H=2927:00 +CPU= 11.834
  16:27:05 STEP 2928 H=2928:00 +CPU= 21.526
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGhq0f+002928
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHhq0f+002928
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAhq0f+002928
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMGGhq0f+002904
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMSHhq0f+002904
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMUAhq0f+002904
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMGGhq0f+002916
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMSHhq0f+002916
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMUAhq0f+002916
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0890_2013050100_123_982_12199534_2_r569474425_121.zip
Uploading the intermediate file: upload_file_121.zip
  16:27:16 STEP 2929 H=2929:00 +CPU= 11.070
  16:27:28 STEP 2930 H=2930:00 +CPU= 11.639
  16:27:44 STEP 2931 H=2931:00 +CPU= 16.391
  16:27:55 STEP 2932 H=2932:00 +CPU= 11.187
[EC_DRHOOK:fedora-pc:1:1:3155:3155] [20230127:162801:1674836881:8867.788] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1538] Received signal#15 (SIGTERM) :: 3996MB (heap), 4419MB (maxrss), 0MB (maxstack), 0 (paging), nsigs = 1
[EC_DRHOOK:fedora-pc:1:1:3155:3155] [20230127:162801:1674836881:8867.789] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1542] Also activating Harakiri-alarm (SIGALRM=14) to expire after 500s elapsed to prevent hangs, nsigs = 1
[EC_DRHOOK:fedora-pc:1:1:3155:3155] [20230127:162801:1674836881:8867.789] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1544] Harakiri signal handler 'signal_harakiri' for signal#14 (SIGALRM) installed at 0x81f3c0 (old at (nil))
[EC_DRHOOK:fedora-pc:1:1:3155:3155] [20230127:162801:1674836881:8867.789] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1617] Signal#15 was caused by unrecognized si_code [memaddr=0x1], nsigs = 1
[EC_DRHOOK:fedora-pc:1:1:3155:3155] [20230127:162801:1674836881:8867.789] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1686] Starting DrHook backtrace for signal#15, nsigs = 1
[EC_DRHOOK:fedora-pc:1:1:3155:3155] [20230127:162801:1674836881:8867.789] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3843] 3996 MB (maxheap), 4419 MB (maxrss), 0 MB (maxstack)
[EC_DRHOOK:fedora-pc:1:1:3155:3155] [20230127:162801:1674836881:8867.789] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :  MASTER 
[EC_DRHOOK:fedora-pc:1:1:3155:3155] [20230127:162801:1674836881:8867.789] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :   CNT0 
[EC_DRHOOK:fedora-pc:1:1:3155:3155] [20230127:162801:1674836881:8867.789] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :    CNT1 
[EC_DRHOOK:fedora-pc:1:1:3155:3155] [20230127:162801:1674836881:8867.789] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :     CNT2 
[EC_DRHOOK:fedora-pc:1:1:3155:3155] [20230127:162801:1674836881:8867.789] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :      CNT3 
[EC_DRHOOK:fedora-pc:1:1:3155:3155] [20230127:162801:1674836881:8867.789] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :       CNT4 
[EC_DRHOOK:fedora-pc:1:1:3155:3155] [20230127:162801:1674836881:8867.789] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :        STEPO 
[EC_DRHOOK:fedora-pc:1:1:3155:3155] [20230127:162801:1674836881:8867.789] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :         SCAN2M 
[EC_DRHOOK:fedora-pc:1:1:3155:3155] [20230127:162801:1674836881:8867.789] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :          GP_MODEL_HEAP 
[EC_DRHOOK:fedora-pc:1:1:3155:3155] [20230127:162801:1674836881:8867.789] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :           GP_MODEL 
[EC_DRHOOK:fedora-pc:1:1:3155:3155] [20230127:162801:1674836881:8867.789] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :            EC_PHYS_DRV 
[EC_DRHOOK:fedora-pc:1:1:3155:3155] [20230127:162801:1674836881:8867.789] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :             >OMP-PHYSICS              (1001) 
[EC_DRHOOK:fedora-pc:1:1:3155:3155] [20230127:162801:1674836881:8867.789] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :              EC_PHYS 
[EC_DRHOOK:fedora-pc:1:1:3155:3155] [20230127:162801:1674836881:8867.789] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :               CALLPAR 
[EC_DRHOOK:fedora-pc:1:1:3155:3155] [20230127:162801:1674836881:8867.789] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :                CUANCAPE2 
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 2013050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0890
(argv4) batchid: 982
(argv5) wuid: 12199534
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc/slots/2
Project directory is: /var/lib/boinc/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534
..mkdir for temp folder for results failed
Copying: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc/slots/2/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc/slots/2/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_ca3fe9caba5cb67463fe74e35fb92705 to: /var/lib/boinc/slots/2/oifs_43r3_ps_0890_2013050100_123_982_12199534.zip
Unzipping the namelist zip file: /var/lib/boinc/slots/2/oifs_43r3_ps_0890_2013050100_123_982_12199534.zip
ic_ancil_file: ic_ancil_12199534
ifsdata_file: ifsdata_12199534
climate_data_file: clim_data_12199534
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_5ee30dc1a1b111e138fc893548aa72b1 to: /var/lib/boinc/slots/2/ic_ancil_12199534.zip
Unzipping the IC ancils zip file: /var/lib/boinc/slots/2/ic_ancil_12199534.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_b12ac726581762866df613f85e40a70a to: /var/lib/boinc/slots/2/ifsdata/ifsdata_12199534.zip
Unzipping the ifsdata_zip file: /var/lib/boinc/slots/2/ifsdata/ifsdata_12199534.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc/slots/2/159l_2/clim_data_12199534.zip
Unzipping the climate data zip file: /var/lib/boinc/slots/2/159l_2/clim_data_12199534.zip
Checking for progress XML file: /var/lib/boinc/slots/2/progress_file_12199534.xml
Opened progress file ok : /var/lib/boinc/slots/2/progress_file_12199534.xml
-- Model is restarting --
Adjusting last_iter, 2933, to previous model restart step.
Creating progress file: /var/lib/boinc/slots/2/progress_file_12199534.xml
last_cpu_time: 38334
upload_file_number: 122
last_iter: 2927
last_upload: 10540800
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0890_2013050100_123_982_12199534_2_r569474425
The child process has been launched with process id: 2814
Executing the command: /var/lib/boinc/slots/2/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:fedora-pc:1:1:2814:2814] [20230127:162959:1674836999:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:fedora-pc:1:1:2814:2814] [20230127:162959:1674836999:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:fedora-pc:1:1:2814:2814] [20230127:162959:1674836999:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:fedora-pc:1:1:2814:2814] [20230127:162959:1674836999:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:fedora-pc:1:1:2814:2814] [20230127:162959:1674836999:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:fedora-pc:1:1:2814:2814] [20230127:162959:1674836999:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:fedora-pc:1:1:2814:2814] [20230127:162959:1674836999:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:fedora-pc:1:1:2814:2814] [20230127:162959:1674836999:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:fedora-pc:1:1:2814:2814] [20230127:162959:1674836999:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:fedora-pc:1:1:2814:2814] [20230127:162959:1674836999:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:fedora-pc:1:1:2814:2814] [20230127:162959:1674836999:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:fedora-pc:1:1:2814:2814] [20230127:162959:1674836999:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:fedora-pc:1:1:2814:2814] [20230127:162959:1674836999:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:fedora-pc:1:1:2814:2814] [20230127:162959:1674836999:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:fedora-pc:1:1:2814:2814] [20230127:162959:1674836999:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:fedora-pc:1:1:2814:2814] [20230127:162959:1674836999:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:fedora-pc:1:1:2814:2814] [20230127:162959:1674836999:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:fedora-pc:1:1:2814:2814] [20230127:162959:1674836999:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:fedora-pc:1:1:2814:2814] [20230127:162959:1674836999:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:fedora-pc:1:1:2814:2814] [20230127:162959:1674836999:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:fedora-pc:1:1:2814:2814] [20230127:162959:1674836999:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:fedora-pc:1:1:2814:2814] [20230127:162959:1674836999:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:fedora-pc:1:1:2814:2814] [20230127:162959:1674836999:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:fedora-pc:1:1:2814:2814] [20230127:162959:1674836999:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:fedora-pc:1:1:2814:2814] [20230127:162959:1674836999:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:fedora-pc:1:1:2814:2814] [20230127:162959:1674836999:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:fedora-pc:1:1:2814:2814] [20230127:162959:1674836999:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:fedora-pc:1:1:2814:2814] [20230127:162959:1674836999:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:fedora-pc:1:1:2814:2814] [20230127:162959:1674836999:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:fedora-pc:1:1:2814:2814] [20230127:162959:1674836999:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  16:30:21 STEP 2928 H=2928:00 +CPU= 19.705
  16:30:32 STEP 2929 H=2929:00 +CPU= 11.160
  16:30:44 STEP 2930 H=2930:00 +CPU= 11.422
  16:31:01 STEP 2931 H=2931:00 +CPU= 17.317
  16:31:12 STEP 2932 H=2932:00 +CPU= 11.087
  16:31:23 STEP 2933 H=2933:00 +CPU= 11.094
  16:31:41 STEP 2934 H=2934:00 +CPU= 17.848
  16:31:52 STEP 2935 H=2935:00 +CPU= 10.612
  16:32:03 STEP 2936 H=2936:00 +CPU= 11.493
  16:32:21 STEP 2937 H=2937:00 +CPU= 17.025
  16:32:32 STEP 2938 H=2938:00 +CPU= 11.434
  16:32:43 STEP 2939 H=2939:00 +CPU= 10.831
  16:33:02 STEP 2940 H=2940:00 +CPU= 18.996
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGhq0f+002940
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHhq0f+002940
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAhq0f+002940
  16:33:13 STEP 2941 H=2941:00 +CPU= 11.128
  16:33:25 STEP 2942 H=2942:00 +CPU= 11.510
  16:33:42 STEP 2943 H=2943:00 +CPU= 17.134
  16:33:53 STEP 2944 H=2944:00 +CPU= 10.852
  16:34:04 STEP 2945 H=2945:00 +CPU= 11.013
  16:34:22 STEP 2946 H=2946:00 +CPU= 17.893
  16:34:33 STEP 2947 H=2947:00 +CPU= 11.182
  16:34:45 STEP 2948 H=2948:00 +CPU= 11.523
  16:35:02 STEP 2949 H=2949:00 +CPU= 17.390
  16:35:14 STEP 2950 H=2950:00 +CPU= 11.565
  16:35:24 STEP 2951 H=2951:00 +CPU= 10.656
  16:35:44 STEP 2952 H=2952:00 +CPU= 19.751
The child process terminated with status: 0
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGhq0f+002952
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHhq0f+002952
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAhq0f+002952
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMGGhq0f+002928
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMSHhq0f+002928
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMUAhq0f+002928
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMGGhq0f+002940
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMSHhq0f+002940
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMUAhq0f+002940
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMGGhq0f+002952
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMSHhq0f+002952
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12199534/ICMUAhq0f+002952
Zipping up the final file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0890_2013050100_123_982_12199534_2_r569474425_122.zip
Uploading the final file: upload_file_122.zip
Uploading trickle at timestep: 10623600
16:39:07 (2706): called boinc_finish(0)

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 May 2023 16:43:27 1536698 22303042 oifs_43r3_ps_0890_2013050100_123_982_12199534_2 10,623,600 38,677 0.0036
09 May 2023 16:43:25 1536698 22303042 oifs_43r3_ps_0890_2013050100_123_982_12199534_2 10,022,400 36,296 0.0036
09 May 2023 16:43:23 1536698 22303042 oifs_43r3_ps_0890_2013050100_123_982_12199534_2 9,158,400 32,932 0.0036
09 May 2023 16:43:20 1536698 22303042 oifs_43r3_ps_0890_2013050100_123_982_12199534_2 8,121,600 28,826 0.0035
09 May 2023 16:42:23 1536698 22303042 oifs_43r3_ps_0890_2013050100_123_982_12199534_2 7,257,600 25,504 0.0035
09 May 2023 16:42:20 1536698 22303042 oifs_43r3_ps_0890_2013050100_123_982_12199534_2 6,393,600 22,102 0.0035
09 May 2023 16:42:18 1536698 22303042 oifs_43r3_ps_0890_2013050100_123_982_12199534_2 5,529,600 18,834 0.0034
09 May 2023 16:42:15 1536698 22303042 oifs_43r3_ps_0890_2013050100_123_982_12199534_2 4,406,400 14,759 0.0033
09 May 2023 16:42:13 1536698 22303042 oifs_43r3_ps_0890_2013050100_123_982_12199534_2 3,542,400 12,138 0.0034
09 May 2023 16:42:10 1536698 22303042 oifs_43r3_ps_0890_2013050100_123_982_12199534_2 2,332,800 7,505 0.0032
09 May 2023 16:40:52 1536698 22303042 oifs_43r3_ps_0890_2013050100_123_982_12199534_2 1,468,800 4,432 0.0030


©2024 climateprediction.net