climateprediction.net home page
Task 22285211

Task 22285211

Name oifs_43r3_ps_0385_1986050100_123_955_12172029_1
Workunit 12172029
Created 24 Dec 2022, 7:32:19 UTC
Sent 5 Jan 2023, 20:11:24 UTC
Report deadline 4 Feb 2023, 20:11:24 UTC
Received 14 Jan 2023, 1:31:49 UTC
Server state Over
Outcome Success
Client state Done
Exit status 0 (0x00000000)
Computer ID 1536310
Run time 1 days 0 hours 11 min 43 sec
CPU time 19 hours 48 min 43 sec
Validate state Valid
Credit 14,873.04
Device peak FLOPS 5.91 GFLOPS
Application version OpenIFS 43r3 Perturbed Surface v1.05
x86_64-pc-linux-gnu
Peak working set size 4,799.29 MB
Peak swap size 4,978.07 MB
Peak disk usage 3,763.56 MB
Stderr
<core_client_version>7.16.16</core_client_version>
<![CDATA[
<stderr_txt>
p: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMSHhq0f+002472
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMUAhq0f+002472
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMGGhq0f+002484
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMSHhq0f+002484
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMUAhq0f+002484
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0385_1986050100_123_955_12172029_1_r1382893106_103.zip
Uploading the intermediate file: upload_file_103.zip
  20:17:30 STEP 2497 H=2497:00 +CPU= 20.221
  20:17:56 STEP 2498 H=2498:00 +CPU= 22.316
  20:18:35 STEP 2499 H=2499:00 +CPU= 32.922
  20:18:59 STEP 2500 H=2500:00 +CPU= 20.111
  20:19:22 STEP 2501 H=2501:00 +CPU= 20.158
  20:20:00 STEP 2502 H=2502:00 +CPU= 33.054
  20:20:25 STEP 2503 H=2503:00 +CPU= 20.786
  20:20:51 STEP 2504 H=2504:00 +CPU= 22.233
  20:21:27 STEP 2505 H=2505:00 +CPU= 30.650
  20:21:53 STEP 2506 H=2506:00 +CPU= 22.008
  20:22:17 STEP 2507 H=2507:00 +CPU= 20.134
  20:22:58 STEP 2508 H=2508:00 +CPU= 35.246
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMGGhq0f+002508
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMSHhq0f+002508
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMUAhq0f+002508
  20:23:22 STEP 2509 H=2509:00 +CPU= 20.230
  20:23:48 STEP 2510 H=2510:00 +CPU= 22.536
  20:24:26 STEP 2511 H=2511:00 +CPU= 32.773
  20:24:52 STEP 2512 H=2512:00 +CPU= 22.089
  20:25:17 STEP 2513 H=2513:00 +CPU= 20.787
  20:25:56 STEP 2514 H=2514:00 +CPU= 33.623
  20:26:23 STEP 2515 H=2515:00 +CPU= 22.433
  20:26:48 STEP 2516 H=2516:00 +CPU= 20.858
  20:27:27 STEP 2517 H=2517:00 +CPU= 33.793
  20:27:53 STEP 2518 H=2518:00 +CPU= 22.351
  20:28:21 STEP 2519 H=2519:00 +CPU= 22.958
  20:29:07 STEP 2520 H=2520:00 +CPU= 38.989
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMGGhq0f+002520
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMSHhq0f+002520
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMUAhq0f+002520
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMGGhq0f+002496
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMSHhq0f+002496
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMUAhq0f+002496
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMGGhq0f+002508
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMSHhq0f+002508
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMUAhq0f+002508
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0385_1986050100_123_955_12172029_1_r1382893106_104.zip
Uploading the intermediate file: upload_file_104.zip
  20:29:30 STEP 2521 H=2521:00 +CPU= 19.784
  20:29:57 STEP 2522 H=2522:00 +CPU= 22.596
  20:30:35 STEP 2523 H=2523:00 +CPU= 32.353
  20:31:00 STEP 2524 H=2524:00 +CPU= 21.481
  20:31:27 STEP 2525 H=2525:00 +CPU= 22.447
  20:32:06 STEP 2526 H=2526:00 +CPU= 33.404
  20:32:30 STEP 2527 H=2527:00 +CPU= 20.643
  20:32:56 STEP 2528 H=2528:00 +CPU= 22.004
  20:33:35 STEP 2529 H=2529:00 +CPU= 34.020
  20:34:02 STEP 2530 H=2530:00 +CPU= 22.629
  20:34:26 STEP 2531 H=2531:00 +CPU= 20.398
  20:35:09 STEP 2532 H=2532:00 +CPU= 36.554
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMGGhq0f+002532
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMSHhq0f+002532
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMUAhq0f+002532
  20:35:37 STEP 2533 H=2533:00 +CPU= 23.086
  20:36:05 STEP 2534 H=2534:00 +CPU= 23.591
  20:36:45 STEP 2535 H=2535:00 +CPU= 34.625
  20:37:14 STEP 2536 H=2536:00 +CPU= 24.315
  20:37:39 STEP 2537 H=2537:00 +CPU= 21.169
  20:38:08 STEP 2538 H=2538:00 +CPU= 25.064
  20:38:29 STEP 2539 H=2539:00 +CPU= 18.558
  20:38:55 STEP 2540 H=2540:00 +CPU= 22.744
  20:39:35 STEP 2541 H=2541:00 +CPU= 33.782
  20:40:08 STEP 2542 H=2542:00 +CPU= 26.082
  20:40:28 STEP 2543 H=2543:00 +CPU= 16.528
  20:41:03 STEP 2544 H=2544:00 +CPU= 30.770
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMGGhq0f+002544
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMSHhq0f+002544
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMUAhq0f+002544
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMGGhq0f+002520
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMSHhq0f+002520
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMUAhq0f+002520
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMGGhq0f+002532
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMSHhq0f+002532
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMUAhq0f+002532
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0385_1986050100_123_955_12172029_1_r1382893106_105.zip
Uploading the intermediate file: upload_file_105.zip
  20:41:21 STEP 2545 H=2545:00 +CPU= 15.351
  20:41:40 STEP 2546 H=2546:00 +CPU= 16.068
  20:42:07 STEP 2547 H=2547:00 +CPU= 23.822
  20:42:25 STEP 2548 H=2548:00 +CPU= 16.083
  20:42:42 STEP 2549 H=2549:00 +CPU= 15.046
  20:43:09 STEP 2550 H=2550:00 +CPU= 23.067
  20:43:25 STEP 2551 H=2551:00 +CPU= 13.800
  20:43:41 STEP 2552 H=2552:00 +CPU= 14.130
  20:44:07 STEP 2553 H=2553:00 +CPU= 22.574
  20:44:23 STEP 2554 H=2554:00 +CPU= 14.150
  20:44:39 STEP 2555 H=2555:00 +CPU= 13.789
  20:45:07 STEP 2556 H=2556:00 +CPU= 24.363
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMGGhq0f+002556
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMSHhq0f+002556
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMUAhq0f+002556
  20:45:23 STEP 2557 H=2557:00 +CPU= 14.050
  20:45:39 STEP 2558 H=2558:00 +CPU= 14.081
Suspend request received from the BOINC client, suspending the child process
[EC_DRHOOK:debian-virtual:1:1:1303962:1303962] [20230106:215338:1673060018:98779.680] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1538] Received signal#15 (SIGTERM) :: 4745MB (heap), 4796MB (maxrss), 0MB (maxstack), 0 (paging), nsigs = 1
[EC_DRHOOK:debian-virtual:1:1:1303962:1303962] [20230106:215338:1673060018:98779.680] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1542] Also activating Harakiri-alarm (SIGALRM=14) to expire after 500s elapsed to prevent hangs, nsigs = 1
[EC_DRHOOK:debian-virtual:1:1:1303962:1303962] [20230106:215338:1673060018:98779.680] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1544] Harakiri signal handler 'signal_harakiri' for signal#14 (SIGALRM) installed at 0x81f3c0 (old at (nil))
[EC_DRHOOK:debian-virtual:1:1:1303962:1303962] [20230106:215338:1673060018:98779.680] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1617] Signal#15 was caused by unrecognized si_code [memaddr=0x1], nsigs = 1
[EC_DRHOOK:debian-virtual:1:1:1303962:1303962] [20230106:215338:1673060018:98779.680] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1686] Starting DrHook backtrace for signal#15, nsigs = 1
[EC_DRHOOK:debian-virtual:1:1:1303962:1303962] [20230106:215338:1673060018:98779.680] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3843] 4745 MB (maxheap), 4796 MB (maxrss), 0 MB (maxstack)
[EC_DRHOOK:debian-virtual:1:1:1303962:1303962] [20230106:215338:1673060018:98779.680] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :  MASTER 
[EC_DRHOOK:debian-virtual:1:1:1303962:1303962] [20230106:215338:1673060018:98779.680] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :   CNT0 
[EC_DRHOOK:debian-virtual:1:1:1303962:1303962] [20230106:215338:1673060018:98779.680] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :    CNT1 
[EC_DRHOOK:debian-virtual:1:1:1303962:1303962] [20230106:215338:1673060018:98779.680] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :     CNT2 
[EC_DRHOOK:debian-virtual:1:1:1303962:1303962] [20230106:215338:1673060018:98779.680] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :      CNT3 
[EC_DRHOOK:debian-virtual:1:1:1303962:1303962] [20230106:215338:1673060018:98779.680] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :       CNT4 
[EC_DRHOOK:debian-virtual:1:1:1303962:1303962] [20230106:215338:1673060018:98779.680] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :        STEPO 
[EC_DRHOOK:debian-virtual:1:1:1303962:1303962] [20230106:215338:1673060018:98779.680] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :         SCAN2M 
[EC_DRHOOK:debian-virtual:1:1:1303962:1303962] [20230106:215338:1673060018:98779.680] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :          GP_MODEL_HEAP 
[EC_DRHOOK:debian-virtual:1:1:1303962:1303962] [20230106:215338:1673060018:98779.680] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :           GP_MODEL 
[EC_DRHOOK:debian-virtual:1:1:1303962:1303962] [20230106:215338:1673060018:98779.680] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :            CALL_SL_HEAP 
[EC_DRHOOK:debian-virtual:1:1:1303962:1303962] [20230106:215338:1673060018:98779.680] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :             CALL_SL 
[EC_DRHOOK:debian-virtual:1:1:1303962:1303962] [20230106:215338:1673060018:98779.680] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :              >OMP-CALL_SL     2        (1005) 
[EC_DRHOOK:debian-virtual:1:1:1303962:1303962] [20230106:215338:1673060018:98779.680] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :               LAPINEB 
[EC_DRHOOK:debian-virtual:1:1:1303962:1303962] [20230106:215338:1673060018:98779.680] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :                LARCINB 
[EC_DRHOOK:debian-virtual:1:1:1303962:1303962] [20230106:215338:1673060018:98779.680] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :                 LAITLI 
[EC_DRHOOK:debian-virtual:1:1:1303962:1303962] [20230106:215339:1673060019:98780.680] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1734] DrHook backtrace done for signal#15, nsigs = 1
[EC_DRHOOK:debian-virtual:1:1:1303962:1303962] [20230106:215339:1673060019:98780.680] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1785] Calling previous signal handler at 0x1ce8bb0 for signal#15, nsigs = 1
forrtl: error (78): process killed (SIGTERM)
Image              PC                Routine            Line        Source             
oifs_43r3_model.e  0000000001CE916B  Unknown               Unknown  Unknown
oifs_43r3_model.e  000000000081FFF1  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001DC9090  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000000C1A981  laitli_                   157  laitli.F90
oifs_43r3_model.e  0000000000DC633B  larcinb_                  957  larcinb.F90
oifs_43r3_model.e  0000000000D7EBA8  lapineb_                  451  lapineb.F90
oifs_43r3_model.e  0000000000D7B935  call_sl_                  363  call_sl.F90
oifs_43r3_model.e  0000000000D784FD  call_sl_heap_              94  call_sl_heap.F90
oifs_43r3_model.e  0000000000BF9958  gp_model_                 533  gp_model.F90
oifs_43r3_model.e  00000000012E9482  gp_model_heap_             74  gp_model_heap.F90
oifs_43r3_model.e  0000000000BBC1CE  scan2m_                   535  scan2m.F90
oifs_43r3_model.e  000000000057A664  stepo_                    327  stepo.F90
oifs_43r3_model.e  000000000055DEEC  cnt4_                    1133  cnt4.F90
oifs_43r3_model.e  00000000005471C9  cnt3_                     267  cnt3.F90
oifs_43r3_model.e  0000000000546412  cnt2_                      88  cnt2.F90
oifs_43r3_model.e  0000000000545E28  cnt1_                      92  cnt1.F90
oifs_43r3_model.e  000000000040708D  cnt0_                     146  cnt0.F90
oifs_43r3_model.e  000000000040220F  MAIN__                     96  master.F90
oifs_43r3_model.e  00000000004021A2  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001DCA390  Unknown               Unknown  Unknown
oifs_43r3_model.e  000000000040206E  Unknown               Unknown  Unknown
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 1986050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0385
(argv4) batchid: 955
(argv5) wuid: 12172029
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/20
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/20/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/20/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_ccedd1136f9d860cacaa9360fa8eb64e to: /var/lib/boinc-client/slots/20/oifs_43r3_ps_0385_1986050100_123_955_12172029.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/20/oifs_43r3_ps_0385_1986050100_123_955_12172029.zip
ic_ancil_file: ic_ancil_12172029
ifsdata_file: ifsdata_12172029
climate_data_file: clim_data_12172029
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_7b4c981ed2175fa2c6f920a3f8d315f8 to: /var/lib/boinc-client/slots/20/ic_ancil_12172029.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/20/ic_ancil_12172029.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_5f4e920da64d0db48afdd2ff31932500 to: /var/lib/boinc-client/slots/20/ifsdata/ifsdata_12172029.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/20/ifsdata/ifsdata_12172029.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/20/159l_2/clim_data_12172029.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/20/159l_2/clim_data_12172029.zip
Checking for progress XML file: /var/lib/boinc-client/slots/20/progress_file_12172029.xml
Opened progress file ok : /var/lib/boinc-client/slots/20/progress_file_12172029.xml
-- Model is restarting --
Adjusting last_iter, 2559, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/20/progress_file_12172029.xml
last_cpu_time: 62892
upload_file_number: 106
last_iter: 2543
last_upload: 9158400
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0385_1986050100_123_955_12172029_1_r1382893106
The child process has been launched with process id: 2086
Executing the command: /var/lib/boinc-client/slots/20/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:debian-virtual:1:1:2086:2086] [20230106:231352:1673064832:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:debian-virtual:1:1:2086:2086] [20230106:231352:1673064832:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:debian-virtual:1:1:2086:2086] [20230106:231352:1673064832:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:debian-virtual:1:1:2086:2086] [20230106:231352:1673064832:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:debian-virtual:1:1:2086:2086] [20230106:231352:1673064832:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:debian-virtual:1:1:2086:2086] [20230106:231352:1673064832:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:debian-virtual:1:1:2086:2086] [20230106:231352:1673064832:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:debian-virtual:1:1:2086:2086] [20230106:231352:1673064832:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:debian-virtual:1:1:2086:2086] [20230106:231352:1673064832:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:debian-virtual:1:1:2086:2086] [20230106:231352:1673064832:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:debian-virtual:1:1:2086:2086] [20230106:231352:1673064832:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:debian-virtual:1:1:2086:2086] [20230106:231352:1673064832:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:debian-virtual:1:1:2086:2086] [20230106:231352:1673064832:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:debian-virtual:1:1:2086:2086] [20230106:231352:1673064832:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:debian-virtual:1:1:2086:2086] [20230106:231352:1673064832:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:debian-virtual:1:1:2086:2086] [20230106:231352:1673064832:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:debian-virtual:1:1:2086:2086] [20230106:231352:1673064832:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:debian-virtual:1:1:2086:2086] [20230106:231352:1673064832:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:debian-virtual:1:1:2086:2086] [20230106:231352:1673064832:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:debian-virtual:1:1:2086:2086] [20230106:231352:1673064832:0.001] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:debian-virtual:1:1:2086:2086] [20230106:231352:1673064832:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:debian-virtual:1:1:2086:2086] [20230106:231352:1673064832:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:debian-virtual:1:1:2086:2086] [20230106:231352:1673064832:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:debian-virtual:1:1:2086:2086] [20230106:231352:1673064832:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:debian-virtual:1:1:2086:2086] [20230106:231352:1673064832:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:debian-virtual:1:1:2086:2086] [20230106:231352:1673064832:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:debian-virtual:1:1:2086:2086] [20230106:231352:1673064832:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:debian-virtual:1:1:2086:2086] [20230106:231352:1673064832:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:debian-virtual:1:1:2086:2086] [20230106:231352:1673064832:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:debian-virtual:1:1:2086:2086] [20230106:231352:1673064832:0.001] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  23:14:22 STEP 2544 H=2544:00 +CPU= 22.418
  23:14:34 STEP 2545 H=2545:00 +CPU= 11.742
  23:14:46 STEP 2546 H=2546:00 +CPU= 12.153
  23:15:11 STEP 2547 H=2547:00 +CPU= 21.555
  23:15:27 STEP 2548 H=2548:00 +CPU= 13.486
  23:15:43 STEP 2549 H=2549:00 +CPU= 14.453
  23:16:06 STEP 2550 H=2550:00 +CPU= 20.258
  23:16:20 STEP 2551 H=2551:00 +CPU= 12.309
  23:16:35 STEP 2552 H=2552:00 +CPU= 12.875
  23:16:56 STEP 2553 H=2553:00 +CPU= 18.750
  23:17:09 STEP 2554 H=2554:00 +CPU= 11.824
  23:17:23 STEP 2555 H=2555:00 +CPU= 12.006
  23:17:46 STEP 2556 H=2556:00 +CPU= 20.086
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMGGhq0f+002556
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMSHhq0f+002556
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMUAhq0f+002556
  23:17:59 STEP 2557 H=2557:00 +CPU= 11.633
  23:18:12 STEP 2558 H=2558:00 +CPU= 11.598
  23:18:33 STEP 2559 H=2559:00 +CPU= 19.164
  23:18:47 STEP 2560 H=2560:00 +CPU= 12.272
  23:19:00 STEP 2561 H=2561:00 +CPU= 11.622
  23:19:21 STEP 2562 H=2562:00 +CPU= 18.756
  23:19:34 STEP 2563 H=2563:00 +CPU= 11.508
  23:19:48 STEP 2564 H=2564:00 +CPU= 11.949
  23:20:09 STEP 2565 H=2565:00 +CPU= 18.375
  23:20:22 STEP 2566 H=2566:00 +CPU= 11.736
  23:20:35 STEP 2567 H=2567:00 +CPU= 11.734
  23:21:00 STEP 2568 H=2568:00 +CPU= 22.758
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMGGhq0f+002568
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMSHhq0f+002568
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMUAhq0f+002568
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMGGhq0f+002544
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMSHhq0f+002544
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMUAhq0f+002544
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMGGhq0f+002556
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMSHhq0f+002556
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMUAhq0f+002556
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0385_1986050100_123_955_12172029_1_r1382893106_106.zip
Uploading the intermediate file: upload_file_106.zip
  23:21:14 STEP 2569 H=2569:00 +CPU= 11.562
  23:21:27 STEP 2570 H=2570:00 +CPU= 11.567
  23:21:48 STEP 2571 H=2571:00 +CPU= 18.465
  23:22:00 STEP 2572 H=2572:00 +CPU= 11.643
  23:22:13 STEP 2573 H=2573:00 +CPU= 11.204
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:23:50 STEP 2574 H=2574:00 +CPU= 19.999
  00:24:11 STEP 2575 H=2575:00 +CPU= 17.718
  00:24:33 STEP 2576 H=2576:00 +CPU= 18.161
  00:25:08 STEP 2577 H=2577:00 +CPU= 27.408
  00:25:29 STEP 2578 H=2578:00 +CPU= 17.793
  00:25:51 STEP 2579 H=2579:00 +CPU= 17.701
  00:26:29 STEP 2580 H=2580:00 +CPU= 30.430
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMGGhq0f+002580
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMSHhq0f+002580
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMUAhq0f+002580
  00:26:51 STEP 2581 H=2581:00 +CPU= 17.916
  00:27:13 STEP 2582 H=2582:00 +CPU= 18.105
  00:27:47 STEP 2583 H=2583:00 +CPU= 27.524
  00:28:10 STEP 2584 H=2584:00 +CPU= 18.319
  00:28:32 STEP 2585 H=2585:00 +CPU= 17.495
  00:29:06 STEP 2586 H=2586:00 +CPU= 27.651
  00:29:27 STEP 2587 H=2587:00 +CPU= 17.460
  00:29:50 STEP 2588 H=2588:00 +CPU= 17.889
  00:30:23 STEP 2589 H=2589:00 +CPU= 27.374
  00:30:45 STEP 2590 H=2590:00 +CPU= 17.807
  00:31:07 STEP 2591 H=2591:00 +CPU= 17.932
  00:31:50 STEP 2592 H=2592:00 +CPU= 32.290
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMGGhq0f+002592
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMSHhq0f+002592
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMUAhq0f+002592
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMGGhq0f+002568
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMSHhq0f+002568
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMUAhq0f+002568
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMGGhq0f+002580
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMSHhq0f+002580
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMUAhq0f+002580
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0385_1986050100_123_955_12172029_1_r1382893106_107.zip
Uploading the intermediate file: upload_file_107.zip
  00:32:14 STEP 2593 H=2593:00 +CPU= 17.741
  00:32:38 STEP 2594 H=2594:00 +CPU= 18.097
  00:33:14 STEP 2595 H=2595:00 +CPU= 27.253
  00:33:38 STEP 2596 H=2596:00 +CPU= 18.039
  00:34:01 STEP 2597 H=2597:00 +CPU= 17.511
  00:34:37 STEP 2598 H=2598:00 +CPU= 27.801
  00:35:00 STEP 2599 H=2599:00 +CPU= 17.158
  00:35:24 STEP 2600 H=2600:00 +CPU= 17.663
  00:36:00 STEP 2601 H=2601:00 +CPU= 26.945
  00:36:22 STEP 2602 H=2602:00 +CPU= 17.546
  00:36:44 STEP 2603 H=2603:00 +CPU= 17.347
  00:37:24 STEP 2604 H=2604:00 +CPU= 29.891
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMGGhq0f+002604
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMSHhq0f+002604
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMUAhq0f+002604
  00:37:46 STEP 2605 H=2605:00 +CPU= 17.222
  00:38:09 STEP 2606 H=2606:00 +CPU= 17.926
  00:38:45 STEP 2607 H=2607:00 +CPU= 26.887
  00:39:09 STEP 2608 H=2608:00 +CPU= 17.898
  00:39:31 STEP 2609 H=2609:00 +CPU= 17.383
  00:40:09 STEP 2610 H=2610:00 +CPU= 27.607
  00:40:31 STEP 2611 H=2611:00 +CPU= 17.563
  00:40:56 STEP 2612 H=2612:00 +CPU= 17.798
  00:41:30 STEP 2613 H=2613:00 +CPU= 27.352
  00:41:54 STEP 2614 H=2614:00 +CPU= 17.724
  00:42:18 STEP 2615 H=2615:00 +CPU= 17.996
  00:42:58 STEP 2616 H=2616:00 +CPU= 32.005
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMGGhq0f+002616
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMSHhq0f+002616
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMUAhq0f+002616
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMGGhq0f+002592
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMSHhq0f+002592
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMUAhq0f+002592
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMGGhq0f+002604
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMSHhq0f+002604
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMUAhq0f+002604
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0385_1986050100_123_955_12172029_1_r1382893106_108.zip
Uploading the intermediate file: upload_file_108.zip
  00:43:23 STEP 2617 H=2617:00 +CPU= 17.617
  00:43:48 STEP 2618 H=2618:00 +CPU= 17.584
  00:44:23 STEP 2619 H=2619:00 +CPU= 26.910
  00:44:48 STEP 2620 H=2620:00 +CPU= 18.187
  00:45:11 STEP 2621 H=2621:00 +CPU= 16.917
  00:45:48 STEP 2622 H=2622:00 +CPU= 27.653
  00:46:11 STEP 2623 H=2623:00 +CPU= 17.326
  00:46:34 STEP 2624 H=2624:00 +CPU= 17.881
  00:47:10 STEP 2625 H=2625:00 +CPU= 26.815
  00:47:33 STEP 2626 H=2626:00 +CPU= 18.089
  00:47:56 STEP 2627 H=2627:00 +CPU= 17.365
  00:48:36 STEP 2628 H=2628:00 +CPU= 29.862
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMGGhq0f+002628
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMSHhq0f+002628
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMUAhq0f+002628
  00:48:59 STEP 2629 H=2629:00 +CPU= 17.512
  00:49:23 STEP 2630 H=2630:00 +CPU= 17.583
  00:49:59 STEP 2631 H=2631:00 +CPU= 27.494
  00:50:21 STEP 2632 H=2632:00 +CPU= 17.708
  00:50:44 STEP 2633 H=2633:00 +CPU= 17.360
  00:51:20 STEP 2634 H=2634:00 +CPU= 27.748
  00:51:42 STEP 2635 H=2635:00 +CPU= 17.298
  00:52:06 STEP 2636 H=2636:00 +CPU= 17.703
  00:52:41 STEP 2637 H=2637:00 +CPU= 27.343
  00:53:06 STEP 2638 H=2638:00 +CPU= 17.822
  00:53:30 STEP 2639 H=2639:00 +CPU= 17.761
  00:54:14 STEP 2640 H=2640:00 +CPU= 32.672
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMGGhq0f+002640
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMSHhq0f+002640
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMUAhq0f+002640
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMGGhq0f+002616
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMSHhq0f+002616
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMUAhq0f+002616
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMGGhq0f+002628
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMSHhq0f+002628
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMUAhq0f+002628
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0385_1986050100_123_955_12172029_1_r1382893106_109.zip
Uploading the intermediate file: upload_file_109.zip
  00:54:38 STEP 2641 H=2641:00 +CPU= 17.240
  00:55:00 STEP 2642 H=2642:00 +CPU= 17.655
  00:55:36 STEP 2643 H=2643:00 +CPU= 27.072
  00:56:00 STEP 2644 H=2644:00 +CPU= 18.198
  00:56:23 STEP 2645 H=2645:00 +CPU= 17.153
  00:57:00 STEP 2646 H=2646:00 +CPU= 27.990
  00:57:23 STEP 2647 H=2647:00 +CPU= 17.226
  00:57:46 STEP 2648 H=2648:00 +CPU= 18.079
  00:58:22 STEP 2649 H=2649:00 +CPU= 27.191
  00:58:45 STEP 2650 H=2650:00 +CPU= 17.748
  00:59:08 STEP 2651 H=2651:00 +CPU= 16.964
  00:59:47 STEP 2652 H=2652:00 +CPU= 30.725
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMGGhq0f+002652
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMSHhq0f+002652
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMUAhq0f+002652
  01:00:10 STEP 2653 H=2653:00 +CPU= 17.377
  01:00:33 STEP 2654 H=2654:00 +CPU= 17.816
  01:01:09 STEP 2655 H=2655:00 +CPU= 27.093
  01:01:33 STEP 2656 H=2656:00 +CPU= 18.067
  01:01:55 STEP 2657 H=2657:00 +CPU= 17.259
  01:02:32 STEP 2658 H=2658:00 +CPU= 28.032
  01:02:56 STEP 2659 H=2659:00 +CPU= 17.124
  01:03:19 STEP 2660 H=2660:00 +CPU= 17.515
  01:03:56 STEP 2661 H=2661:00 +CPU= 27.415
  01:04:19 STEP 2662 H=2662:00 +CPU= 17.700
  01:04:42 STEP 2663 H=2663:00 +CPU= 18.031
  01:05:27 STEP 2664 H=2664:00 +CPU= 32.874
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMGGhq0f+002664
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMSHhq0f+002664
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMUAhq0f+002664
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMGGhq0f+002640
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMSHhq0f+002640
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMUAhq0f+002640
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMGGhq0f+002652
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMSHhq0f+002652
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMUAhq0f+002652
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0385_1986050100_123_955_12172029_1_r1382893106_110.zip
Uploading the intermediate file: upload_file_110.zip
  01:05:49 STEP 2665 H=2665:00 +CPU= 16.982
  01:06:13 STEP 2666 H=2666:00 +CPU= 17.933
  01:06:49 STEP 2667 H=2667:00 +CPU= 27.318
  01:07:12 STEP 2668 H=2668:00 +CPU= 17.557
  01:07:34 STEP 2669 H=2669:00 +CPU= 17.313
  01:08:11 STEP 2670 H=2670:00 +CPU= 27.632
  01:08:33 STEP 2671 H=2671:00 +CPU= 17.471
  01:08:57 STEP 2672 H=2672:00 +CPU= 17.815
  01:09:32 STEP 2673 H=2673:00 +CPU= 27.164
  01:09:57 STEP 2674 H=2674:00 +CPU= 18.301
  01:10:21 STEP 2675 H=2675:00 +CPU= 17.346
  01:11:00 STEP 2676 H=2676:00 +CPU= 30.210
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMGGhq0f+002676
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMSHhq0f+002676
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMUAhq0f+002676
  01:11:24 STEP 2677 H=2677:00 +CPU= 17.410
  01:11:48 STEP 2678 H=2678:00 +CPU= 17.900
  01:12:23 STEP 2679 H=2679:00 +CPU= 27.338
  01:12:47 STEP 2680 H=2680:00 +CPU= 17.785
  01:13:09 STEP 2681 H=2681:00 +CPU= 17.301
  01:13:47 STEP 2682 H=2682:00 +CPU= 27.630
  01:14:09 STEP 2683 H=2683:00 +CPU= 17.337
  01:14:33 STEP 2684 H=2684:00 +CPU= 18.142
  01:15:08 STEP 2685 H=2685:00 +CPU= 27.000
  01:15:31 STEP 2686 H=2686:00 +CPU= 17.758
  01:15:54 STEP 2687 H=2687:00 +CPU= 17.698
  01:16:37 STEP 2688 H=2688:00 +CPU= 32.291
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMGGhq0f+002688
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMSHhq0f+002688
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMUAhq0f+002688
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMGGhq0f+002664
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMSHhq0f+002664
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMUAhq0f+002664
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMGGhq0f+002676
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMSHhq0f+002676
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMUAhq0f+002676
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0385_1986050100_123_955_12172029_1_r1382893106_111.zip
Uploading the intermediate file: upload_file_111.zip
  01:16:59 STEP 2689 H=2689:00 +CPU= 17.514
  01:17:21 STEP 2690 H=2690:00 +CPU= 17.672
  01:17:57 STEP 2691 H=2691:00 +CPU= 27.758
  01:18:20 STEP 2692 H=2692:00 +CPU= 18.167
  01:18:43 STEP 2693 H=2693:00 +CPU= 17.148
  01:19:19 STEP 2694 H=2694:00 +CPU= 28.294
  01:19:41 STEP 2695 H=2695:00 +CPU= 17.004
  01:20:02 STEP 2696 H=2696:00 +CPU= 17.285
  01:20:37 STEP 2697 H=2697:00 +CPU= 27.375
  01:20:59 STEP 2698 H=2698:00 +CPU= 17.987
  01:21:23 STEP 2699 H=2699:00 +CPU= 17.128
  01:22:01 STEP 2700 H=2700:00 +CPU= 30.793
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMGGhq0f+002700
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMSHhq0f+002700
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMUAhq0f+002700
  01:22:24 STEP 2701 H=2701:00 +CPU= 17.913
  01:22:47 STEP 2702 H=2702:00 +CPU= 17.829
  01:23:23 STEP 2703 H=2703:00 +CPU= 27.937
  01:23:45 STEP 2704 H=2704:00 +CPU= 17.838
  01:24:07 STEP 2705 H=2705:00 +CPU= 17.422
  01:24:42 STEP 2706 H=2706:00 +CPU= 28.399
  01:25:06 STEP 2707 H=2707:00 +CPU= 17.894
  01:25:29 STEP 2708 H=2708:00 +CPU= 18.036
  01:26:05 STEP 2709 H=2709:00 +CPU= 27.212
  01:26:27 STEP 2710 H=2710:00 +CPU= 17.642
  01:26:50 STEP 2711 H=2711:00 +CPU= 17.606
  01:27:35 STEP 2712 H=2712:00 +CPU= 32.607
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMGGhq0f+002712
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMSHhq0f+002712
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMUAhq0f+002712
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMGGhq0f+002688
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMSHhq0f+002688
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMUAhq0f+002688
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMGGhq0f+002700
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMSHhq0f+002700
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMUAhq0f+002700
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0385_1986050100_123_955_12172029_1_r1382893106_112.zip
Uploading the intermediate file: upload_file_112.zip
  01:27:59 STEP 2713 H=2713:00 +CPU= 17.743
  01:28:21 STEP 2714 H=2714:00 +CPU= 17.871
  01:28:58 STEP 2715 H=2715:00 +CPU= 27.337
  01:29:22 STEP 2716 H=2716:00 +CPU= 17.911
  01:29:45 STEP 2717 H=2717:00 +CPU= 17.318
  01:30:22 STEP 2718 H=2718:00 +CPU= 28.352
  01:30:45 STEP 2719 H=2719:00 +CPU= 17.833
  01:31:10 STEP 2720 H=2720:00 +CPU= 18.282
  01:31:46 STEP 2721 H=2721:00 +CPU= 27.384
  01:32:10 STEP 2722 H=2722:00 +CPU= 17.749
  01:32:32 STEP 2723 H=2723:00 +CPU= 17.190
  01:33:12 STEP 2724 H=2724:00 +CPU= 30.923
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMGGhq0f+002724
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMSHhq0f+002724
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMUAhq0f+002724
  01:33:36 STEP 2725 H=2725:00 +CPU= 17.364
  01:33:59 STEP 2726 H=2726:00 +CPU= 17.921
  01:34:35 STEP 2727 H=2727:00 +CPU= 27.625
  01:34:59 STEP 2728 H=2728:00 +CPU= 17.761
  01:35:21 STEP 2729 H=2729:00 +CPU= 17.394
  01:35:59 STEP 2730 H=2730:00 +CPU= 27.743
  01:36:21 STEP 2731 H=2731:00 +CPU= 17.205
  01:36:41 STEP 2732 H=2732:00 +CPU= 15.758
  01:37:11 STEP 2733 H=2733:00 +CPU= 24.708
  01:37:30 STEP 2734 H=2734:00 +CPU= 16.402
  01:37:50 STEP 2735 H=2735:00 +CPU= 16.706
  01:38:24 STEP 2736 H=2736:00 +CPU= 27.656
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMGGhq0f+002736
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMSHhq0f+002736
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMUAhq0f+002736
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMGGhq0f+002712
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMSHhq0f+002712
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMUAhq0f+002712
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMGGhq0f+002724
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMSHhq0f+002724
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMUAhq0f+002724
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0385_1986050100_123_955_12172029_1_r1382893106_113.zip
Uploading the intermediate file: upload_file_113.zip
  01:38:42 STEP 2737 H=2737:00 +CPU= 15.260
  01:38:59 STEP 2738 H=2738:00 +CPU= 14.978
  01:39:26 STEP 2739 H=2739:00 +CPU= 23.359
  01:39:44 STEP 2740 H=2740:00 +CPU= 14.964
  01:40:01 STEP 2741 H=2741:00 +CPU= 15.182
  01:40:29 STEP 2742 H=2742:00 +CPU= 23.985
  01:40:45 STEP 2743 H=2743:00 +CPU= 13.783
  01:40:59 STEP 2744 H=2744:00 +CPU= 12.778
  01:41:26 STEP 2745 H=2745:00 +CPU= 23.267
  01:41:42 STEP 2746 H=2746:00 +CPU= 14.113
  01:41:57 STEP 2747 H=2747:00 +CPU= 13.346
  01:42:26 STEP 2748 H=2748:00 +CPU= 24.123
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMGGhq0f+002748
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMSHhq0f+002748
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMUAhq0f+002748
  01:42:41 STEP 2749 H=2749:00 +CPU= 13.358
  01:42:59 STEP 2750 H=2750:00 +CPU= 15.625
  01:43:23 STEP 2751 H=2751:00 +CPU= 20.454
  01:43:36 STEP 2752 H=2752:00 +CPU= 11.868
  01:43:50 STEP 2753 H=2753:00 +CPU= 11.745
  01:44:12 STEP 2754 H=2754:00 +CPU= 19.443
  01:44:26 STEP 2755 H=2755:00 +CPU= 12.339
  01:44:41 STEP 2756 H=2756:00 +CPU= 12.741
  01:45:04 STEP 2757 H=2757:00 +CPU= 20.702
  01:45:18 STEP 2758 H=2758:00 +CPU= 12.384
  01:45:33 STEP 2759 H=2759:00 +CPU= 12.665
  01:45:56 STEP 2760 H=2760:00 +CPU= 21.914
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMGGhq0f+002760
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMSHhq0f+002760
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMUAhq0f+002760
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMGGhq0f+002736
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMSHhq0f+002736
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMUAhq0f+002736
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMGGhq0f+002748
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMSHhq0f+002748
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMUAhq0f+002748
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0385_1986050100_123_955_12172029_1_r1382893106_114.zip
Uploading the intermediate file: upload_file_114.zip
  01:46:09 STEP 2761 H=2761:00 +CPU= 11.007
  01:46:21 STEP 2762 H=2762:00 +CPU= 11.314
  01:46:41 STEP 2763 H=2763:00 +CPU= 17.808
  01:46:54 STEP 2764 H=2764:00 +CPU= 11.439
  01:47:07 STEP 2765 H=2765:00 +CPU= 11.187
  01:47:28 STEP 2766 H=2766:00 +CPU= 18.848
  01:47:42 STEP 2767 H=2767:00 +CPU= 11.771
  01:47:55 STEP 2768 H=2768:00 +CPU= 11.817
  01:48:16 STEP 2769 H=2769:00 +CPU= 18.693
  01:48:30 STEP 2770 H=2770:00 +CPU= 12.135
  01:48:43 STEP 2771 H=2771:00 +CPU= 11.840
  01:49:06 STEP 2772 H=2772:00 +CPU= 20.575
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMGGhq0f+002772
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMSHhq0f+002772
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMUAhq0f+002772
  01:49:22 STEP 2773 H=2773:00 +CPU= 14.083
  01:49:40 STEP 2774 H=2774:00 +CPU= 15.196
  01:50:05 STEP 2775 H=2775:00 +CPU= 23.973
  01:50:23 STEP 2776 H=2776:00 +CPU= 15.885
  01:50:40 STEP 2777 H=2777:00 +CPU= 14.197
  01:51:09 STEP 2778 H=2778:00 +CPU= 25.146
  01:51:22 STEP 2779 H=2779:00 +CPU= 13.129
  01:51:38 STEP 2780 H=2780:00 +CPU= 14.108
  01:52:05 STEP 2781 H=2781:00 +CPU= 23.734
  01:52:22 STEP 2782 H=2782:00 +CPU= 14.418
  01:52:40 STEP 2783 H=2783:00 +CPU= 15.998
  01:53:15 STEP 2784 H=2784:00 +CPU= 29.869
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMGGhq0f+002784
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMSHhq0f+002784
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMUAhq0f+002784
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMGGhq0f+002760
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMSHhq0f+002760
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMUAhq0f+002760
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMGGhq0f+002772
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMSHhq0f+002772
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMUAhq0f+002772
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0385_1986050100_123_955_12172029_1_r1382893106_115.zip
Uploading the intermediate file: upload_file_115.zip
  01:53:33 STEP 2785 H=2785:00 +CPU= 15.456
Uploading trickle at timestep: 10022400
  01:53:49 STEP 2786 H=2786:00 +CPU= 14.483
  01:54:16 STEP 2787 H=2787:00 +CPU= 23.074
  01:54:36 STEP 2788 H=2788:00 +CPU= 17.279
  01:54:53 STEP 2789 H=2789:00 +CPU= 14.960
  01:55:24 STEP 2790 H=2790:00 +CPU= 25.934
  01:55:47 STEP 2791 H=2791:00 +CPU= 18.876
  01:56:10 STEP 2792 H=2792:00 +CPU= 18.904
  01:56:46 STEP 2793 H=2793:00 +CPU= 29.114
  01:57:09 STEP 2794 H=2794:00 +CPU= 18.761
  01:57:32 STEP 2795 H=2795:00 +CPU= 18.651
  01:58:12 STEP 2796 H=2796:00 +CPU= 32.288
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMGGhq0f+002796
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMSHhq0f+002796
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMUAhq0f+002796
  01:58:35 STEP 2797 H=2797:00 +CPU= 19.097
  01:58:59 STEP 2798 H=2798:00 +CPU= 18.863
  01:59:36 STEP 2799 H=2799:00 +CPU= 29.161
  01:59:59 STEP 2800 H=2800:00 +CPU= 18.931
  02:00:22 STEP 2801 H=2801:00 +CPU= 18.267
  02:00:58 STEP 2802 H=2802:00 +CPU= 28.948
  02:01:21 STEP 2803 H=2803:00 +CPU= 18.588
  02:01:43 STEP 2804 H=2804:00 +CPU= 18.213
  02:02:19 STEP 2805 H=2805:00 +CPU= 28.596
  02:02:43 STEP 2806 H=2806:00 +CPU= 19.212
  02:03:06 STEP 2807 H=2807:00 +CPU= 19.048
  02:03:49 STEP 2808 H=2808:00 +CPU= 33.487
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMGGhq0f+002808
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMSHhq0f+002808
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMUAhq0f+002808
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMGGhq0f+002784
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMSHhq0f+002784
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMUAhq0f+002784
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMGGhq0f+002796
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMSHhq0f+002796
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMUAhq0f+002796
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0385_1986050100_123_955_12172029_1_r1382893106_116.zip
Uploading the intermediate file: upload_file_116.zip
  02:04:12 STEP 2809 H=2809:00 +CPU= 18.362
  02:04:34 STEP 2810 H=2810:00 +CPU= 18.533
  02:05:10 STEP 2811 H=2811:00 +CPU= 29.030
  02:05:34 STEP 2812 H=2812:00 +CPU= 19.251
  02:05:56 STEP 2813 H=2813:00 +CPU= 18.038
  02:06:33 STEP 2814 H=2814:00 +CPU= 29.319
  02:06:55 STEP 2815 H=2815:00 +CPU= 18.106
  02:07:19 STEP 2816 H=2816:00 +CPU= 18.631
  02:07:54 STEP 2817 H=2817:00 +CPU= 28.819
  02:08:19 STEP 2818 H=2818:00 +CPU= 19.895
  02:08:42 STEP 2819 H=2819:00 +CPU= 18.649
  02:09:22 STEP 2820 H=2820:00 +CPU= 32.347
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMGGhq0f+002820
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMSHhq0f+002820
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMUAhq0f+002820
  02:09:46 STEP 2821 H=2821:00 +CPU= 18.789
  02:10:10 STEP 2822 H=2822:00 +CPU= 19.271
  02:10:46 STEP 2823 H=2823:00 +CPU= 28.639
  02:11:10 STEP 2824 H=2824:00 +CPU= 19.184
  02:11:33 STEP 2825 H=2825:00 +CPU= 18.489
  02:12:09 STEP 2826 H=2826:00 +CPU= 29.410
  02:12:33 STEP 2827 H=2827:00 +CPU= 18.395
  02:12:56 STEP 2828 H=2828:00 +CPU= 19.161
  02:13:33 STEP 2829 H=2829:00 +CPU= 28.820
  02:13:57 STEP 2830 H=2830:00 +CPU= 19.613
  02:14:21 STEP 2831 H=2831:00 +CPU= 19.419
  02:15:03 STEP 2832 H=2832:00 +CPU= 34.087
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMGGhq0f+002832
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMSHhq0f+002832
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMUAhq0f+002832
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMGGhq0f+002808
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMSHhq0f+002808
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMUAhq0f+002808
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMGGhq0f+002820
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMSHhq0f+002820
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMUAhq0f+002820
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0385_1986050100_123_955_12172029_1_r1382893106_117.zip
Uploading the intermediate file: upload_file_117.zip
  02:15:27 STEP 2833 H=2833:00 +CPU= 18.600
  02:15:52 STEP 2834 H=2834:00 +CPU= 19.624
  02:16:27 STEP 2835 H=2835:00 +CPU= 29.038
  02:16:51 STEP 2836 H=2836:00 +CPU= 19.191
  02:17:14 STEP 2837 H=2837:00 +CPU= 18.443
  02:17:51 STEP 2838 H=2838:00 +CPU= 29.844
  02:18:14 STEP 2839 H=2839:00 +CPU= 18.716
  02:18:38 STEP 2840 H=2840:00 +CPU= 19.110
  02:19:13 STEP 2841 H=2841:00 +CPU= 28.743
  02:19:37 STEP 2842 H=2842:00 +CPU= 18.977
  02:20:00 STEP 2843 H=2843:00 +CPU= 18.484
  02:20:40 STEP 2844 H=2844:00 +CPU= 32.187
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMGGhq0f+002844
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMSHhq0f+002844
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMUAhq0f+002844
  02:21:03 STEP 2845 H=2845:00 +CPU= 18.616
  02:21:26 STEP 2846 H=2846:00 +CPU= 18.709
  02:22:03 STEP 2847 H=2847:00 +CPU= 29.042
  02:22:26 STEP 2848 H=2848:00 +CPU= 18.681
  02:22:48 STEP 2849 H=2849:00 +CPU= 18.022
  02:23:24 STEP 2850 H=2850:00 +CPU= 29.425
  02:23:46 STEP 2851 H=2851:00 +CPU= 18.754
  02:24:10 STEP 2852 H=2852:00 +CPU= 18.945
  02:24:45 STEP 2853 H=2853:00 +CPU= 28.355
  02:25:09 STEP 2854 H=2854:00 +CPU= 18.774
  02:25:32 STEP 2855 H=2855:00 +CPU= 18.923
  02:26:14 STEP 2856 H=2856:00 +CPU= 33.766
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMGGhq0f+002856
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMSHhq0f+002856
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMUAhq0f+002856
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMGGhq0f+002832
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMSHhq0f+002832
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMUAhq0f+002832
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMGGhq0f+002844
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMSHhq0f+002844
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMUAhq0f+002844
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0385_1986050100_123_955_12172029_1_r1382893106_118.zip
Uploading the intermediate file: upload_file_118.zip
  02:26:37 STEP 2857 H=2857:00 +CPU= 18.483
  02:27:01 STEP 2858 H=2858:00 +CPU= 18.917
  02:27:36 STEP 2859 H=2859:00 +CPU= 28.732
  02:27:59 STEP 2860 H=2860:00 +CPU= 18.396
  02:28:22 STEP 2861 H=2861:00 +CPU= 18.170
  02:28:58 STEP 2862 H=2862:00 +CPU= 29.544
  02:29:21 STEP 2863 H=2863:00 +CPU= 18.430
  02:29:45 STEP 2864 H=2864:00 +CPU= 18.857
  02:30:20 STEP 2865 H=2865:00 +CPU= 28.331
  02:30:43 STEP 2866 H=2866:00 +CPU= 19.326
  02:31:05 STEP 2867 H=2867:00 +CPU= 18.216
  02:31:45 STEP 2868 H=2868:00 +CPU= 31.852
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMGGhq0f+002868
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMSHhq0f+002868
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMUAhq0f+002868
  02:32:08 STEP 2869 H=2869:00 +CPU= 18.316
  02:32:31 STEP 2870 H=2870:00 +CPU= 18.996
  02:33:07 STEP 2871 H=2871:00 +CPU= 29.265
  02:33:32 STEP 2872 H=2872:00 +CPU= 19.186
  02:33:55 STEP 2873 H=2873:00 +CPU= 18.182
  02:34:31 STEP 2874 H=2874:00 +CPU= 28.897
  02:34:54 STEP 2875 H=2875:00 +CPU= 18.557
  02:35:16 STEP 2876 H=2876:00 +CPU= 18.710
  02:35:51 STEP 2877 H=2877:00 +CPU= 28.673
  02:36:15 STEP 2878 H=2878:00 +CPU= 19.096
  02:36:38 STEP 2879 H=2879:00 +CPU= 19.108
  02:37:19 STEP 2880 H=2880:00 +CPU= 34.007
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMGGhq0f+002880
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMSHhq0f+002880
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMUAhq0f+002880
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMGGhq0f+002856
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMSHhq0f+002856
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMUAhq0f+002856
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMGGhq0f+002868
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMSHhq0f+002868
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMUAhq0f+002868
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0385_1986050100_123_955_12172029_1_r1382893106_119.zip
Uploading the intermediate file: upload_file_119.zip
  02:37:42 STEP 2881 H=2881:00 +CPU= 18.254
  02:38:05 STEP 2882 H=2882:00 +CPU= 18.593
  02:38:41 STEP 2883 H=2883:00 +CPU= 28.331
  02:39:04 STEP 2884 H=2884:00 +CPU= 18.336
  02:39:27 STEP 2885 H=2885:00 +CPU= 18.029
  02:40:05 STEP 2886 H=2886:00 +CPU= 29.127
  02:40:27 STEP 2887 H=2887:00 +CPU= 18.110
  02:40:51 STEP 2888 H=2888:00 +CPU= 18.687
  02:41:27 STEP 2889 H=2889:00 +CPU= 28.488
  02:41:51 STEP 2890 H=2890:00 +CPU= 18.599
  02:42:14 STEP 2891 H=2891:00 +CPU= 18.336
  02:42:53 STEP 2892 H=2892:00 +CPU= 31.579
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMGGhq0f+002892
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMSHhq0f+002892
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMUAhq0f+002892
  02:43:15 STEP 2893 H=2893:00 +CPU= 18.197
  02:43:39 STEP 2894 H=2894:00 +CPU= 18.939
  02:44:15 STEP 2895 H=2895:00 +CPU= 28.825
  02:44:38 STEP 2896 H=2896:00 +CPU= 18.634
  02:45:02 STEP 2897 H=2897:00 +CPU= 18.226
  02:45:39 STEP 2898 H=2898:00 +CPU= 29.350
  02:46:02 STEP 2899 H=2899:00 +CPU= 18.187
  02:46:27 STEP 2900 H=2900:00 +CPU= 19.166
  02:47:03 STEP 2901 H=2901:00 +CPU= 28.667
  02:47:26 STEP 2902 H=2902:00 +CPU= 18.744
  02:47:51 STEP 2903 H=2903:00 +CPU= 19.187
  02:48:34 STEP 2904 H=2904:00 +CPU= 33.850
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMGGhq0f+002904
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMSHhq0f+002904
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMUAhq0f+002904
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMGGhq0f+002880
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMSHhq0f+002880
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMUAhq0f+002880
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMGGhq0f+002892
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMSHhq0f+002892
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMUAhq0f+002892
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0385_1986050100_123_955_12172029_1_r1382893106_120.zip
Uploading the intermediate file: upload_file_120.zip
  02:48:58 STEP 2905 H=2905:00 +CPU= 18.811
  02:49:22 STEP 2906 H=2906:00 +CPU= 19.456
  02:49:58 STEP 2907 H=2907:00 +CPU= 28.433
  02:50:22 STEP 2908 H=2908:00 +CPU= 18.937
  02:50:44 STEP 2909 H=2909:00 +CPU= 17.953
  02:51:21 STEP 2910 H=2910:00 +CPU= 29.454
  02:51:45 STEP 2911 H=2911:00 +CPU= 18.569
  02:52:08 STEP 2912 H=2912:00 +CPU= 18.306
  02:52:44 STEP 2913 H=2913:00 +CPU= 28.714
  02:53:08 STEP 2914 H=2914:00 +CPU= 19.056
  02:53:31 STEP 2915 H=2915:00 +CPU= 18.202
  02:54:12 STEP 2916 H=2916:00 +CPU= 32.022
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMGGhq0f+002916
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMSHhq0f+002916
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMUAhq0f+002916
  02:54:35 STEP 2917 H=2917:00 +CPU= 18.431
  02:54:59 STEP 2918 H=2918:00 +CPU= 18.701
  02:55:36 STEP 2919 H=2919:00 +CPU= 29.395
  02:56:00 STEP 2920 H=2920:00 +CPU= 18.816
  02:56:24 STEP 2921 H=2921:00 +CPU= 18.009
  02:57:01 STEP 2922 H=2922:00 +CPU= 29.315
  02:57:25 STEP 2923 H=2923:00 +CPU= 18.218
  02:57:48 STEP 2924 H=2924:00 +CPU= 19.030
  02:58:25 STEP 2925 H=2925:00 +CPU= 29.426
  02:58:50 STEP 2926 H=2926:00 +CPU= 19.692
  02:59:14 STEP 2927 H=2927:00 +CPU= 19.179
  02:59:56 STEP 2928 H=2928:00 +CPU= 34.271
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMGGhq0f+002928
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMSHhq0f+002928
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMUAhq0f+002928
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMGGhq0f+002904
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMSHhq0f+002904
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMUAhq0f+002904
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMGGhq0f+002916
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMSHhq0f+002916
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMUAhq0f+002916
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0385_1986050100_123_955_12172029_1_r1382893106_121.zip
Uploading the intermediate file: upload_file_121.zip
  03:00:20 STEP 2929 H=2929:00 +CPU= 18.790
  03:00:42 STEP 2930 H=2930:00 +CPU= 18.557
  03:01:19 STEP 2931 H=2931:00 +CPU= 29.030
  03:01:42 STEP 2932 H=2932:00 +CPU= 18.771
  03:02:04 STEP 2933 H=2933:00 +CPU= 17.793
  03:02:41 STEP 2934 H=2934:00 +CPU= 29.851
  03:03:03 STEP 2935 H=2935:00 +CPU= 18.427
  03:03:28 STEP 2936 H=2936:00 +CPU= 19.400
  03:04:04 STEP 2937 H=2937:00 +CPU= 29.030
  03:04:28 STEP 2938 H=2938:00 +CPU= 18.787
  03:04:50 STEP 2939 H=2939:00 +CPU= 18.143
  03:05:30 STEP 2940 H=2940:00 +CPU= 32.283
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMGGhq0f+002940
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMSHhq0f+002940
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMUAhq0f+002940
  03:05:53 STEP 2941 H=2941:00 +CPU= 18.377
  03:06:17 STEP 2942 H=2942:00 +CPU= 18.947
  03:06:53 STEP 2943 H=2943:00 +CPU= 29.424
  03:07:16 STEP 2944 H=2944:00 +CPU= 18.596
  03:07:38 STEP 2945 H=2945:00 +CPU= 18.019
  03:08:14 STEP 2946 H=2946:00 +CPU= 29.686
  03:08:36 STEP 2947 H=2947:00 +CPU= 18.121
  03:09:01 STEP 2948 H=2948:00 +CPU= 19.075
  03:09:37 STEP 2949 H=2949:00 +CPU= 28.797
  03:10:00 STEP 2950 H=2950:00 +CPU= 18.762
  03:10:24 STEP 2951 H=2951:00 +CPU= 19.262
  03:10:58 STEP 2952 H=2952:00 +CPU= 30.894
The child process terminated with status: 0
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMGGhq0f+002952
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMSHhq0f+002952
Moving to projects directory: /var/lib/boinc-client/slots/20/ICMUAhq0f+002952
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMGGhq0f+002928
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMGGhq0f+002940
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMSHhq0f+002940
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMUAhq0f+002940
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMSHhq0f+002952
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMUAhq0f+002928
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMSHhq0f+002928
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMUAhq0f+002952
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12172029/ICMGGhq0f+002952
Zipping up the final file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0385_1986050100_123_955_12172029_1_r1382893106_122.zip
Uploading the final file: upload_file_122.zip
Uploading trickle at timestep: 10623600
03:14:21 (1385): called boinc_finish(0)

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 May 2023 16:11:30 1536310 22285211 oifs_43r3_ps_0385_1986050100_123_955_12172029_1 10,623,600 71,323 0.0067
09 May 2023 16:11:30 1536310 22285211 oifs_43r3_ps_0385_1986050100_123_955_12172029_1 10,623,600 71,323 0.0067
09 May 2023 16:11:28 1536310 22285211 oifs_43r3_ps_0385_1986050100_123_955_12172029_1 10,022,400 67,579 0.0067
09 May 2023 16:11:08 1536310 22285211 oifs_43r3_ps_0385_1986050100_123_955_12172029_1 8,640,000 58,738 0.0068
09 May 2023 16:11:08 1536310 22285211 oifs_43r3_ps_0385_1986050100_123_955_12172029_1 7,776,000 51,663 0.0066
09 May 2023 16:11:08 1536310 22285211 oifs_43r3_ps_0385_1986050100_123_955_12172029_1 6,912,000 44,475 0.0064
09 May 2023 16:11:09 1536310 22285211 oifs_43r3_ps_0385_1986050100_123_955_12172029_1 6,048,000 37,079 0.0061
09 May 2023 16:11:08 1536310 22285211 oifs_43r3_ps_0385_1986050100_123_955_12172029_1 5,184,000 29,776 0.0057
09 May 2023 16:11:09 1536310 22285211 oifs_43r3_ps_0385_1986050100_123_955_12172029_1 4,320,000 22,484 0.0052
09 May 2023 16:11:09 1536310 22285211 oifs_43r3_ps_0385_1986050100_123_955_12172029_1 3,456,000 15,583 0.0045
09 May 2023 16:11:08 1536310 22285211 oifs_43r3_ps_0385_1986050100_123_955_12172029_1 2,592,000 11,660 0.0045
09 May 2023 16:11:09 1536310 22285211 oifs_43r3_ps_0385_1986050100_123_955_12172029_1 1,728,000 7,786 0.0045
09 May 2023 16:11:08 1536310 22285211 oifs_43r3_ps_0385_1986050100_123_955_12172029_1 864,000 3,906 0.0045


©2024 climateprediction.net