climateprediction.net home page
Task 22310195

Task 22310195

Name oifs_43r3_bl_a2gz_2016092300_15_991_12209990_0
Workunit 12209990
Created 13 Feb 2023, 17:54:34 UTC
Sent 14 Feb 2023, 0:23:59 UTC
Report deadline 15 Apr 2023, 0:23:59 UTC
Received 19 Feb 2023, 0:25:56 UTC
Server state Over
Outcome Success
Client state Done
Exit status 0 (0x00000000)
Computer ID 1536642
Run time 13 hours 19 min 41 sec
CPU time 13 hours 17 min 38 sec
Validate state Valid
Credit 1,813.14
Device peak FLOPS 4.94 GFLOPS
Application version OpenIFS 43r3 Baroclinic Lifecycle v1.11
x86_64-pc-linux-gnu
Peak working set size 5,066.90 MB
Peak swap size 5,930.78 MB
Peak disk usage 2,119.33 MB
Stderr
<core_client_version>7.16.6</core_client_version>
<![CDATA[
<stderr_txt>
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMGGh7zg+000972
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMSHh7zg+000972
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMUAh7zg+000972
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMGGh7zg+000984
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMSHh7zg+000984
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMUAh7zg+000984
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMGGh7zg+000996
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMSHh7zg+000996
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMUAh7zg+000996
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMGGh7zg+001008
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMSHh7zg+001008
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMUAh7zg+001008
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMGGh7zg+001020
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMSHh7zg+001020
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMUAh7zg+001020
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMGGh7zg+001032
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMSHh7zg+001032
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMUAh7zg+001032
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMGGh7zg+001044
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMSHh7zg+001044
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMUAh7zg+001044
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a2gz_2016092300_15_991_12209990_0_r627146782_10.zip
Uploading the intermediate file: upload_file_10.zip
  05:53:55 STEP 1057 H= 264:15 +CPU= 33.620
  05:54:28 STEP 1058 H= 264:30 +CPU= 32.584
  05:55:01 STEP 1059 H= 264:45 +CPU= 32.381
  05:55:32 STEP 1060 H= 265:00 +CPU= 31.518
  05:56:03 STEP 1061 H= 265:15 +CPU= 31.322
  05:56:36 STEP 1062 H= 265:30 +CPU= 32.624
  05:57:08 STEP 1063 H= 265:45 +CPU= 31.419
  05:57:41 STEP 1064 H= 266:00 +CPU= 33.723
  05:58:13 STEP 1065 H= 266:15 +CPU= 31.123
  05:58:43 STEP 1066 H= 266:30 +CPU= 30.540
  05:59:15 STEP 1067 H= 266:45 +CPU= 31.322
  05:59:54 STEP 1068 H= 267:00 +CPU= 39.779
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001068
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001068
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001068
  06:00:28 STEP 1069 H= 267:15 +CPU= 33.088
  06:00:59 STEP 1070 H= 267:30 +CPU= 31.922
[EC_DRHOOK:ruokavarasto.koirankoppi.com:1:1:1573332:1573332] [20230218:060109:1676692869:27581.106] [signal_drhook@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1538] Received signal#15 (SIGTERM) :: 5397MB (heap), 4897MB (maxrss), 0MB (maxstack), 0 (paging), nsigs = 1
[EC_DRHOOK:ruokavarasto.koirankoppi.com:1:1:1573332:1573332] [20230218:060109:1676692869:27581.374] [signal_drhook@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1542] Also activating Harakiri-alarm (SIGALRM=14) to expire after 500s elapsed to prevent hangs, nsigs = 1
[EC_DRHOOK:ruokavarasto.koirankoppi.com:1:1:1573332:1573332] [20230218:060109:1676692869:27581.374] [signal_drhook@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1544] Harakiri signal handler 'signal_harakiri' for signal#14 (SIGALRM) installed at 0x81f0c0 (old at (nil))
[EC_DRHOOK:ruokavarasto.koirankoppi.com:1:1:1573332:1573332] [20230218:060109:1676692869:27581.374] [signal_drhook@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1617] Signal#15 was caused by unrecognized si_code [memaddr=0x1], nsigs = 1
[EC_DRHOOK:ruokavarasto.koirankoppi.com:1:1:1573332:1573332] [20230218:060109:1676692869:27581.374] [signal_drhook@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1686] Starting DrHook backtrace for signal#15, nsigs = 1
[EC_DRHOOK:ruokavarasto.koirankoppi.com:1:1:1573332:1573332] [20230218:060109:1676692869:27581.375] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3843] 5397 MB (maxheap), 4897 MB (maxrss), 0 MB (maxstack)
[EC_DRHOOK:ruokavarasto.koirankoppi.com:1:1:1573332:1573332] [20230218:060109:1676692869:27581.395] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :  MASTER 
[EC_DRHOOK:ruokavarasto.koirankoppi.com:1:1:1573332:1573332] [20230218:060109:1676692869:27581.477] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :   CNT0 
[EC_DRHOOK:ruokavarasto.koirankoppi.com:1:1:1573332:1573332] [20230218:060109:1676692869:27581.511] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :    CNT1 
[EC_DRHOOK:ruokavarasto.koirankoppi.com:1:1:1573332:1573332] [20230218:060109:1676692869:27581.511] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :     CNT2 
[EC_DRHOOK:ruokavarasto.koirankoppi.com:1:1:1573332:1573332] [20230218:060109:1676692869:27581.519] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :      CNT3 
[EC_DRHOOK:ruokavarasto.koirankoppi.com:1:1:1573332:1573332] [20230218:060109:1676692869:27581.519] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :       CNT4 
[EC_DRHOOK:ruokavarasto.koirankoppi.com:1:1:1573332:1573332] [20230218:060109:1676692869:27581.519] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :        STEPO 
[EC_DRHOOK:ruokavarasto.koirankoppi.com:1:1:1573332:1573332] [20230218:060109:1676692869:27581.519] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :         SCAN2M 
[EC_DRHOOK:ruokavarasto.koirankoppi.com:1:1:1573332:1573332] [20230218:060109:1676692869:27581.519] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :          GP_MODEL_HEAP 
[EC_DRHOOK:ruokavarasto.koirankoppi.com:1:1:1573332:1573332] [20230218:060109:1676692869:27581.519] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :           GP_MODEL 
[EC_DRHOOK:ruokavarasto.koirankoppi.com:1:1:1573332:1573332] [20230218:060109:1676692869:27581.519] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :            CALL_SL_HEAP 
[EC_DRHOOK:ruokavarasto.koirankoppi.com:1:1:1573332:1573332] [20230218:060109:1676692869:27581.519] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :             CALL_SL 
[EC_DRHOOK:ruokavarasto.koirankoppi.com:1:1:1573332:1573332] [20230218:060109:1676692869:27581.519] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :              >OMP-CALL_SL     1        (1004) 
[EC_DRHOOK:ruokavarasto.koirankoppi.com:1:1:1573332:1573332] [20230218:060109:1676692869:27581.519] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :               LAPINEA 
[EC_DRHOOK:ruokavarasto.koirankoppi.com:1:1:1573332:1573332] [20230218:060109:1676692869:27581.519] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :                LARMES 
[EC_DRHOOK:ruokavarasto.koirankoppi.com:1:1:1573332:1573332] [20230218:060109:1676692869:27581.519] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :                 LARCINA 
[EC_DRHOOK:ruokavarasto.koirankoppi.com:1:1:1573332:1573332] [20230218:060109:1676692869:27581.519] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :                  LAITLI 
[EC_DRHOOK:ruokavarasto.koirankoppi.com:1:1:1573332:1573332] [20230218:060110:1676692870:27582.519] [signal_drhook@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1734] DrHook backtrace done for signal#15, nsigs = 1
[EC_DRHOOK:ruokavarasto.koirankoppi.com:1:1:1573332:1573332] [20230218:060110:1676692870:27582.519] [signal_drhook@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1785] Calling previous signal handler at 0x401140 for signal#15, nsigs = 1
forrtl: error (78): process killed (SIGTERM)
Image              PC                Routine            Line        Source             
oifs_43r3_model.e  000000000081FCF1  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001D8BB20  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000000FDDE67  laitli_                   157  laitli.F90
oifs_43r3_model.e  0000000001065984  larcina_                  405  larcina.F90
oifs_43r3_model.e  0000000001062038  larmes_                   603  larmes.F90
oifs_43r3_model.e  0000000001058F5D  lapinea_                  270  lapinea.F90
oifs_43r3_model.e  0000000000FF4E6C  call_sl_                  300  call_sl.F90
oifs_43r3_model.e  0000000001081A7D  call_sl_heap_              94  call_sl_heap.F90
oifs_43r3_model.e  0000000000E67648  gp_model_                 533  gp_model.F90
oifs_43r3_model.e  00000000015C1E62  gp_model_heap_             74  gp_model_heap.F90
oifs_43r3_model.e  00000000015AC40E  scan2m_                   535  scan2m.F90
oifs_43r3_model.e  00000000006620B4  stepo_                    327  stepo.F90
oifs_43r3_model.e  00000000004A0E7C  cnt4_                    1133  cnt4.F90
oifs_43r3_model.e  000000000049DC29  cnt3_                     267  cnt3.F90
oifs_43r3_model.e  000000000049CE72  cnt2_                      88  cnt2.F90
oifs_43r3_model.e  000000000049CD68  cnt1_                      92  cnt1.F90
oifs_43r3_model.e  00000000004078AD  cnt0_                     146  cnt0.F90
oifs_43r3_model.e  000000000040276F  MAIN__                     96  master.F90
oifs_43r3_model.e  000000000040270D  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001D8287A  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001D840D7  Unknown               Unknown  Unknown
oifs_43r3_model.e  00000000004025E5  Unknown               Unknown  Unknown
(argv0) ../../projects/climateprediction.net/oifs_43r3_bl_1.11_x86_64-pc-linux-gnu
(argv1) start_date: 2016092300
(argv2) exptid: h7zg
(argv3) unique_member_id: a2gz
(argv4) batchid: 991
(argv5) wuid: 12209990
(argv6) fclen: 15
(argv7) app_name: oifs_43r3_bl
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/0
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_bl
version: 1.11
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_app_1.11_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/0/oifs_43r3_bl_app_1.11_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/0/oifs_43r3_bl_app_1.11_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_a9e6878e22271b0da04527215a26bfa8 to: /var/lib/boinc-client/slots/0/oifs_43r3_bl_a2gz_2016092300_15_991_12209990.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/0/oifs_43r3_bl_a2gz_2016092300_15_991_12209990.zip
ic_ancil_file: ic_ancil_12209990
ifsdata_file: ifsdata_12209990
climate_data_file: clim_data_12209990
horiz_resolution: 159
vert_resolution: 91
grid_type: l_2
upload_interval: 96
utstep: 900
nfrres: restart dump frequency (steps) 48
Copying IC ancils from: ../../projects/climateprediction.net/jf_f7ecfd27aa1e68bbe7e8d19800e25c73 to: /var/lib/boinc-client/slots/0/ic_ancil_12209990.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/0/ic_ancil_12209990.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_adbfd9dac2e339413583e4b9b864ae9a to: /var/lib/boinc-client/slots/0/ifsdata/ifsdata_12209990.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/0/ifsdata/ifsdata_12209990.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/0/159l_2/clim_data_12209990.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/0/159l_2/clim_data_12209990.zip
Checking for progress XML file: /var/lib/boinc-client/slots/0/progress_file_12209990.xml
Opened progress file ok : /var/lib/boinc-client/slots/0/progress_file_12209990.xml
-- Model is restarting --
Adjusting last_iter, 1071, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/0/progress_file_12209990.xml
last_cpu_time: 35339
upload_file_number: 11
last_iter: 1057
last_upload: 950400
model_completed: 0
total_length_of_simulation: 1296000
result_base_name: oifs_43r3_bl_a2gz_2016092300_15_991_12209990_0_r627146782
The child process has been launched with process id: 1595596
Executing the command: /var/lib/boinc-client/slots/0/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:ruokavarasto.koirankoppi.com:1:1:1595596:1595596] [20230218:222127:1676751687:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2091] fp = 0x30e4a00
[EC_DRHOOK:ruokavarasto.koirankoppi.com:1:1:1595596:1595596] [20230218:222127:1676751687:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:ruokavarasto.koirankoppi.com:1:1:1595596:1595596] [20230218:222127:1676751687:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:ruokavarasto.koirankoppi.com:1:1:1595596:1595596] [20230218:222127:1676751687:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:ruokavarasto.koirankoppi.com:1:1:1595596:1595596] [20230218:222127:1676751687:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:ruokavarasto.koirankoppi.com:1:1:1595596:1595596] [20230218:222127:1676751687:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:ruokavarasto.koirankoppi.com:1:1:1595596:1595596] [20230218:222127:1676751687:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:ruokavarasto.koirankoppi.com:1:1:1595596:1595596] [20230218:222127:1676751687:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:ruokavarasto.koirankoppi.com:1:1:1595596:1595596] [20230218:222127:1676751687:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:ruokavarasto.koirankoppi.com:1:1:1595596:1595596] [20230218:222127:1676751687:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:ruokavarasto.koirankoppi.com:1:1:1595596:1595596] [20230218:222127:1676751687:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:ruokavarasto.koirankoppi.com:1:1:1595596:1595596] [20230218:222127:1676751687:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:ruokavarasto.koirankoppi.com:1:1:1595596:1595596] [20230218:222127:1676751687:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:ruokavarasto.koirankoppi.com:1:1:1595596:1595596] [20230218:222127:1676751687:0.000] [ignore_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:ruokavarasto.koirankoppi.com:1:1:1595596:1595596] [20230218:222127:1676751687:0.000] [restore_default_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:ruokavarasto.koirankoppi.com:1:1:1595596:1595596] [20230218:222127:1676751687:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:ruokavarasto.koirankoppi.com:1:1:1595596:1595596] [20230218:222127:1676751687:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f520 (old at (nil))
[EC_DRHOOK:ruokavarasto.koirankoppi.com:1:1:1595596:1595596] [20230218:222127:1676751687:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:ruokavarasto.koirankoppi.com:1:1:1595596:1595596] [20230218:222127:1676751687:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f520 (old at (nil))
[EC_DRHOOK:ruokavarasto.koirankoppi.com:1:1:1595596:1595596] [20230218:222127:1676751687:0.000] [trapfpe_treatment@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:ruokavarasto.koirankoppi.com:1:1:1595596:1595596] [20230218:222127:1676751687:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:ruokavarasto.koirankoppi.com:1:1:1595596:1595596] [20230218:222127:1676751687:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:ruokavarasto.koirankoppi.com:1:1:1595596:1595596] [20230218:222127:1676751687:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f520 (old at (nil))
[EC_DRHOOK:ruokavarasto.koirankoppi.com:1:1:1595596:1595596] [20230218:222127:1676751687:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:ruokavarasto.koirankoppi.com:1:1:1595596:1595596] [20230218:222127:1676751687:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:ruokavarasto.koirankoppi.com:1:1:1595596:1595596] [20230218:222127:1676751687:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:ruokavarasto.koirankoppi.com:1:1:1595596:1595596] [20230218:222127:1676751687:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f520 (old at (nil))
[EC_DRHOOK:ruokavarasto.koirankoppi.com:1:1:1595596:1595596] [20230218:222127:1676751687:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f520 (old at (nil))
[EC_DRHOOK:ruokavarasto.koirankoppi.com:1:1:1595596:1595596] [20230218:222127:1676751687:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f520 (old at (nil))
[EC_DRHOOK:ruokavarasto.koirankoppi.com:1:1:1595596:1595596] [20230218:222127:1676751687:0.000] [catch_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  22:21:49 STEP 1056 H= 264:00 +CPU= 18.379
  22:22:08 STEP 1057 H= 264:15 +CPU= 19.560
  22:22:32 STEP 1058 H= 264:30 +CPU= 23.938
  22:22:57 STEP 1059 H= 264:45 +CPU= 24.534
  22:23:27 STEP 1060 H= 265:00 +CPU= 29.993
  22:23:58 STEP 1061 H= 265:15 +CPU= 30.869
  22:24:29 STEP 1062 H= 265:30 +CPU= 31.318
  22:25:02 STEP 1063 H= 265:45 +CPU= 32.557
  22:25:34 STEP 1064 H= 266:00 +CPU= 32.289
  22:26:05 STEP 1065 H= 266:15 +CPU= 30.808
  22:26:37 STEP 1066 H= 266:30 +CPU= 31.929
  22:27:08 STEP 1067 H= 266:45 +CPU= 31.358
  22:27:48 STEP 1068 H= 267:00 +CPU= 39.455
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001068
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001068
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001068
  22:28:21 STEP 1069 H= 267:15 +CPU= 32.695
  22:28:52 STEP 1070 H= 267:30 +CPU= 31.436
  22:29:24 STEP 1071 H= 267:45 +CPU= 31.690
  22:29:57 STEP 1072 H= 268:00 +CPU= 32.858
  22:30:28 STEP 1073 H= 268:15 +CPU= 31.446
  22:30:59 STEP 1074 H= 268:30 +CPU= 31.019
  22:31:32 STEP 1075 H= 268:45 +CPU= 32.168
  22:32:04 STEP 1076 H= 269:00 +CPU= 32.725
  22:32:36 STEP 1077 H= 269:15 +CPU= 31.778
  22:33:07 STEP 1078 H= 269:30 +CPU= 30.904
  22:33:39 STEP 1079 H= 269:45 +CPU= 31.348
  22:34:19 STEP 1080 H= 270:00 +CPU= 40.586
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001080
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001080
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001080
  22:34:51 STEP 1081 H= 270:15 +CPU= 31.336
  22:35:21 STEP 1082 H= 270:30 +CPU= 30.737
  22:35:54 STEP 1083 H= 270:45 +CPU= 32.742
  22:36:27 STEP 1084 H= 271:00 +CPU= 32.410
  22:36:57 STEP 1085 H= 271:15 +CPU= 30.580
  22:37:31 STEP 1086 H= 271:30 +CPU= 33.269
  22:38:02 STEP 1087 H= 271:45 +CPU= 31.236
  22:38:34 STEP 1088 H= 272:00 +CPU= 32.198
  22:39:07 STEP 1089 H= 272:15 +CPU= 33.277
  22:39:40 STEP 1090 H= 272:30 +CPU= 32.572
  22:40:12 STEP 1091 H= 272:45 +CPU= 31.808
  22:40:53 STEP 1092 H= 273:00 +CPU= 41.308
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001092
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001092
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001092
  22:41:27 STEP 1093 H= 273:15 +CPU= 33.293
  22:41:59 STEP 1094 H= 273:30 +CPU= 32.380
  22:42:29 STEP 1095 H= 273:45 +CPU= 29.983
  22:43:02 STEP 1096 H= 274:00 +CPU= 32.997
  22:43:33 STEP 1097 H= 274:15 +CPU= 31.095
  22:44:06 STEP 1098 H= 274:30 +CPU= 32.635
  22:44:38 STEP 1099 H= 274:45 +CPU= 31.707
  22:45:10 STEP 1100 H= 275:00 +CPU= 32.161
  22:45:43 STEP 1101 H= 275:15 +CPU= 32.488
  22:46:13 STEP 1102 H= 275:30 +CPU= 30.208
  22:46:44 STEP 1103 H= 275:45 +CPU= 30.565
  22:47:29 STEP 1104 H= 276:00 +CPU= 44.998
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001104
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001104
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001104
  22:48:02 STEP 1105 H= 276:15 +CPU= 32.411
  22:48:32 STEP 1106 H= 276:30 +CPU= 30.720
  22:49:04 STEP 1107 H= 276:45 +CPU= 31.086
  22:49:35 STEP 1108 H= 277:00 +CPU= 31.639
  22:50:06 STEP 1109 H= 277:15 +CPU= 30.847
  22:50:37 STEP 1110 H= 277:30 +CPU= 31.077
  22:51:09 STEP 1111 H= 277:45 +CPU= 31.856
  22:51:40 STEP 1112 H= 278:00 +CPU= 30.684
  22:52:12 STEP 1113 H= 278:15 +CPU= 31.700
  22:52:44 STEP 1114 H= 278:30 +CPU= 31.564
  22:53:16 STEP 1115 H= 278:45 +CPU= 32.441
  22:53:55 STEP 1116 H= 279:00 +CPU= 39.201
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001116
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001116
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001116
  22:54:26 STEP 1117 H= 279:15 +CPU= 30.403
  22:54:58 STEP 1118 H= 279:30 +CPU= 32.391
  22:55:31 STEP 1119 H= 279:45 +CPU= 32.414
  22:56:03 STEP 1120 H= 280:00 +CPU= 32.130
  22:56:35 STEP 1121 H= 280:15 +CPU= 31.955
  22:57:07 STEP 1122 H= 280:30 +CPU= 31.917
  22:57:39 STEP 1123 H= 280:45 +CPU= 32.017
  22:58:12 STEP 1124 H= 281:00 +CPU= 33.020
  22:58:43 STEP 1125 H= 281:15 +CPU= 30.937
  22:59:14 STEP 1126 H= 281:30 +CPU= 31.212
  22:59:46 STEP 1127 H= 281:45 +CPU= 32.109
  23:00:27 STEP 1128 H= 282:00 +CPU= 40.633
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001128
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001128
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001128
  23:01:01 STEP 1129 H= 282:15 +CPU= 33.748
  23:01:32 STEP 1130 H= 282:30 +CPU= 30.923
  23:02:05 STEP 1131 H= 282:45 +CPU= 32.820
  23:02:36 STEP 1132 H= 283:00 +CPU= 31.448
  23:03:09 STEP 1133 H= 283:15 +CPU= 32.523
  23:03:42 STEP 1134 H= 283:30 +CPU= 32.832
  23:04:14 STEP 1135 H= 283:45 +CPU= 32.576
  23:04:47 STEP 1136 H= 284:00 +CPU= 32.596
  23:05:21 STEP 1137 H= 284:15 +CPU= 33.719
  23:05:51 STEP 1138 H= 284:30 +CPU= 29.945
  23:06:22 STEP 1139 H= 284:45 +CPU= 30.978
  23:07:00 STEP 1140 H= 285:00 +CPU= 38.362
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001140
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001140
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001140
  23:07:32 STEP 1141 H= 285:15 +CPU= 31.466
  23:08:04 STEP 1142 H= 285:30 +CPU= 32.265
  23:08:37 STEP 1143 H= 285:45 +CPU= 32.419
  23:09:10 STEP 1144 H= 286:00 +CPU= 32.922
  23:09:43 STEP 1145 H= 286:15 +CPU= 32.826
  23:10:15 STEP 1146 H= 286:30 +CPU= 31.911
  23:10:48 STEP 1147 H= 286:45 +CPU= 33.050
  23:11:20 STEP 1148 H= 287:00 +CPU= 32.248
  23:11:52 STEP 1149 H= 287:15 +CPU= 32.280
  23:12:24 STEP 1150 H= 287:30 +CPU= 32.196
  23:12:58 STEP 1151 H= 287:45 +CPU= 33.460
  23:13:48 STEP 1152 H= 288:00 +CPU= 44.278
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001152
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001152
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001152
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMGGh7zg+001056
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMSHh7zg+001056
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMUAh7zg+001056
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMGGh7zg+001068
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMSHh7zg+001068
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMUAh7zg+001068
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMGGh7zg+001080
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMSHh7zg+001080
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMUAh7zg+001080
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMGGh7zg+001092
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMSHh7zg+001092
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMUAh7zg+001092
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMGGh7zg+001104
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMSHh7zg+001104
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMUAh7zg+001104
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMGGh7zg+001116
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMSHh7zg+001116
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMUAh7zg+001116
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMGGh7zg+001128
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMSHh7zg+001128
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMUAh7zg+001128
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMGGh7zg+001140
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMSHh7zg+001140
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMUAh7zg+001140
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a2gz_2016092300_15_991_12209990_0_r627146782_11.zip
Uploading the intermediate file: upload_file_11.zip
  23:14:22 STEP 1153 H= 288:15 +CPU= 33.145
  23:14:53 STEP 1154 H= 288:30 +CPU= 31.092
  23:15:25 STEP 1155 H= 288:45 +CPU= 31.832
  23:16:01 STEP 1156 H= 289:00 +CPU= 35.461
  23:16:33 STEP 1157 H= 289:15 +CPU= 32.136
  23:17:05 STEP 1158 H= 289:30 +CPU= 32.504
  23:17:38 STEP 1159 H= 289:45 +CPU= 32.337
  23:18:10 STEP 1160 H= 290:00 +CPU= 31.521
  23:18:42 STEP 1161 H= 290:15 +CPU= 32.616
  23:19:14 STEP 1162 H= 290:30 +CPU= 31.990
  23:19:47 STEP 1163 H= 290:45 +CPU= 33.013
  23:20:27 STEP 1164 H= 291:00 +CPU= 39.148
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001164
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001164
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001164
  23:20:59 STEP 1165 H= 291:15 +CPU= 32.520
  23:21:33 STEP 1166 H= 291:30 +CPU= 33.986
  23:22:04 STEP 1167 H= 291:45 +CPU= 30.164
  23:22:36 STEP 1168 H= 292:00 +CPU= 32.149
  23:23:07 STEP 1169 H= 292:15 +CPU= 30.704
  23:23:38 STEP 1170 H= 292:30 +CPU= 31.455
  23:24:10 STEP 1171 H= 292:45 +CPU= 31.370
  23:24:41 STEP 1172 H= 293:00 +CPU= 31.654
  23:25:13 STEP 1173 H= 293:15 +CPU= 31.151
  23:25:44 STEP 1174 H= 293:30 +CPU= 31.124
  23:26:15 STEP 1175 H= 293:45 +CPU= 31.524
  23:26:55 STEP 1176 H= 294:00 +CPU= 39.446
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001176
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001176
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001176
  23:27:26 STEP 1177 H= 294:15 +CPU= 31.289
  23:28:00 STEP 1178 H= 294:30 +CPU= 33.402
  23:28:33 STEP 1179 H= 294:45 +CPU= 33.309
  23:29:07 STEP 1180 H= 295:00 +CPU= 33.718
  23:29:38 STEP 1181 H= 295:15 +CPU= 31.426
  23:30:11 STEP 1182 H= 295:30 +CPU= 32.798
  23:30:42 STEP 1183 H= 295:45 +CPU= 31.112
  23:31:14 STEP 1184 H= 296:00 +CPU= 31.830
  23:31:44 STEP 1185 H= 296:15 +CPU= 30.125
  23:32:15 STEP 1186 H= 296:30 +CPU= 30.541
  23:32:47 STEP 1187 H= 296:45 +CPU= 31.941
  23:33:27 STEP 1188 H= 297:00 +CPU= 40.489
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001188
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001188
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001188
  23:34:00 STEP 1189 H= 297:15 +CPU= 32.116
  23:34:32 STEP 1190 H= 297:30 +CPU= 32.096
  23:35:03 STEP 1191 H= 297:45 +CPU= 31.113
  23:35:35 STEP 1192 H= 298:00 +CPU= 32.268
  23:36:07 STEP 1193 H= 298:15 +CPU= 31.891
  23:36:39 STEP 1194 H= 298:30 +CPU= 32.316
  23:37:12 STEP 1195 H= 298:45 +CPU= 32.121
  23:37:43 STEP 1196 H= 299:00 +CPU= 31.340
  23:38:17 STEP 1197 H= 299:15 +CPU= 33.930
  23:38:48 STEP 1198 H= 299:30 +CPU= 31.009
  23:39:19 STEP 1199 H= 299:45 +CPU= 31.279
  23:40:09 STEP 1200 H= 300:00 +CPU= 44.587
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001200
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001200
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001200
  23:40:40 STEP 1201 H= 300:15 +CPU= 31.209
  23:41:12 STEP 1202 H= 300:30 +CPU= 31.987
  23:41:43 STEP 1203 H= 300:45 +CPU= 30.691
  23:42:15 STEP 1204 H= 301:00 +CPU= 32.239
  23:42:48 STEP 1205 H= 301:15 +CPU= 32.316
  23:43:21 STEP 1206 H= 301:30 +CPU= 32.739
  23:43:53 STEP 1207 H= 301:45 +CPU= 32.305
  23:44:24 STEP 1208 H= 302:00 +CPU= 31.394
  23:44:58 STEP 1209 H= 302:15 +CPU= 33.259
  23:45:29 STEP 1210 H= 302:30 +CPU= 31.019
  23:46:00 STEP 1211 H= 302:45 +CPU= 31.498
  23:46:39 STEP 1212 H= 303:00 +CPU= 39.031
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001212
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001212
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001212
  23:47:10 STEP 1213 H= 303:15 +CPU= 30.672
  23:47:43 STEP 1214 H= 303:30 +CPU= 32.729
  23:48:15 STEP 1215 H= 303:45 +CPU= 31.746
  23:48:48 STEP 1216 H= 304:00 +CPU= 33.670
  23:49:20 STEP 1217 H= 304:15 +CPU= 31.818
  23:49:53 STEP 1218 H= 304:30 +CPU= 32.182
  23:50:25 STEP 1219 H= 304:45 +CPU= 31.863
  23:50:56 STEP 1220 H= 305:00 +CPU= 31.545
  23:51:28 STEP 1221 H= 305:15 +CPU= 32.051
  23:52:01 STEP 1222 H= 305:30 +CPU= 32.560
  23:52:33 STEP 1223 H= 305:45 +CPU= 32.421
  23:53:13 STEP 1224 H= 306:00 +CPU= 39.732
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001224
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001224
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001224
  23:53:45 STEP 1225 H= 306:15 +CPU= 31.490
  23:54:16 STEP 1226 H= 306:30 +CPU= 31.210
  23:54:48 STEP 1227 H= 306:45 +CPU= 32.391
  23:55:22 STEP 1228 H= 307:00 +CPU= 33.086
  23:55:52 STEP 1229 H= 307:15 +CPU= 30.180
  23:56:24 STEP 1230 H= 307:30 +CPU= 31.998
  23:56:57 STEP 1231 H= 307:45 +CPU= 32.634
  23:57:30 STEP 1232 H= 308:00 +CPU= 33.145
  23:58:02 STEP 1233 H= 308:15 +CPU= 32.032
  23:58:34 STEP 1234 H= 308:30 +CPU= 31.934
  23:59:05 STEP 1235 H= 308:45 +CPU= 31.320
  23:59:45 STEP 1236 H= 309:00 +CPU= 39.939
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001236
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001236
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001236
  00:00:16 STEP 1237 H= 309:15 +CPU= 30.834
  00:00:48 STEP 1238 H= 309:30 +CPU= 32.213
  00:01:20 STEP 1239 H= 309:45 +CPU= 31.527
  00:01:54 STEP 1240 H= 310:00 +CPU= 33.797
  00:02:28 STEP 1241 H= 310:15 +CPU= 33.348
  00:03:01 STEP 1242 H= 310:30 +CPU= 32.688
  00:03:33 STEP 1243 H= 310:45 +CPU= 31.618
  00:04:06 STEP 1244 H= 311:00 +CPU= 33.022
  00:04:38 STEP 1245 H= 311:15 +CPU= 32.778
  00:05:10 STEP 1246 H= 311:30 +CPU= 31.044
  00:05:40 STEP 1247 H= 311:45 +CPU= 30.804
  00:06:27 STEP 1248 H= 312:00 +CPU= 44.747
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001248
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001248
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001248
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMGGh7zg+001152
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMSHh7zg+001152
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMUAh7zg+001152
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMGGh7zg+001164
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMSHh7zg+001164
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMUAh7zg+001164
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMGGh7zg+001176
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMSHh7zg+001176
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMUAh7zg+001176
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMGGh7zg+001188
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMSHh7zg+001188
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMUAh7zg+001188
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMGGh7zg+001200
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMSHh7zg+001200
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMUAh7zg+001200
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMGGh7zg+001212
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMSHh7zg+001212
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMUAh7zg+001212
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMGGh7zg+001224
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMSHh7zg+001224
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMUAh7zg+001224
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMGGh7zg+001236
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMSHh7zg+001236
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMUAh7zg+001236
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a2gz_2016092300_15_991_12209990_0_r627146782_12.zip
Uploading the intermediate file: upload_file_12.zip
  00:07:02 STEP 1249 H= 312:15 +CPU= 34.737
  00:07:36 STEP 1250 H= 312:30 +CPU= 33.645
  00:08:10 STEP 1251 H= 312:45 +CPU= 33.527
  00:08:41 STEP 1252 H= 313:00 +CPU= 31.481
  00:09:14 STEP 1253 H= 313:15 +CPU= 32.369
  00:09:46 STEP 1254 H= 313:30 +CPU= 31.850
  00:10:16 STEP 1255 H= 313:45 +CPU= 30.654
  00:10:49 STEP 1256 H= 314:00 +CPU= 32.357
  00:11:22 STEP 1257 H= 314:15 +CPU= 32.882
  00:11:54 STEP 1258 H= 314:30 +CPU= 32.519
  00:12:26 STEP 1259 H= 314:45 +CPU= 31.454
  00:13:06 STEP 1260 H= 315:00 +CPU= 39.915
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001260
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001260
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001260
  00:13:38 STEP 1261 H= 315:15 +CPU= 32.050
  00:14:10 STEP 1262 H= 315:30 +CPU= 32.585
  00:14:43 STEP 1263 H= 315:45 +CPU= 32.725
  00:15:16 STEP 1264 H= 316:00 +CPU= 32.440
  00:15:48 STEP 1265 H= 316:15 +CPU= 32.156
  00:16:20 STEP 1266 H= 316:30 +CPU= 31.882
  00:16:53 STEP 1267 H= 316:45 +CPU= 32.912
  00:17:26 STEP 1268 H= 317:00 +CPU= 33.260
  00:17:59 STEP 1269 H= 317:15 +CPU= 32.255
  00:18:32 STEP 1270 H= 317:30 +CPU= 32.932
  00:19:05 STEP 1271 H= 317:45 +CPU= 33.244
  00:19:44 STEP 1272 H= 318:00 +CPU= 38.737
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001272
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001272
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001272
  00:20:16 STEP 1273 H= 318:15 +CPU= 32.057
  00:20:48 STEP 1274 H= 318:30 +CPU= 31.551
  00:21:20 STEP 1275 H= 318:45 +CPU= 31.989
  00:21:53 STEP 1276 H= 319:00 +CPU= 32.733
  00:22:24 STEP 1277 H= 319:15 +CPU= 31.402
  00:22:56 STEP 1278 H= 319:30 +CPU= 32.261
  00:23:29 STEP 1279 H= 319:45 +CPU= 32.218
  00:24:01 STEP 1280 H= 320:00 +CPU= 32.023
  00:24:33 STEP 1281 H= 320:15 +CPU= 32.052
  00:25:04 STEP 1282 H= 320:30 +CPU= 31.552
  00:25:36 STEP 1283 H= 320:45 +CPU= 31.547
  00:26:17 STEP 1284 H= 321:00 +CPU= 40.776
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001284
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001284
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001284
  00:26:49 STEP 1285 H= 321:15 +CPU= 32.139
  00:27:23 STEP 1286 H= 321:30 +CPU= 33.604
  00:27:55 STEP 1287 H= 321:45 +CPU= 32.028
  00:28:27 STEP 1288 H= 322:00 +CPU= 31.804
  00:28:58 STEP 1289 H= 322:15 +CPU= 31.319
  00:29:30 STEP 1290 H= 322:30 +CPU= 31.646
  00:30:02 STEP 1291 H= 322:45 +CPU= 32.614
  00:30:36 STEP 1292 H= 323:00 +CPU= 33.615
  00:31:09 STEP 1293 H= 323:15 +CPU= 32.574
  00:31:41 STEP 1294 H= 323:30 +CPU= 32.780
  00:32:13 STEP 1295 H= 323:45 +CPU= 31.519
  00:33:01 STEP 1296 H= 324:00 +CPU= 41.809
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001296
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001296
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001296
  00:33:34 STEP 1297 H= 324:15 +CPU= 32.484
  00:34:08 STEP 1298 H= 324:30 +CPU= 33.275
  00:34:40 STEP 1299 H= 324:45 +CPU= 31.933
  00:35:12 STEP 1300 H= 325:00 +CPU= 32.444
  00:35:44 STEP 1301 H= 325:15 +CPU= 31.645
  00:36:16 STEP 1302 H= 325:30 +CPU= 31.865
  00:36:49 STEP 1303 H= 325:45 +CPU= 32.867
  00:37:22 STEP 1304 H= 326:00 +CPU= 33.412
  00:37:54 STEP 1305 H= 326:15 +CPU= 32.226
  00:38:26 STEP 1306 H= 326:30 +CPU= 31.192
  00:38:55 STEP 1307 H= 326:45 +CPU= 29.760
  00:39:35 STEP 1308 H= 327:00 +CPU= 39.399
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001308
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001308
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001308
  00:40:07 STEP 1309 H= 327:15 +CPU= 32.259
  00:40:40 STEP 1310 H= 327:30 +CPU= 32.554
  00:41:12 STEP 1311 H= 327:45 +CPU= 31.702
  00:41:45 STEP 1312 H= 328:00 +CPU= 33.088
  00:42:16 STEP 1313 H= 328:15 +CPU= 31.708
  00:42:49 STEP 1314 H= 328:30 +CPU= 32.645
  00:43:20 STEP 1315 H= 328:45 +CPU= 31.269
  00:43:54 STEP 1316 H= 329:00 +CPU= 33.720
  00:44:25 STEP 1317 H= 329:15 +CPU= 31.084
  00:44:57 STEP 1318 H= 329:30 +CPU= 31.555
  00:45:27 STEP 1319 H= 329:45 +CPU= 30.207
  00:46:08 STEP 1320 H= 330:00 +CPU= 40.784
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001320
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001320
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001320
  00:46:39 STEP 1321 H= 330:15 +CPU= 30.737
  00:47:11 STEP 1322 H= 330:30 +CPU= 31.723
  00:47:42 STEP 1323 H= 330:45 +CPU= 31.432
  00:48:15 STEP 1324 H= 331:00 +CPU= 32.479
  00:48:46 STEP 1325 H= 331:15 +CPU= 31.264
  00:49:19 STEP 1326 H= 331:30 +CPU= 32.530
  00:49:49 STEP 1327 H= 331:45 +CPU= 30.365
  00:50:21 STEP 1328 H= 332:00 +CPU= 31.862
  00:50:53 STEP 1329 H= 332:15 +CPU= 31.626
  00:51:24 STEP 1330 H= 332:30 +CPU= 31.693
  00:51:57 STEP 1331 H= 332:45 +CPU= 32.810
  00:52:37 STEP 1332 H= 333:00 +CPU= 40.108
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001332
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001332
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001332
  00:53:09 STEP 1333 H= 333:15 +CPU= 31.105
  00:53:41 STEP 1334 H= 333:30 +CPU= 32.558
  00:54:12 STEP 1335 H= 333:45 +CPU= 31.048
  00:54:45 STEP 1336 H= 334:00 +CPU= 32.411
  00:55:17 STEP 1337 H= 334:15 +CPU= 32.462
  00:55:50 STEP 1338 H= 334:30 +CPU= 32.356
  00:56:21 STEP 1339 H= 334:45 +CPU= 30.990
  00:56:53 STEP 1340 H= 335:00 +CPU= 32.303
  00:57:25 STEP 1341 H= 335:15 +CPU= 32.048
  00:57:58 STEP 1342 H= 335:30 +CPU= 32.387
  00:58:29 STEP 1343 H= 335:45 +CPU= 30.917
  00:59:14 STEP 1344 H= 336:00 +CPU= 41.497
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001344
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001344
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001344
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMGGh7zg+001248
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMSHh7zg+001248
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMUAh7zg+001248
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMGGh7zg+001260
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMSHh7zg+001260
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMUAh7zg+001260
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMGGh7zg+001272
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMSHh7zg+001272
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMUAh7zg+001272
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMGGh7zg+001284
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMSHh7zg+001284
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMUAh7zg+001284
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMGGh7zg+001296
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMSHh7zg+001296
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMUAh7zg+001296
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMGGh7zg+001308
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMSHh7zg+001308
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMUAh7zg+001308
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMGGh7zg+001320
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMSHh7zg+001320
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMUAh7zg+001320
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMGGh7zg+001332
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMSHh7zg+001332
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMUAh7zg+001332
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a2gz_2016092300_15_991_12209990_0_r627146782_13.zip
Uploading the intermediate file: upload_file_13.zip
  00:59:47 STEP 1345 H= 336:15 +CPU= 33.023
  01:00:12 STEP 1346 H= 336:30 +CPU= 24.943
  01:00:38 STEP 1347 H= 336:45 +CPU= 25.458
  01:01:06 STEP 1348 H= 337:00 +CPU= 28.356
  01:01:32 STEP 1349 H= 337:15 +CPU= 25.122
  01:01:56 STEP 1350 H= 337:30 +CPU= 24.731
  01:02:22 STEP 1351 H= 337:45 +CPU= 25.516
  01:02:47 STEP 1352 H= 338:00 +CPU= 25.196
  01:03:12 STEP 1353 H= 338:15 +CPU= 24.598
  01:03:39 STEP 1354 H= 338:30 +CPU= 26.923
  01:04:11 STEP 1355 H= 338:45 +CPU= 31.860
  01:04:52 STEP 1356 H= 339:00 +CPU= 41.242
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001356
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001356
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001356
  01:05:23 STEP 1357 H= 339:15 +CPU= 31.148
  01:05:57 STEP 1358 H= 339:30 +CPU= 33.519
  01:06:29 STEP 1359 H= 339:45 +CPU= 31.757
  01:07:02 STEP 1360 H= 340:00 +CPU= 33.194
  01:07:34 STEP 1361 H= 340:15 +CPU= 31.673
  01:08:07 STEP 1362 H= 340:30 +CPU= 32.559
  01:08:39 STEP 1363 H= 340:45 +CPU= 31.839
  01:09:10 STEP 1364 H= 341:00 +CPU= 31.785
  01:09:43 STEP 1365 H= 341:15 +CPU= 32.788
  01:10:14 STEP 1366 H= 341:30 +CPU= 31.059
  01:10:45 STEP 1367 H= 341:45 +CPU= 30.513
  01:11:26 STEP 1368 H= 342:00 +CPU= 40.840
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001368
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001368
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001368
  01:11:58 STEP 1369 H= 342:15 +CPU= 32.314
  01:12:31 STEP 1370 H= 342:30 +CPU= 32.267
  01:13:03 STEP 1371 H= 342:45 +CPU= 32.385
  01:13:35 STEP 1372 H= 343:00 +CPU= 32.031
  01:14:07 STEP 1373 H= 343:15 +CPU= 31.777
  01:14:38 STEP 1374 H= 343:30 +CPU= 30.728
  01:15:11 STEP 1375 H= 343:45 +CPU= 33.352
  01:15:47 STEP 1376 H= 344:00 +CPU= 34.570
  01:16:20 STEP 1377 H= 344:15 +CPU= 32.377
  01:16:54 STEP 1378 H= 344:30 +CPU= 33.166
  01:17:27 STEP 1379 H= 344:45 +CPU= 32.308
  01:18:12 STEP 1380 H= 345:00 +CPU= 41.676
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001380
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001380
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001380
  01:18:46 STEP 1381 H= 345:15 +CPU= 33.083
  01:19:20 STEP 1382 H= 345:30 +CPU= 32.470
  01:19:54 STEP 1383 H= 345:45 +CPU= 32.419
  01:20:27 STEP 1384 H= 346:00 +CPU= 32.301
  01:20:58 STEP 1385 H= 346:15 +CPU= 31.451
  01:21:31 STEP 1386 H= 346:30 +CPU= 32.297
  01:22:02 STEP 1387 H= 346:45 +CPU= 31.621
  01:22:35 STEP 1388 H= 347:00 +CPU= 32.185
  01:23:06 STEP 1389 H= 347:15 +CPU= 31.659
  01:23:39 STEP 1390 H= 347:30 +CPU= 33.101
  01:24:10 STEP 1391 H= 347:45 +CPU= 30.767
  01:24:56 STEP 1392 H= 348:00 +CPU= 45.092
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001392
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001392
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001392
  01:25:27 STEP 1393 H= 348:15 +CPU= 30.725
  01:25:59 STEP 1394 H= 348:30 +CPU= 31.753
  01:26:31 STEP 1395 H= 348:45 +CPU= 32.502
  01:27:03 STEP 1396 H= 349:00 +CPU= 31.500
  01:27:35 STEP 1397 H= 349:15 +CPU= 31.861
  01:28:06 STEP 1398 H= 349:30 +CPU= 30.995
  01:28:38 STEP 1399 H= 349:45 +CPU= 32.134
  01:29:11 STEP 1400 H= 350:00 +CPU= 33.276
  01:29:44 STEP 1401 H= 350:15 +CPU= 32.591
  01:30:16 STEP 1402 H= 350:30 +CPU= 32.094
  01:30:49 STEP 1403 H= 350:45 +CPU= 32.528
  01:31:29 STEP 1404 H= 351:00 +CPU= 40.578
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001404
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001404
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001404
  01:32:01 STEP 1405 H= 351:15 +CPU= 31.907
  01:32:34 STEP 1406 H= 351:30 +CPU= 32.619
  01:33:06 STEP 1407 H= 351:45 +CPU= 32.323
  01:33:39 STEP 1408 H= 352:00 +CPU= 32.539
  01:34:11 STEP 1409 H= 352:15 +CPU= 31.907
  01:34:43 STEP 1410 H= 352:30 +CPU= 31.830
  01:35:14 STEP 1411 H= 352:45 +CPU= 31.008
  01:35:46 STEP 1412 H= 353:00 +CPU= 32.237
  01:36:16 STEP 1413 H= 353:15 +CPU= 29.762
  01:36:49 STEP 1414 H= 353:30 +CPU= 32.810
  01:37:20 STEP 1415 H= 353:45 +CPU= 31.260
  01:38:01 STEP 1416 H= 354:00 +CPU= 40.384
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001416
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001416
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001416
  01:38:33 STEP 1417 H= 354:15 +CPU= 31.976
  01:39:04 STEP 1418 H= 354:30 +CPU= 31.541
  01:39:36 STEP 1419 H= 354:45 +CPU= 31.766
  01:40:08 STEP 1420 H= 355:00 +CPU= 32.158
  01:40:41 STEP 1421 H= 355:15 +CPU= 32.066
  01:41:13 STEP 1422 H= 355:30 +CPU= 32.409
  01:41:45 STEP 1423 H= 355:45 +CPU= 31.545
  01:42:17 STEP 1424 H= 356:00 +CPU= 32.285
  01:42:48 STEP 1425 H= 356:15 +CPU= 30.985
  01:43:20 STEP 1426 H= 356:30 +CPU= 31.869
  01:43:53 STEP 1427 H= 356:45 +CPU= 32.823
  01:44:34 STEP 1428 H= 357:00 +CPU= 40.882
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001428
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001428
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001428
  01:45:06 STEP 1429 H= 357:15 +CPU= 31.746
  01:45:38 STEP 1430 H= 357:30 +CPU= 31.740
  01:46:09 STEP 1431 H= 357:45 +CPU= 31.305
  01:46:41 STEP 1432 H= 358:00 +CPU= 32.196
  01:47:13 STEP 1433 H= 358:15 +CPU= 31.478
  01:47:45 STEP 1434 H= 358:30 +CPU= 32.022
  01:48:17 STEP 1435 H= 358:45 +CPU= 32.409
  01:48:49 STEP 1436 H= 359:00 +CPU= 31.714
  01:49:21 STEP 1437 H= 359:15 +CPU= 31.715
  01:49:53 STEP 1438 H= 359:30 +CPU= 32.345
  01:50:27 STEP 1439 H= 359:45 +CPU= 33.244
  01:51:10 STEP 1440 H= 360:00 +CPU= 41.665
..The child process terminated with status: 0
>>> Printing last 70 lines from file: NODE.001_01

  ---------------------------------------------------------------------------------------------------------- I ----------------
 
DDH------------------------------------------------------------------------
I                                                                         I
I TIME =  360.000H DDH * Diagnostic Var.,     AREA                GLOBAL  I
I                                                                         I
I-------------------------------------------------------------------------I
I   VARIABLE   I      Initial value         I        Current value        I
I-------------------------------------------------------------------------I
I  OMEGA (P/S) I     -0.4820964656E-21      I       0.1682634291E-05      I
ICloud fractionI      0.0000000000E+00      I       0.1753165820E-01      I
I  Relat. Hum. I      0.3856394818E+00      I       0.4089527086E+00      I
I  PASS 01     I      0.0000000000E+00      I       0.2982830538E+03      I
DDH------------------------------------------------------------------------
  
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
 NSTEP =  1440 SCAN2M_HPOS  P
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ICMGGh7zg+001440 MODE=a
 IO-STREAM CLOSED - ICMGGh7zg+001440
 MPI-TASK:   1 -    311830401 BYTES IN      7 RECORDS TRANSFERRED IN    0.0007 SECONDS445472.0 Mbytes/s, TOTAL TIME=    0.0179( 3.9%)
 01:51:10 STEP 1440 H= 360:00 +CPU= 41.665
 END CNT3
 NSTEP =  1440 CNT0   000000000  29.701  29.701  29.973   0.991               0               0               0               0               0
IO-STREAM STATISTICS, TOTAL NO OF REC/BYTES READ           34/  1069125708 WRITTEN        11344/************
 -TASK-OPENED-OPEN-RECS IN -KBYTE IN    -RECS OUT-KBYTE OUT-WALL   -WALL IN-WALL
  OU-TOT IN -TOT OUT
   1    203    0       34      1044068    11344 ************    46.4     1.1    45.3     1.7    72.0
===-=== START OF TIMING STATISTICS ===-===
 
STATS FOR ALL TASKS
 NUM ROUTINE                                     CALLS  MEAN(ms)   MAX(ms)   FRAC(%)  UNBAL(%)
   0 CNT0     - COMPLETE EXECUTION                   1 *********       0.0    100.00   -100.00
   1 CNT4     - FORWARD INTEGRATION                  1 *********       0.0    100.00   -100.00
   8 SCAN2M - GRID-POINT DYNAMICS                  417    5331.8    5331.8      0.00      0.00
   9 SPCM     - SPECTRAL COMP.                     384     869.1     869.1      0.00      0.00
  10 SCAN2M - PHYSICS                              385 -543067.9       0.0    103.42   -103.42
  11 IOPACK   - OUTPUT P.P. RESULTS                 32    1844.9    1844.9      0.00      0.00
  12 SPNORM   - SPECTRAL NORM COMP.                 17      34.7      34.7      0.00      0.00
  17 GRIDFPOS IN CNT4                               32      40.6      40.6      0.00      0.00
  27 RESTART FILES - WRITING                         8    7375.1    7375.1      0.00      0.00
  28 RESTART FILES - READING                         1    2301.3    2301.3      0.00      0.00
  29 SU4FPOS IN CNT4                                32       0.1       0.1      0.00      0.00
  30 DYNFPOS IN CNT4                                32    7364.8    7364.8      0.00      0.00
  31 POSDDH IN STEPO                                 2      64.9      64.9      0.00      0.00
  37 CPGLAG   - SL COMPUTATIONS                    385    7221.0    7221.0      0.00      0.00
  39 SU0YOMB                                         1     381.1     381.1      0.00      0.00
  51 SCAN2M   - SL COMM. PART 1                    385      27.0      27.0      0.00      0.00
  54 SPCM     - M TO S/S TO M TRANSP.              384     392.2     392.2      0.00      0.00
  55 SPCIMPF  - S TO M/M TO S TRANSP.              384     126.4     126.4      0.00      0.00
  56 SPNORM   - SPECTRAL NORM COMM.                 17       0.2       0.2      0.00      0.00
 102 LTINV_CTL   - INVERSE LEGENDRE TRANSFORM      836     542.0     542.0      0.00      0.00
 103 LTDIR_CTL   - DIRECT LEGENDRE TRANSFORM       800     326.1     326.1      0.00      0.00
 106 FTDIR_CTL   - DIRECT FOURIER TRANSFORM        800     120.9     120.9      0.00      0.00
 107 FTINV_CTL   - INVERSE FOURIER TRANSFORM       836     167.7     167.7      0.00      0.00
 140 SULEG       - COMP. OF LEGENDRE POL.            1      49.2      49.2      0.00      0.00
 152 LTINV_CTL   - M TO L TRANSPOSITION            836      81.1      81.1      0.00      0.00
 153 LTDIR_CTL   - L TO M TRANSPOSITION            800      39.1      39.1      0.00      0.00
 157 FTINV_CTL   - L TO G TRANSPOSITION            836      45.3      45.3      0.00      0.00
 158 FTDIR_CTL   - G TO L TRANSPOSITION            800      24.3      24.3      0.00      0.00
 400 GSTATS                                      45376       0.0       0.0      0.00      0.00
 401 GSTATS HOOK                                 42168       0.0       0.0      0.00      0.00
TOTAL MEASURED IMBALANCE =       0.0 SECONDS,  0.0 PERCENT
TOTAL WALLCLOCK TIME  -202164.9 CPU TIME   12519.3 VECTOR TIME    12519.3


===-=== END   OF TIMING STATISTICS ===-===
  *** END CNT0 *** 
------------------------------------------------
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001440
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001440
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001440
Adding to the zip: /var/lib/boinc-client/slots/0/NODE.001_01
Adding to the zip: /var/lib/boinc-client/slots/0/ifs.stat
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMGGh7zg+001392
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMSHh7zg+001368
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMGGh7zg+001428
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMUAh7zg+001356
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMGGh7zg+001404
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMGGh7zg+001344
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMGGh7zg+001440
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMUAh7zg+001416
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMUAh7zg+001368
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMUAh7zg+001404
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMUAh7zg+001428
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMUAh7zg+001440
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMSHh7zg+001440
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMSHh7zg+001344
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMSHh7zg+001356
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMGGh7zg+001356
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMSHh7zg+001428
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMSHh7zg+001404
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMGGh7zg+001368
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMGGh7zg+001380
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMUAh7zg+001380
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMUAh7zg+001344
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMUAh7zg+001392
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMSHh7zg+001392
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMSHh7zg+001380
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMSHh7zg+001416
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209990/ICMGGh7zg+001416
Zipping up the final file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a2gz_2016092300_15_991_12209990_0_r627146782_14.zip
Uploading the final file: upload_file_14.zip
Uploading trickle at timestep: 1295100
01:54:46 (1595580): called boinc_finish(0)

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 May 2023 16:52:15 1536642 22310195 oifs_43r3_bl_a2gz_2016092300_15_991_12209990_0 1,295,100 47,858 0.0370


©2024 cpdn.org