climateprediction.net home page
Task 22258953

Task 22258953

Name oifs_43r3_ps_0672_1988050100_123_957_12174316_0
Workunit 12174316
Created 21 Dec 2022, 14:31:23 UTC
Sent 4 Jan 2023, 23:32:16 UTC
Report deadline 3 Feb 2023, 23:32:16 UTC
Received 6 Jan 2023, 1:37:20 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 1 (0x00000001) Unknown error code
Computer ID 1522740
Run time 23 hours 47 min 38 sec
CPU time 19 hours 10 min 20 sec
Validate state Invalid
Credit 7,378.56
Device peak FLOPS 3.43 GFLOPS
Application version OpenIFS 43r3 Perturbed Surface v1.05
x86_64-pc-linux-gnu
Peak working set size 4,417.21 MB
Peak swap size 4,977.81 MB
Peak disk usage 2,761.45 MB
Stderr
<core_client_version>7.16.6</core_client_version>
<![CDATA[
<message>
process exited with code 1 (0x1, -255)</message>
<stderr_txt>
rss), 0MB (maxstack), 0 (paging), nsigs = 1
[EC_DRHOOK:dex3703-xps129q23:1:1:29025:29025] [20230105:084838:1672937318:57684.977] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1542] Also activating Harakiri-alarm (SIGALRM=14) to expire after 500s elapsed to prevent hangs, nsigs = 1
[EC_DRHOOK:dex3703-xps129q23:1:1:29025:29025] [20230105:084838:1672937318:57684.977] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1544] Harakiri signal handler 'signal_harakiri' for signal#14 (SIGALRM) installed at 0x81f3c0 (old at (nil))
[EC_DRHOOK:dex3703-xps129q23:1:1:29025:29025] [20230105:084838:1672937318:57684.977] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1617] Signal#15 was caused by unrecognized si_code [memaddr=0x1], nsigs = 1
[EC_DRHOOK:dex3703-xps129q23:1:1:29025:29025] [20230105:084838:1672937318:57684.977] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1686] Starting DrHook backtrace for signal#15, nsigs = 1
[EC_DRHOOK:dex3703-xps129q23:1:1:29025:29025] [20230105:084838:1672937318:57684.977] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3843] 3992 MB (maxheap), 4403 MB (maxrss), 0 MB (maxstack)
[EC_DRHOOK:dex3703-xps129q23:1:1:29025:29025] [20230105:084838:1672937318:57684.977] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :  MASTER 
[EC_DRHOOK:dex3703-xps129q23:1:1:29025:29025] [20230105:084838:1672937318:57684.977] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :   CNT0 
[EC_DRHOOK:dex3703-xps129q23:1:1:29025:29025] [20230105:084838:1672937318:57684.977] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :    CNT1 
[EC_DRHOOK:dex3703-xps129q23:1:1:29025:29025] [20230105:084838:1672937318:57684.977] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :     CNT2 
[EC_DRHOOK:dex3703-xps129q23:1:1:29025:29025] [20230105:084838:1672937318:57684.977] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :      CNT3 
[EC_DRHOOK:dex3703-xps129q23:1:1:29025:29025] [20230105:084838:1672937318:57684.977] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :       CNT4 
[EC_DRHOOK:dex3703-xps129q23:1:1:29025:29025] [20230105:084838:1672937318:57684.977] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :        STEPO 
[EC_DRHOOK:dex3703-xps129q23:1:1:29025:29025] [20230105:084838:1672937318:57684.977] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :         SCAN2M 
[EC_DRHOOK:dex3703-xps129q23:1:1:29025:29025] [20230105:084838:1672937318:57684.977] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :          GP_MODEL_HEAP 
[EC_DRHOOK:dex3703-xps129q23:1:1:29025:29025] [20230105:084838:1672937318:57684.977] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :           GP_MODEL 
[EC_DRHOOK:dex3703-xps129q23:1:1:29025:29025] [20230105:084838:1672937318:57684.977] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :            EC_PHYS_DRV 
[EC_DRHOOK:dex3703-xps129q23:1:1:29025:29025] [20230105:084838:1672937318:57684.977] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :             >OMP-PHYSICS CLDPP T/S    (1002) 
[EC_DRHOOK:dex3703-xps129q23:1:1:29025:29025] [20230105:084838:1672937318:57684.977] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :              EC_PHYS 
[EC_DRHOOK:dex3703-xps129q23:1:1:29025:29025] [20230105:084838:1672937318:57684.977] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :               CALLPAR 
[EC_DRHOOK:dex3703-xps129q23:1:1:29025:29025] [20230105:084838:1672937318:57684.977] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :                STATE_UPDATE 
[EC_DRHOOK:dex3703-xps129q23:1:1:29025:29025] [20230105:084839:1672937319:57685.977] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1734] DrHook backtrace done for signal#15, nsigs = 1
[EC_DRHOOK:dex3703-xps129q23:1:1:29025:29025] [20230105:084839:1672937319:57685.977] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1785] Calling previous signal handler at 0x1ce8bb0 for signal#15, nsigs = 1
forrtl: error (78): process killed (SIGTERM)
Image              PC                Routine            Line        Source             
oifs_43r3_model.e  0000000001CE916B  Unknown               Unknown  Unknown
oifs_43r3_model.e  000000000081FFF1  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001DC9090  Unknown               Unknown  Unknown
oifs_43r3_model.e  000000000100A684  state_update_              34  state_update.F90
oifs_43r3_model.e  0000000000F01039  callpar_                  557  callpar.F90
oifs_43r3_model.e  0000000000E4940F  ec_phys_                  670  ec_phys.F90
oifs_43r3_model.e  0000000000E3610C  ec_phys_drv_              599  ec_phys_drv.F90
oifs_43r3_model.e  0000000000BFA3D3  gp_model_                 613  gp_model.F90
oifs_43r3_model.e  00000000012E9482  gp_model_heap_             74  gp_model_heap.F90
oifs_43r3_model.e  0000000000BBC1CE  scan2m_                   535  scan2m.F90
oifs_43r3_model.e  000000000057A664  stepo_                    327  stepo.F90
oifs_43r3_model.e  000000000055DEEC  cnt4_                    1133  cnt4.F90
oifs_43r3_model.e  00000000005471C9  cnt3_                     267  cnt3.F90
oifs_43r3_model.e  0000000000546412  cnt2_                      88  cnt2.F90
oifs_43r3_model.e  0000000000545E28  cnt1_                      92  cnt1.F90
oifs_43r3_model.e  000000000040708D  cnt0_                     146  cnt0.F90
oifs_43r3_model.e  000000000040220F  MAIN__                     96  master.F90
oifs_43r3_model.e  00000000004021A2  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001DCA390  Unknown               Unknown  Unknown
oifs_43r3_model.e  000000000040206E  Unknown               Unknown  Unknown
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 1988050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0672
(argv4) batchid: 957
(argv5) wuid: 12174316
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/4
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/4/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/4/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_5a81dfaa939c546bbc8cc485fa036468 to: /var/lib/boinc-client/slots/4/oifs_43r3_ps_0672_1988050100_123_957_12174316.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/4/oifs_43r3_ps_0672_1988050100_123_957_12174316.zip
ic_ancil_file: ic_ancil_12174316
ifsdata_file: ifsdata_12174316
climate_data_file: clim_data_12174316
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_f8151baa63a8b7d829e2c96faf3f3812 to: /var/lib/boinc-client/slots/4/ic_ancil_12174316.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/4/ic_ancil_12174316.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_b3db85ea6d5c7ebaec6c17fafd63b80f to: /var/lib/boinc-client/slots/4/ifsdata/ifsdata_12174316.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/4/ifsdata/ifsdata_12174316.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/4/159l_2/clim_data_12174316.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/4/159l_2/clim_data_12174316.zip
Checking for progress XML file: /var/lib/boinc-client/slots/4/progress_file_12174316.xml
Opened progress file ok : /var/lib/boinc-client/slots/4/progress_file_12174316.xml
-- Model is restarting --
Adjusting last_iter, 993, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/4/progress_file_12174316.xml
last_cpu_time: 46601
upload_file_number: 41
last_iter: 983
last_upload: 3542400
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0672_1988050100_123_957_12174316_0_r2028365346
The child process has been launched with process id: 1667
Executing the command: /var/lib/boinc-client/slots/4/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:dex3703-xps129q23:1:1:1667:1667] [20230105:085056:1672937456:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:dex3703-xps129q23:1:1:1667:1667] [20230105:085056:1672937456:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:dex3703-xps129q23:1:1:1667:1667] [20230105:085056:1672937456:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:dex3703-xps129q23:1:1:1667:1667] [20230105:085056:1672937456:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:dex3703-xps129q23:1:1:1667:1667] [20230105:085056:1672937456:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:dex3703-xps129q23:1:1:1667:1667] [20230105:085056:1672937456:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:dex3703-xps129q23:1:1:1667:1667] [20230105:085056:1672937456:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:dex3703-xps129q23:1:1:1667:1667] [20230105:085056:1672937456:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:dex3703-xps129q23:1:1:1667:1667] [20230105:085056:1672937456:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:dex3703-xps129q23:1:1:1667:1667] [20230105:085056:1672937456:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:dex3703-xps129q23:1:1:1667:1667] [20230105:085056:1672937456:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:dex3703-xps129q23:1:1:1667:1667] [20230105:085056:1672937456:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:dex3703-xps129q23:1:1:1667:1667] [20230105:085056:1672937456:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:dex3703-xps129q23:1:1:1667:1667] [20230105:085056:1672937456:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:dex3703-xps129q23:1:1:1667:1667] [20230105:085056:1672937456:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:dex3703-xps129q23:1:1:1667:1667] [20230105:085056:1672937456:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:dex3703-xps129q23:1:1:1667:1667] [20230105:085056:1672937456:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:dex3703-xps129q23:1:1:1667:1667] [20230105:085056:1672937456:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:dex3703-xps129q23:1:1:1667:1667] [20230105:085056:1672937456:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:dex3703-xps129q23:1:1:1667:1667] [20230105:085056:1672937456:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:dex3703-xps129q23:1:1:1667:1667] [20230105:085056:1672937456:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:dex3703-xps129q23:1:1:1667:1667] [20230105:085056:1672937456:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:dex3703-xps129q23:1:1:1667:1667] [20230105:085056:1672937456:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:dex3703-xps129q23:1:1:1667:1667] [20230105:085056:1672937456:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:dex3703-xps129q23:1:1:1667:1667] [20230105:085056:1672937456:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:dex3703-xps129q23:1:1:1667:1667] [20230105:085056:1672937456:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:dex3703-xps129q23:1:1:1667:1667] [20230105:085056:1672937456:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:dex3703-xps129q23:1:1:1667:1667] [20230105:085056:1672937456:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:dex3703-xps129q23:1:1:1667:1667] [20230105:085056:1672937456:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:dex3703-xps129q23:1:1:1667:1667] [20230105:085056:1672937456:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  08:52:23 STEP  984 H= 984:00 +CPU= 65.996
  08:53:08 STEP  985 H= 985:00 +CPU= 36.284
  08:53:56 STEP  986 H= 986:00 +CPU= 38.451
  08:55:08 STEP  987 H= 987:00 +CPU= 57.437
  08:55:55 STEP  988 H= 988:00 +CPU= 38.502
  08:56:41 STEP  989 H= 989:00 +CPU= 37.749
  08:57:55 STEP  990 H= 990:00 +CPU= 60.102
  08:58:41 STEP  991 H= 991:00 +CPU= 37.262
  08:59:28 STEP  992 H= 992:00 +CPU= 38.442
  09:00:38 STEP  993 H= 993:00 +CPU= 57.060
  09:01:24 STEP  994 H= 994:00 +CPU= 37.997
  09:02:09 STEP  995 H= 995:00 +CPU= 36.397
  09:03:30 STEP  996 H= 996:00 +CPU= 61.318
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000996
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000996
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000996
  09:04:17 STEP  997 H= 997:00 +CPU= 34.892
  09:05:07 STEP  998 H= 998:00 +CPU= 36.587
  09:06:19 STEP  999 H= 999:00 +CPU= 55.288
  09:07:07 STEP 1000 H=1000:00 +CPU= 35.827
  09:07:53 STEP 1001 H=1001:00 +CPU= 34.803
  09:09:08 STEP 1002 H=1002:00 +CPU= 56.319
  09:09:55 STEP 1003 H=1003:00 +CPU= 35.142
  09:10:44 STEP 1004 H=1004:00 +CPU= 36.505
  09:11:59 STEP 1005 H=1005:00 +CPU= 55.255
  09:12:47 STEP 1006 H=1006:00 +CPU= 36.272
  09:13:37 STEP 1007 H=1007:00 +CPU= 36.150
  09:15:03 STEP 1008 H=1008:00 +CPU= 68.932
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001008
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001008
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001008
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMGGhq0f+000984
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMSHhq0f+000984
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMUAhq0f+000984
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMGGhq0f+000996
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMSHhq0f+000996
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMUAhq0f+000996
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0672_1988050100_123_957_12174316_0_r2028365346_41.zip
Uploading the intermediate file: upload_file_41.zip
  09:15:50 STEP 1009 H=1009:00 +CPU= 37.299
  09:16:39 STEP 1010 H=1010:00 +CPU= 39.082
  09:17:52 STEP 1011 H=1011:00 +CPU= 59.234
  09:18:41 STEP 1012 H=1012:00 +CPU= 38.787
  09:19:28 STEP 1013 H=1013:00 +CPU= 37.539
  09:20:41 STEP 1014 H=1014:00 +CPU= 59.658
  09:21:27 STEP 1015 H=1015:00 +CPU= 36.600
  09:22:17 STEP 1016 H=1016:00 +CPU= 36.288
  09:23:30 STEP 1017 H=1017:00 +CPU= 55.579
  09:24:20 STEP 1018 H=1018:00 +CPU= 36.920
  09:25:08 STEP 1019 H=1019:00 +CPU= 35.423
  09:26:31 STEP 1020 H=1020:00 +CPU= 66.285
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001020
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001020
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001020
  09:27:17 STEP 1021 H=1021:00 +CPU= 36.910
  09:28:05 STEP 1022 H=1022:00 +CPU= 38.672
  09:29:17 STEP 1023 H=1023:00 +CPU= 58.540
  09:30:06 STEP 1024 H=1024:00 +CPU= 39.009
  09:30:53 STEP 1025 H=1025:00 +CPU= 38.001
  09:32:08 STEP 1026 H=1026:00 +CPU= 60.266
  09:32:54 STEP 1027 H=1027:00 +CPU= 36.814
  09:33:40 STEP 1028 H=1028:00 +CPU= 37.634
  09:34:51 STEP 1029 H=1029:00 +CPU= 58.014
  09:35:38 STEP 1030 H=1030:00 +CPU= 37.888
  09:36:25 STEP 1031 H=1031:00 +CPU= 37.852
  09:37:52 STEP 1032 H=1032:00 +CPU= 70.041
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001032
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001032
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001032
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMGGhq0f+001008
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMSHhq0f+001008
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMUAhq0f+001008
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMGGhq0f+001020
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMSHhq0f+001020
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMUAhq0f+001020
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0672_1988050100_123_957_12174316_0_r2028365346_42.zip
Uploading the intermediate file: upload_file_42.zip
  09:38:38 STEP 1033 H=1033:00 +CPU= 36.931
  09:39:27 STEP 1034 H=1034:00 +CPU= 39.035
  09:40:37 STEP 1035 H=1035:00 +CPU= 57.756
  09:41:24 STEP 1036 H=1036:00 +CPU= 37.866
  09:42:10 STEP 1037 H=1037:00 +CPU= 37.144
  09:43:23 STEP 1038 H=1038:00 +CPU= 59.788
  09:44:08 STEP 1039 H=1039:00 +CPU= 36.348
  09:44:55 STEP 1040 H=1040:00 +CPU= 38.143
  09:46:08 STEP 1041 H=1041:00 +CPU= 58.339
  09:46:54 STEP 1042 H=1042:00 +CPU= 37.811
  09:47:39 STEP 1043 H=1043:00 +CPU= 36.692
  09:48:59 STEP 1044 H=1044:00 +CPU= 64.416
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001044
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001044
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001044
  09:49:44 STEP 1045 H=1045:00 +CPU= 36.548
  09:50:30 STEP 1046 H=1046:00 +CPU= 37.539
  09:51:41 STEP 1047 H=1047:00 +CPU= 58.180
  09:52:27 STEP 1048 H=1048:00 +CPU= 37.763
  09:53:12 STEP 1049 H=1049:00 +CPU= 36.663
  09:54:26 STEP 1050 H=1050:00 +CPU= 59.946
  09:55:11 STEP 1051 H=1051:00 +CPU= 36.832
  09:55:58 STEP 1052 H=1052:00 +CPU= 37.981
  09:57:09 STEP 1053 H=1053:00 +CPU= 58.698
  09:57:57 STEP 1054 H=1054:00 +CPU= 38.169
  09:58:44 STEP 1055 H=1055:00 +CPU= 38.056
  10:00:08 STEP 1056 H=1056:00 +CPU= 68.411
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001056
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001056
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001056
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMGGhq0f+001032
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMSHhq0f+001032
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMUAhq0f+001032
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMGGhq0f+001044
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMSHhq0f+001044
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMUAhq0f+001044
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0672_1988050100_123_957_12174316_0_r2028365346_43.zip
Uploading the intermediate file: upload_file_43.zip
  10:00:55 STEP 1057 H=1057:00 +CPU= 37.440
  10:01:43 STEP 1058 H=1058:00 +CPU= 39.128
  10:02:55 STEP 1059 H=1059:00 +CPU= 57.938
  10:03:43 STEP 1060 H=1060:00 +CPU= 38.708
  10:04:30 STEP 1061 H=1061:00 +CPU= 37.781
  10:05:48 STEP 1062 H=1062:00 +CPU= 62.130
  10:06:36 STEP 1063 H=1063:00 +CPU= 38.157
  10:07:25 STEP 1064 H=1064:00 +CPU= 38.989
  10:08:41 STEP 1065 H=1065:00 +CPU= 59.992
  10:09:30 STEP 1066 H=1066:00 +CPU= 39.078
  10:10:15 STEP 1067 H=1067:00 +CPU= 36.806
  10:11:36 STEP 1068 H=1068:00 +CPU= 65.076
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001068
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001068
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001068
  10:12:20 STEP 1069 H=1069:00 +CPU= 36.480
  10:13:07 STEP 1070 H=1070:00 +CPU= 37.935
  10:14:19 STEP 1071 H=1071:00 +CPU= 58.543
  10:15:08 STEP 1072 H=1072:00 +CPU= 39.313
  10:15:55 STEP 1073 H=1073:00 +CPU= 37.393
  10:17:11 STEP 1074 H=1074:00 +CPU= 61.218
  10:17:58 STEP 1075 H=1075:00 +CPU= 37.766
  10:18:46 STEP 1076 H=1076:00 +CPU= 39.487
  10:20:02 STEP 1077 H=1077:00 +CPU= 61.224
  10:20:50 STEP 1078 H=1078:00 +CPU= 39.794
  10:21:39 STEP 1079 H=1079:00 +CPU= 40.238
  10:23:07 STEP 1080 H=1080:00 +CPU= 71.556
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001080
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001080
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001080
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMGGhq0f+001056
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMSHhq0f+001056
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMUAhq0f+001056
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMGGhq0f+001068
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMSHhq0f+001068
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMUAhq0f+001068
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0672_1988050100_123_957_12174316_0_r2028365346_44.zip
Uploading the intermediate file: upload_file_44.zip
  10:23:54 STEP 1081 H=1081:00 +CPU= 37.427
  10:24:42 STEP 1082 H=1082:00 +CPU= 39.819
  10:25:57 STEP 1083 H=1083:00 +CPU= 61.158
  10:26:44 STEP 1084 H=1084:00 +CPU= 39.291
  10:27:31 STEP 1085 H=1085:00 +CPU= 38.423
  10:28:47 STEP 1086 H=1086:00 +CPU= 62.624
  10:29:34 STEP 1087 H=1087:00 +CPU= 38.686
  10:30:22 STEP 1088 H=1088:00 +CPU= 39.598
  10:31:36 STEP 1089 H=1089:00 +CPU= 60.976
  10:32:25 STEP 1090 H=1090:00 +CPU= 39.373
  10:33:12 STEP 1091 H=1091:00 +CPU= 35.540
  10:34:36 STEP 1092 H=1092:00 +CPU= 63.208
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001092
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001092
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001092
  10:35:23 STEP 1093 H=1093:00 +CPU= 35.656
  10:36:11 STEP 1094 H=1094:00 +CPU= 37.100
  10:37:25 STEP 1095 H=1095:00 +CPU= 56.623
  10:38:13 STEP 1096 H=1096:00 +CPU= 36.985
  10:38:59 STEP 1097 H=1097:00 +CPU= 36.040
  10:40:17 STEP 1098 H=1098:00 +CPU= 63.393
  10:41:05 STEP 1099 H=1099:00 +CPU= 39.431
  10:41:53 STEP 1100 H=1100:00 +CPU= 39.193
  10:43:07 STEP 1101 H=1101:00 +CPU= 61.213
  10:43:56 STEP 1102 H=1102:00 +CPU= 40.679
  10:44:44 STEP 1103 H=1103:00 +CPU= 39.670
  10:46:15 STEP 1104 H=1104:00 +CPU= 73.685
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001104
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001104
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001104
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMGGhq0f+001080
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMSHhq0f+001080
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMUAhq0f+001080
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMGGhq0f+001092
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMSHhq0f+001092
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMUAhq0f+001092
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0672_1988050100_123_957_12174316_0_r2028365346_45.zip
Uploading the intermediate file: upload_file_45.zip
  10:47:03 STEP 1105 H=1105:00 +CPU= 39.028
  10:47:52 STEP 1106 H=1106:00 +CPU= 40.226
  10:49:08 STEP 1107 H=1107:00 +CPU= 61.703
  10:49:57 STEP 1108 H=1108:00 +CPU= 39.637
  10:50:42 STEP 1109 H=1109:00 +CPU= 37.668
  10:51:59 STEP 1110 H=1110:00 +CPU= 62.691
  10:52:44 STEP 1111 H=1111:00 +CPU= 37.344
  10:53:32 STEP 1112 H=1112:00 +CPU= 39.399
  10:54:43 STEP 1113 H=1113:00 +CPU= 59.292
  10:55:31 STEP 1114 H=1114:00 +CPU= 39.594
  10:56:18 STEP 1115 H=1115:00 +CPU= 38.281
  10:57:38 STEP 1116 H=1116:00 +CPU= 66.162
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001116
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001116
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001116
  10:58:26 STEP 1117 H=1117:00 +CPU= 38.930
  10:59:13 STEP 1118 H=1118:00 +CPU= 39.079
  11:00:28 STEP 1119 H=1119:00 +CPU= 61.420
  11:01:19 STEP 1120 H=1120:00 +CPU= 41.230
  11:02:07 STEP 1121 H=1121:00 +CPU= 39.476
  11:03:24 STEP 1122 H=1122:00 +CPU= 62.620
  11:04:11 STEP 1123 H=1123:00 +CPU= 38.614
  11:05:00 STEP 1124 H=1124:00 +CPU= 40.426
  11:06:15 STEP 1125 H=1125:00 +CPU= 61.110
  11:07:04 STEP 1126 H=1126:00 +CPU= 40.118
  11:07:51 STEP 1127 H=1127:00 +CPU= 39.462
  11:09:18 STEP 1128 H=1128:00 +CPU= 70.952
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001128
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001128
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001128
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMGGhq0f+001104
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMSHhq0f+001104
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMUAhq0f+001104
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMGGhq0f+001116
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMSHhq0f+001116
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMUAhq0f+001116
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0672_1988050100_123_957_12174316_0_r2028365346_46.zip
Uploading the intermediate file: upload_file_46.zip
  11:10:06 STEP 1129 H=1129:00 +CPU= 38.404
  11:10:52 STEP 1130 H=1130:00 +CPU= 38.606
  11:12:06 STEP 1131 H=1131:00 +CPU= 60.885
  11:12:56 STEP 1132 H=1132:00 +CPU= 40.403
  11:13:43 STEP 1133 H=1133:00 +CPU= 37.981
  11:15:00 STEP 1134 H=1134:00 +CPU= 64.588
  11:15:49 STEP 1135 H=1135:00 +CPU= 40.438
  11:16:38 STEP 1136 H=1136:00 +CPU= 41.219
  11:17:56 STEP 1137 H=1137:00 +CPU= 64.374
  11:18:45 STEP 1138 H=1138:00 +CPU= 41.213
  11:19:33 STEP 1139 H=1139:00 +CPU= 39.698
  11:20:56 STEP 1140 H=1140:00 +CPU= 70.237
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001140
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001140
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001140
  11:21:43 STEP 1141 H=1141:00 +CPU= 39.305
  11:22:33 STEP 1142 H=1142:00 +CPU= 40.905
  11:23:49 STEP 1143 H=1143:00 +CPU= 63.890
  11:24:40 STEP 1144 H=1144:00 +CPU= 41.396
  11:25:27 STEP 1145 H=1145:00 +CPU= 39.208
  11:26:46 STEP 1146 H=1146:00 +CPU= 65.336
  11:27:35 STEP 1147 H=1147:00 +CPU= 40.983
  11:28:24 STEP 1148 H=1148:00 +CPU= 40.131
  11:29:39 STEP 1149 H=1149:00 +CPU= 62.994
  11:30:27 STEP 1150 H=1150:00 +CPU= 40.989
  11:31:13 STEP 1151 H=1151:00 +CPU= 38.641
  11:32:42 STEP 1152 H=1152:00 +CPU= 72.617
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001152
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001152
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001152
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMGGhq0f+001128
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMSHhq0f+001128
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMUAhq0f+001128
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMGGhq0f+001140
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMSHhq0f+001140
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMUAhq0f+001140
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0672_1988050100_123_957_12174316_0_r2028365346_47.zip
Uploading the intermediate file: upload_file_47.zip
  11:33:30 STEP 1153 H=1153:00 +CPU= 39.030
  11:34:20 STEP 1154 H=1154:00 +CPU= 40.488
  11:35:34 STEP 1155 H=1155:00 +CPU= 61.152
  11:36:25 STEP 1156 H=1156:00 +CPU= 40.885
  11:37:13 STEP 1157 H=1157:00 +CPU= 39.451
  11:38:30 STEP 1158 H=1158:00 +CPU= 63.156
  11:39:20 STEP 1159 H=1159:00 +CPU= 39.765
  11:40:09 STEP 1160 H=1160:00 +CPU= 40.429
  11:41:23 STEP 1161 H=1161:00 +CPU= 60.157
  11:42:14 STEP 1162 H=1162:00 +CPU= 40.911
  11:43:01 STEP 1163 H=1163:00 +CPU= 38.657
  11:44:24 STEP 1164 H=1164:00 +CPU= 68.420
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001164
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001164
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001164
  11:45:14 STEP 1165 H=1165:00 +CPU= 39.670
  11:46:02 STEP 1166 H=1166:00 +CPU= 39.557
  11:47:16 STEP 1167 H=1167:00 +CPU= 60.321
  11:48:05 STEP 1168 H=1168:00 +CPU= 39.470
  11:48:52 STEP 1169 H=1169:00 +CPU= 39.146
  11:50:09 STEP 1170 H=1170:00 +CPU= 62.569
  11:50:56 STEP 1171 H=1171:00 +CPU= 37.191
  11:51:44 STEP 1172 H=1172:00 +CPU= 38.435
  11:52:58 STEP 1173 H=1173:00 +CPU= 58.754
  11:53:48 STEP 1174 H=1174:00 +CPU= 39.611
  11:54:37 STEP 1175 H=1175:00 +CPU= 39.308
  11:56:07 STEP 1176 H=1176:00 +CPU= 70.540
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001176
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001176
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001176
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMGGhq0f+001152
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMSHhq0f+001152
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMUAhq0f+001152
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMGGhq0f+001164
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMSHhq0f+001164
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMUAhq0f+001164
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0672_1988050100_123_957_12174316_0_r2028365346_48.zip
Uploading the intermediate file: upload_file_48.zip
  11:56:55 STEP 1177 H=1177:00 +CPU= 38.185
  11:57:45 STEP 1178 H=1178:00 +CPU= 39.415
  11:59:00 STEP 1179 H=1179:00 +CPU= 60.033
  11:59:50 STEP 1180 H=1180:00 +CPU= 39.188
  12:00:37 STEP 1181 H=1181:00 +CPU= 38.010
  12:01:54 STEP 1182 H=1182:00 +CPU= 61.144
  12:02:41 STEP 1183 H=1183:00 +CPU= 38.241
  12:03:31 STEP 1184 H=1184:00 +CPU= 39.411
  12:04:45 STEP 1185 H=1185:00 +CPU= 59.183
  12:05:35 STEP 1186 H=1186:00 +CPU= 39.748
  12:06:22 STEP 1187 H=1187:00 +CPU= 37.750
  12:07:47 STEP 1188 H=1188:00 +CPU= 66.603
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001188
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001188
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001188
  12:08:33 STEP 1189 H=1189:00 +CPU= 37.756
  12:09:21 STEP 1190 H=1190:00 +CPU= 39.054
  12:10:36 STEP 1191 H=1191:00 +CPU= 59.867
  12:11:25 STEP 1192 H=1192:00 +CPU= 39.110
  12:12:14 STEP 1193 H=1193:00 +CPU= 38.766
  12:13:29 STEP 1194 H=1194:00 +CPU= 60.778
  12:14:16 STEP 1195 H=1195:00 +CPU= 37.655
  12:15:03 STEP 1196 H=1196:00 +CPU= 38.353
  12:16:18 STEP 1197 H=1197:00 +CPU= 59.724
  12:17:07 STEP 1198 H=1198:00 +CPU= 38.839
  12:17:55 STEP 1199 H=1199:00 +CPU= 36.356
  12:19:20 STEP 1200 H=1200:00 +CPU= 68.618
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001200
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001200
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001200
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMGGhq0f+001176
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMSHhq0f+001176
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMUAhq0f+001176
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMGGhq0f+001188
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMSHhq0f+001188
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMUAhq0f+001188
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0672_1988050100_123_957_12174316_0_r2028365346_49.zip
Uploading the intermediate file: upload_file_49.zip
  12:20:06 STEP 1201 H=1201:00 +CPU= 36.997
  12:20:54 STEP 1202 H=1202:00 +CPU= 38.780
  12:22:06 STEP 1203 H=1203:00 +CPU= 58.226
  12:22:54 STEP 1204 H=1204:00 +CPU= 38.372
  12:23:40 STEP 1205 H=1205:00 +CPU= 37.765
  12:24:56 STEP 1206 H=1206:00 +CPU= 60.621
  12:25:41 STEP 1207 H=1207:00 +CPU= 36.839
  12:26:28 STEP 1208 H=1208:00 +CPU= 39.144
  12:27:40 STEP 1209 H=1209:00 +CPU= 59.922
  12:28:27 STEP 1210 H=1210:00 +CPU= 39.147
  12:29:14 STEP 1211 H=1211:00 +CPU= 39.096
  12:30:34 STEP 1212 H=1212:00 +CPU= 67.227
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001212
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001212
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001212
  12:31:20 STEP 1213 H=1213:00 +CPU= 38.494
  12:32:08 STEP 1214 H=1214:00 +CPU= 39.533
  12:33:20 STEP 1215 H=1215:00 +CPU= 59.555
  12:34:06 STEP 1216 H=1216:00 +CPU= 38.708
  12:34:51 STEP 1217 H=1217:00 +CPU= 36.911
  12:36:04 STEP 1218 H=1218:00 +CPU= 60.353
  12:36:49 STEP 1219 H=1219:00 +CPU= 37.433
  12:37:37 STEP 1220 H=1220:00 +CPU= 39.316
  12:38:50 STEP 1221 H=1221:00 +CPU= 59.882
  12:39:37 STEP 1222 H=1222:00 +CPU= 39.324
  12:40:24 STEP 1223 H=1223:00 +CPU= 39.378
  12:41:47 STEP 1224 H=1224:00 +CPU= 68.757
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001224
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001224
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001224
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMGGhq0f+001200
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMSHhq0f+001200
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMUAhq0f+001200
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMGGhq0f+001212
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMSHhq0f+001212
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMUAhq0f+001212
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0672_1988050100_123_957_12174316_0_r2028365346_50.zip
Uploading the intermediate file: upload_file_50.zip
Uploading trickle at timestep: 4406400
  12:42:31 STEP 1225 H=1225:00 +CPU= 36.813
  12:43:21 STEP 1226 H=1226:00 +CPU= 40.558
  12:44:33 STEP 1227 H=1227:00 +CPU= 59.958
  12:45:22 STEP 1228 H=1228:00 +CPU= 39.763
  12:46:10 STEP 1229 H=1229:00 +CPU= 38.699
  12:47:26 STEP 1230 H=1230:00 +CPU= 63.134
  12:48:13 STEP 1231 H=1231:00 +CPU= 38.405
  12:49:02 STEP 1232 H=1232:00 +CPU= 39.540
  12:50:16 STEP 1233 H=1233:00 +CPU= 60.960
  12:51:03 STEP 1234 H=1234:00 +CPU= 39.068
  12:51:48 STEP 1235 H=1235:00 +CPU= 37.526
  12:53:11 STEP 1236 H=1236:00 +CPU= 63.599
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001236
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001236
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001236
  12:53:57 STEP 1237 H=1237:00 +CPU= 35.697
  12:54:47 STEP 1238 H=1238:00 +CPU= 37.630
  12:56:01 STEP 1239 H=1239:00 +CPU= 56.935
  12:56:51 STEP 1240 H=1240:00 +CPU= 37.822
  12:57:39 STEP 1241 H=1241:00 +CPU= 36.947
  12:58:55 STEP 1242 H=1242:00 +CPU= 58.311
  12:59:42 STEP 1243 H=1243:00 +CPU= 36.522
  13:00:30 STEP 1244 H=1244:00 +CPU= 36.805
  13:01:44 STEP 1245 H=1245:00 +CPU= 57.023
  13:02:32 STEP 1246 H=1246:00 +CPU= 37.249
  13:03:20 STEP 1247 H=1247:00 +CPU= 36.814
  13:04:45 STEP 1248 H=1248:00 +CPU= 68.473
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001248
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001248
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001248
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMGGhq0f+001224
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMSHhq0f+001224
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMUAhq0f+001224
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMGGhq0f+001236
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMSHhq0f+001236
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMUAhq0f+001236
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0672_1988050100_123_957_12174316_0_r2028365346_51.zip
Uploading the intermediate file: upload_file_51.zip
  13:05:29 STEP 1249 H=1249:00 +CPU= 36.306
  13:06:17 STEP 1250 H=1250:00 +CPU= 38.674
  13:07:29 STEP 1251 H=1251:00 +CPU= 58.955
  13:08:17 STEP 1252 H=1252:00 +CPU= 38.738
  13:09:03 STEP 1253 H=1253:00 +CPU= 36.844
  13:10:15 STEP 1254 H=1254:00 +CPU= 59.154
  13:10:59 STEP 1255 H=1255:00 +CPU= 36.100
  13:11:45 STEP 1256 H=1256:00 +CPU= 37.061
  13:12:55 STEP 1257 H=1257:00 +CPU= 57.588
  13:13:40 STEP 1258 H=1258:00 +CPU= 37.611
  13:14:24 STEP 1259 H=1259:00 +CPU= 36.433
  13:15:41 STEP 1260 H=1260:00 +CPU= 63.313
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001260
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001260
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001260
  13:16:26 STEP 1261 H=1261:00 +CPU= 36.537
  13:17:11 STEP 1262 H=1262:00 +CPU= 37.842
  13:18:23 STEP 1263 H=1263:00 +CPU= 57.625
  13:19:12 STEP 1264 H=1264:00 +CPU= 39.130
  13:19:59 STEP 1265 H=1265:00 +CPU= 37.415
  13:21:15 STEP 1266 H=1266:00 +CPU= 60.520
  13:22:01 STEP 1267 H=1267:00 +CPU= 36.818
  13:22:48 STEP 1268 H=1268:00 +CPU= 37.511
  13:24:00 STEP 1269 H=1269:00 +CPU= 58.692
  13:24:48 STEP 1270 H=1270:00 +CPU= 38.177
  13:25:34 STEP 1271 H=1271:00 +CPU= 37.703
  13:27:00 STEP 1272 H=1272:00 +CPU= 69.421
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001272
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001272
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001272
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMGGhq0f+001248
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMSHhq0f+001248
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMUAhq0f+001248
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMGGhq0f+001260
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMSHhq0f+001260
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMUAhq0f+001260
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0672_1988050100_123_957_12174316_0_r2028365346_52.zip
Uploading the intermediate file: upload_file_52.zip
  13:27:46 STEP 1273 H=1273:00 +CPU= 36.932
  13:28:33 STEP 1274 H=1274:00 +CPU= 37.843
  13:29:45 STEP 1275 H=1275:00 +CPU= 58.388
  13:30:32 STEP 1276 H=1276:00 +CPU= 38.456
  13:31:18 STEP 1277 H=1277:00 +CPU= 37.002
  13:32:30 STEP 1278 H=1278:00 +CPU= 58.680
  13:33:15 STEP 1279 H=1279:00 +CPU= 36.769
  13:34:02 STEP 1280 H=1280:00 +CPU= 38.379
  13:35:16 STEP 1281 H=1281:00 +CPU= 58.916
  13:36:03 STEP 1282 H=1282:00 +CPU= 37.831
  13:36:50 STEP 1283 H=1283:00 +CPU= 38.156
  13:38:13 STEP 1284 H=1284:00 +CPU= 66.037
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001284
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001284
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001284
  13:39:00 STEP 1285 H=1285:00 +CPU= 37.367
  13:39:47 STEP 1286 H=1286:00 +CPU= 38.339
  13:41:00 STEP 1287 H=1287:00 +CPU= 59.273
  13:41:48 STEP 1288 H=1288:00 +CPU= 38.953
  13:42:35 STEP 1289 H=1289:00 +CPU= 37.344
  13:43:50 STEP 1290 H=1290:00 +CPU= 60.299
  13:44:38 STEP 1291 H=1291:00 +CPU= 35.212
  13:45:26 STEP 1292 H=1292:00 +CPU= 36.522
  13:46:41 STEP 1293 H=1293:00 +CPU= 55.531
  13:47:30 STEP 1294 H=1294:00 +CPU= 36.221
  13:48:19 STEP 1295 H=1295:00 +CPU= 36.639
  13:49:45 STEP 1296 H=1296:00 +CPU= 69.234
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001296
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001296
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001296
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMGGhq0f+001272
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMSHhq0f+001272
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMUAhq0f+001272
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMGGhq0f+001284
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMSHhq0f+001284
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMUAhq0f+001284
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0672_1988050100_123_957_12174316_0_r2028365346_53.zip
Uploading the intermediate file: upload_file_53.zip
  13:50:29 STEP 1297 H=1297:00 +CPU= 36.079
  13:51:16 STEP 1298 H=1298:00 +CPU= 37.746
  13:52:28 STEP 1299 H=1299:00 +CPU= 58.377
  13:53:15 STEP 1300 H=1300:00 +CPU= 38.110
  13:54:01 STEP 1301 H=1301:00 +CPU= 37.079
  13:55:14 STEP 1302 H=1302:00 +CPU= 60.758
  13:56:00 STEP 1303 H=1303:00 +CPU= 37.896
  13:56:45 STEP 1304 H=1304:00 +CPU= 37.995
  13:57:56 STEP 1305 H=1305:00 +CPU= 58.432
  13:58:42 STEP 1306 H=1306:00 +CPU= 37.380
  13:59:28 STEP 1307 H=1307:00 +CPU= 38.077
  14:00:46 STEP 1308 H=1308:00 +CPU= 65.529
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001308
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001308
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001308
  14:01:32 STEP 1309 H=1309:00 +CPU= 37.435
  14:02:19 STEP 1310 H=1310:00 +CPU= 39.272
  14:03:33 STEP 1311 H=1311:00 +CPU= 60.864
  14:04:22 STEP 1312 H=1312:00 +CPU= 39.703
  14:05:08 STEP 1313 H=1313:00 +CPU= 38.280
  14:06:23 STEP 1314 H=1314:00 +CPU= 61.531
  14:07:10 STEP 1315 H=1315:00 +CPU= 38.334
  14:07:58 STEP 1316 H=1316:00 +CPU= 38.872
  14:09:13 STEP 1317 H=1317:00 +CPU= 61.246
  14:10:03 STEP 1318 H=1318:00 +CPU= 37.752
  14:10:53 STEP 1319 H=1319:00 +CPU= 37.433
  14:12:20 STEP 1320 H=1320:00 +CPU= 70.554
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001320
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001320
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001320
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMGGhq0f+001296
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMSHhq0f+001296
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMUAhq0f+001296
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMGGhq0f+001308
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMSHhq0f+001308
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMUAhq0f+001308
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0672_1988050100_123_957_12174316_0_r2028365346_54.zip
Uploading the intermediate file: upload_file_54.zip
  14:13:06 STEP 1321 H=1321:00 +CPU= 37.779
  14:13:55 STEP 1322 H=1322:00 +CPU= 39.572
  14:15:08 STEP 1323 H=1323:00 +CPU= 60.603
  14:15:57 STEP 1324 H=1324:00 +CPU= 40.033
  14:16:43 STEP 1325 H=1325:00 +CPU= 38.166
  14:17:58 STEP 1326 H=1326:00 +CPU= 62.098
  14:18:45 STEP 1327 H=1327:00 +CPU= 38.696
  14:19:33 STEP 1328 H=1328:00 +CPU= 39.745
  14:20:44 STEP 1329 H=1329:00 +CPU= 58.977
  14:21:32 STEP 1330 H=1330:00 +CPU= 39.987
  14:22:18 STEP 1331 H=1331:00 +CPU= 38.600
  14:23:41 STEP 1332 H=1332:00 +CPU= 67.673
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001332
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001332
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001332
  14:24:26 STEP 1333 H=1333:00 +CPU= 37.030
  14:25:12 STEP 1334 H=1334:00 +CPU= 38.599
  14:26:23 STEP 1335 H=1335:00 +CPU= 58.747
  14:27:10 STEP 1336 H=1336:00 +CPU= 38.898
  14:27:54 STEP 1337 H=1337:00 +CPU= 36.897
  14:29:08 STEP 1338 H=1338:00 +CPU= 60.903
  14:29:52 STEP 1339 H=1339:00 +CPU= 37.315
  14:30:39 STEP 1340 H=1340:00 +CPU= 38.975
  14:31:49 STEP 1341 H=1341:00 +CPU= 57.505
  14:32:37 STEP 1342 H=1342:00 +CPU= 38.241
  14:33:23 STEP 1343 H=1343:00 +CPU= 37.181
  14:34:47 STEP 1344 H=1344:00 +CPU= 68.516
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001344
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001344
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001344
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMGGhq0f+001320
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMSHhq0f+001320
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMUAhq0f+001320
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMGGhq0f+001332
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMSHhq0f+001332
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMUAhq0f+001332
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0672_1988050100_123_957_12174316_0_r2028365346_55.zip
Uploading the intermediate file: upload_file_55.zip
  14:35:33 STEP 1345 H=1345:00 +CPU= 36.341
  14:36:19 STEP 1346 H=1346:00 +CPU= 37.938
  14:37:29 STEP 1347 H=1347:00 +CPU= 58.014
  14:38:15 STEP 1348 H=1348:00 +CPU= 37.754
  14:39:00 STEP 1349 H=1349:00 +CPU= 36.305
  14:40:13 STEP 1350 H=1350:00 +CPU= 59.510
  14:40:59 STEP 1351 H=1351:00 +CPU= 37.123
  14:41:46 STEP 1352 H=1352:00 +CPU= 37.731
  14:43:00 STEP 1353 H=1353:00 +CPU= 58.451
  14:43:46 STEP 1354 H=1354:00 +CPU= 38.248
  14:44:34 STEP 1355 H=1355:00 +CPU= 37.587
  14:45:55 STEP 1356 H=1356:00 +CPU= 65.128
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001356
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001356
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001356
  14:46:41 STEP 1357 H=1357:00 +CPU= 37.400
  14:47:30 STEP 1358 H=1358:00 +CPU= 39.100
  14:48:46 STEP 1359 H=1359:00 +CPU= 59.799
  14:49:36 STEP 1360 H=1360:00 +CPU= 39.752
  14:50:23 STEP 1361 H=1361:00 +CPU= 38.005
  14:51:38 STEP 1362 H=1362:00 +CPU= 60.165
  14:52:25 STEP 1363 H=1363:00 +CPU= 37.306
  14:53:15 STEP 1364 H=1364:00 +CPU= 39.447
  14:54:26 STEP 1365 H=1365:00 +CPU= 57.835
  14:55:14 STEP 1366 H=1366:00 +CPU= 38.448
  14:56:02 STEP 1367 H=1367:00 +CPU= 38.674
  14:57:29 STEP 1368 H=1368:00 +CPU= 69.154
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001368
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001368
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001368
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMGGhq0f+001344
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMSHhq0f+001344
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMUAhq0f+001344
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMGGhq0f+001356
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMSHhq0f+001356
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMUAhq0f+001356
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0672_1988050100_123_957_12174316_0_r2028365346_56.zip
Uploading the intermediate file: upload_file_56.zip
  14:58:17 STEP 1369 H=1369:00 +CPU= 37.830
  14:59:07 STEP 1370 H=1370:00 +CPU= 39.461
  15:00:21 STEP 1371 H=1371:00 +CPU= 59.795
  15:01:10 STEP 1372 H=1372:00 +CPU= 38.723
  15:01:57 STEP 1373 H=1373:00 +CPU= 37.457
  15:03:13 STEP 1374 H=1374:00 +CPU= 60.925
  15:04:00 STEP 1375 H=1375:00 +CPU= 38.448
  15:04:49 STEP 1376 H=1376:00 +CPU= 38.862
  15:06:05 STEP 1377 H=1377:00 +CPU= 60.469
  15:06:53 STEP 1378 H=1378:00 +CPU= 38.391
  15:07:38 STEP 1379 H=1379:00 +CPU= 36.817
  15:08:56 STEP 1380 H=1380:00 +CPU= 64.121
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001380
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001380
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001380
  15:09:42 STEP 1381 H=1381:00 +CPU= 36.844
  15:10:28 STEP 1382 H=1382:00 +CPU= 37.987
  15:11:40 STEP 1383 H=1383:00 +CPU= 58.219
  15:12:28 STEP 1384 H=1384:00 +CPU= 38.238
  15:13:14 STEP 1385 H=1385:00 +CPU= 37.611
  15:14:29 STEP 1386 H=1386:00 +CPU= 60.126
  15:15:16 STEP 1387 H=1387:00 +CPU= 37.285
  15:16:04 STEP 1388 H=1388:00 +CPU= 38.742
  15:17:15 STEP 1389 H=1389:00 +CPU= 58.171
  15:18:03 STEP 1390 H=1390:00 +CPU= 38.380
  15:18:51 STEP 1391 H=1391:00 +CPU= 38.832
  15:20:18 STEP 1392 H=1392:00 +CPU= 64.951
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001392
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001392
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001392
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMGGhq0f+001368
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMSHhq0f+001368
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMUAhq0f+001368
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMGGhq0f+001380
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMSHhq0f+001380
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMUAhq0f+001380
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0672_1988050100_123_957_12174316_0_r2028365346_57.zip
Uploading the intermediate file: upload_file_57.zip
  15:21:04 STEP 1393 H=1393:00 +CPU= 35.002
  15:21:54 STEP 1394 H=1394:00 +CPU= 36.463
  15:23:08 STEP 1395 H=1395:00 +CPU= 55.382
  15:23:57 STEP 1396 H=1396:00 +CPU= 36.463
  15:24:45 STEP 1397 H=1397:00 +CPU= 35.442
  15:26:03 STEP 1398 H=1398:00 +CPU= 57.345
  15:26:51 STEP 1399 H=1399:00 +CPU= 36.031
  15:27:42 STEP 1400 H=1400:00 +CPU= 37.718
  15:28:58 STEP 1401 H=1401:00 +CPU= 57.997
  15:29:47 STEP 1402 H=1402:00 +CPU= 37.204
  15:30:35 STEP 1403 H=1403:00 +CPU= 36.149
  15:31:57 STEP 1404 H=1404:00 +CPU= 63.456
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001404
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001404
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001404
  15:32:44 STEP 1405 H=1405:00 +CPU= 35.989
  15:33:31 STEP 1406 H=1406:00 +CPU= 36.434
  15:34:45 STEP 1407 H=1407:00 +CPU= 56.912
  15:35:35 STEP 1408 H=1408:00 +CPU= 37.536
  15:36:21 STEP 1409 H=1409:00 +CPU= 35.681
  15:37:37 STEP 1410 H=1410:00 +CPU= 58.254
  15:38:23 STEP 1411 H=1411:00 +CPU= 35.545
  15:39:15 STEP 1412 H=1412:00 +CPU= 37.856
  15:40:29 STEP 1413 H=1413:00 +CPU= 56.820
  15:41:17 STEP 1414 H=1414:00 +CPU= 37.091
  15:42:04 STEP 1415 H=1415:00 +CPU= 36.933
  15:43:30 STEP 1416 H=1416:00 +CPU= 67.459
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001416
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001416
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001416
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMGGhq0f+001392
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMSHhq0f+001392
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMUAhq0f+001392
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMGGhq0f+001404
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMSHhq0f+001404
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMUAhq0f+001404
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0672_1988050100_123_957_12174316_0_r2028365346_58.zip
Uploading the intermediate file: upload_file_58.zip
  15:44:19 STEP 1417 H=1417:00 +CPU= 36.584
  15:45:07 STEP 1418 H=1418:00 +CPU= 37.486
  15:46:21 STEP 1419 H=1419:00 +CPU= 57.139
  15:47:08 STEP 1420 H=1420:00 +CPU= 36.700
  15:47:56 STEP 1421 H=1421:00 +CPU= 36.742
  15:49:09 STEP 1422 H=1422:00 +CPU= 57.671
  15:49:55 STEP 1423 H=1423:00 +CPU= 35.745
  15:50:44 STEP 1424 H=1424:00 +CPU= 37.303
  15:51:57 STEP 1425 H=1425:00 +CPU= 56.441
  15:52:45 STEP 1426 H=1426:00 +CPU= 37.303
  15:53:32 STEP 1427 H=1427:00 +CPU= 36.176
  15:54:54 STEP 1428 H=1428:00 +CPU= 63.805
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001428
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001428
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001428
  15:55:40 STEP 1429 H=1429:00 +CPU= 35.691
  15:56:27 STEP 1430 H=1430:00 +CPU= 36.743
  15:57:39 STEP 1431 H=1431:00 +CPU= 56.608
  15:58:25 STEP 1432 H=1432:00 +CPU= 38.331
  15:59:11 STEP 1433 H=1433:00 +CPU= 37.736
  16:00:24 STEP 1434 H=1434:00 +CPU= 60.846
  16:01:09 STEP 1435 H=1435:00 +CPU= 36.793
  16:01:57 STEP 1436 H=1436:00 +CPU= 39.464
  16:03:08 STEP 1437 H=1437:00 +CPU= 58.867
  16:03:54 STEP 1438 H=1438:00 +CPU= 37.843
  16:04:41 STEP 1439 H=1439:00 +CPU= 38.264
  16:06:06 STEP 1440 H=1440:00 +CPU= 69.106
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001440
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001440
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001440
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMGGhq0f+001416
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMSHhq0f+001416
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMUAhq0f+001416
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMGGhq0f+001428
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMSHhq0f+001428
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMUAhq0f+001428
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0672_1988050100_123_957_12174316_0_r2028365346_59.zip
Uploading the intermediate file: upload_file_59.zip
  16:06:52 STEP 1441 H=1441:00 +CPU= 37.001
  16:07:38 STEP 1442 H=1442:00 +CPU= 38.047
  16:08:49 STEP 1443 H=1443:00 +CPU= 58.313
  16:09:36 STEP 1444 H=1444:00 +CPU= 37.863
  16:10:20 STEP 1445 H=1445:00 +CPU= 36.939
  16:11:33 STEP 1446 H=1446:00 +CPU= 60.520
  16:12:18 STEP 1447 H=1447:00 +CPU= 36.967
  16:13:04 STEP 1448 H=1448:00 +CPU= 38.141
  16:14:15 STEP 1449 H=1449:00 +CPU= 58.612
  16:15:02 STEP 1450 H=1450:00 +CPU= 38.172
  16:15:49 STEP 1451 H=1451:00 +CPU= 37.982
  16:17:10 STEP 1452 H=1452:00 +CPU= 65.503
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001452
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001452
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001452
  16:17:57 STEP 1453 H=1453:00 +CPU= 37.535
  16:18:48 STEP 1454 H=1454:00 +CPU= 40.127
  16:20:01 STEP 1455 H=1455:00 +CPU= 59.516
  16:20:50 STEP 1456 H=1456:00 +CPU= 39.123
  16:21:38 STEP 1457 H=1457:00 +CPU= 38.006
  16:22:54 STEP 1458 H=1458:00 +CPU= 61.353
  16:23:41 STEP 1459 H=1459:00 +CPU= 37.445
  16:24:31 STEP 1460 H=1460:00 +CPU= 39.428
  16:25:46 STEP 1461 H=1461:00 +CPU= 60.223
  16:26:36 STEP 1462 H=1462:00 +CPU= 37.700
  16:27:28 STEP 1463 H=1463:00 +CPU= 37.104
  16:28:58 STEP 1464 H=1464:00 +CPU= 71.562
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+001464
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+001464
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+001464
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMGGhq0f+001440
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMSHhq0f+001440
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMUAhq0f+001440
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMGGhq0f+001452
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMSHhq0f+001452
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174316/ICMUAhq0f+001452
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0672_1988050100_123_957_12174316_0_r2028365346_60.zip
Uploading the intermediate file: upload_file_60.zip
Uploading trickle at timestep: 5270400
  16:29:45 STEP 1465 H=1465:00 +CPU= 37.449
  16:30:36 STEP 1466 H=1466:00 +CPU= 39.946
  16:31:51 STEP 1467 H=1467:00 +CPU= 60.371
  16:32:41 STEP 1468 H=1468:00 +CPU= 39.142
  16:33:29 STEP 1469 H=1469:00 +CPU= 37.880
  16:34:45 STEP 1470 H=1470:00 +CPU= 61.053
  16:35:33 STEP 1471 H=1471:00 +CPU= 38.020
The child process terminated with status: 0
..Failed, model did not complete successfully

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 May 2023 16:09:42 1522740 22258953 oifs_43r3_ps_0672_1988050100_123_957_12174316_0 5,270,400 68,699 0.0130
09 May 2023 16:09:29 1522740 22258953 oifs_43r3_ps_0672_1988050100_123_957_12174316_0 4,406,400 57,789 0.0131
09 May 2023 16:09:12 1522740 22258953 oifs_43r3_ps_0672_1988050100_123_957_12174316_0 3,456,000 45,045 0.0130
09 May 2023 16:08:58 1522740 22258953 oifs_43r3_ps_0672_1988050100_123_957_12174316_0 2,592,000 33,836 0.0131
09 May 2023 16:08:44 1522740 22258953 oifs_43r3_ps_0672_1988050100_123_957_12174316_0 1,728,000 22,647 0.0131
09 May 2023 16:08:32 1522740 22258953 oifs_43r3_ps_0672_1988050100_123_957_12174316_0 864,000 11,556 0.0134


©2024 climateprediction.net