<core_client_version>7.18.1</core_client_version>
<![CDATA[
<stderr_txt>
0000000000E+000
NSTEP = 1584 SCAN2M_HPOS P
IO-STREAM SETUP - IOTYPE = 2 NUMIOPROCS = 1 CPATH =
ICMGGh7zg+001584 MODE=w
IO-STREAM CLOSED - ICMGGh7zg+001584
MPI-TASK: 1 - 320863428 BYTES IN 6 RECORDS TRANSFERRED IN 0.0005 SECONDS641726.9 Mbytes/s, TOTAL TIME= 0.0123( 4.1%)
NSTEP = 1584 STEPO 0AA000000
IO-STREAM SETUP - IOTYPE = 2 NUMIOPROCS = 1 CPATH =
./ICMSHh7zg+001584 MODE=w
IO-STREAM CLOSED - ./ICMSHh7zg+001584
MPI-TASK: 1 - 707399250 BYTES IN 202 RECORDS TRANSFERRED IN 0.0129 SECONDS 54837.2 Mbytes/s, TOTAL TIME= 0.2751( 4.7%)
IO-STREAM SETUP - IOTYPE = 2 NUMIOPROCS = 1 CPATH =
./ICMUAh7zg+001584 MODE=w
IO-STREAM CLOSED - ./ICMUAh7zg+001584
MPI-TASK: 1 - 5133814848 BYTES IN 125 RECORDS TRANSFERRED IN 0.0083 SECONDS618531.9 Mbytes/s, TOTAL TIME= 0.2499( 3.3%)
STEPO_FPOS WAS : B00000F00E
IO-STREAM SETUP - IOTYPE = 2 NUMIOPROCS = 1 CPATH =
./ICMUAh7zg+001584 MODE=a
IO-STREAM CLOSED - ./ICMUAh7zg+001584
MPI-TASK: 1 - 10 BYTES IN 2 RECORDS TRANSFERRED IN 0.0003 SECONDS 0.0 Mbytes/s, TOTAL TIME= 0.0050( 6.0%)
STEPO_FPOS WAS : V00000000Y
IO-STREAM SETUP - IOTYPE = 2 NUMIOPROCS = 1 CPATH =
./ICMUAh7zg+001584 MODE=a
IO-STREAM CLOSED - ./ICMUAh7zg+001584
MPI-TASK: 1 - 312283699 BYTES IN 4 RECORDS TRANSFERRED IN 0.0004 SECONDS780709.2 Mbytes/s, TOTAL TIME= 0.0069( 5.8%)
STEPO_FPOS WAS : T00000000M
NSTEP = 1584 STEPO 0AAA00AAA
MAXGPFV : MAX. VALUE = 0.000000000000000E+000
MAXGPFV : MAX. VALUE = 0.000000000000000E+000
NSTEP = 1584 SCAN2M_HPOS P
IO-STREAM SETUP - IOTYPE = 2 NUMIOPROCS = 1 CPATH =
ICMGGh7zg+001584 MODE=a
IO-STREAM CLOSED - ICMGGh7zg+001584
MPI-TASK: 1 - 320863428 BYTES IN 7 RECORDS TRANSFERRED IN 0.0006 SECONDS534772.4 Mbytes/s, TOTAL TIME= 0.0140( 4.3%)
09:51:24 STEP 1584 H= 396:00 +CPU= 39.349
ISTEP= 1585 RSOLINC= 1361.4482 RI0= 1365.0000 ZI0= 1361.4482 RIP0= 1361.4482
ISTEP= 1585 ZI0= 1361.4482 ZSEASON= 1.000000000 REA=149597870000.00 RDEASO=149597870000.00
NSTEP = 1585 STEPO 0AAA00AAA
------------------------------------------------
CNT0 not found; string returned was: 'STEPO'
>>> Printing last 8 lines from file: ifs.stat
09:50:44 F00000000 STEPO 1584 29.793 29.793 30.342 66:16 66:59 0.19689844928827E-05 2GB 0MB
09:50:48 0AA000000 STEPO 1584 3.518 3.518 3.527 66:19 67:02 0.19689844928827E-05 2GB 0MB
09:50:49 FULLPOS-B DYNFPOS 1584 1.609 1.609 1.612 66:21 67:04 0.19689844928827E-05 2GB 0MB
09:50:53 FULLPOS-V DYNFPOS 1584 3.276 3.276 3.360 66:24 67:07 0.19689844928827E-05 2GB 0MB
09:50:53 FULLPOS-T DYNFPOS 1584 0.503 0.503 0.560 66:25 67:08 0.19689844928827E-05 2GB 0MB
09:50:54 0AAA00AAA STEPO 1584 0.475 0.475 0.487 66:25 67:08 0.19689844928827E-05 2GB 0MB
09:51:24 0AAA00AAA STEPO 1585 29.758 29.758 29.966 66:55 67:38 0.19689844928827E-05 2GB 0MB
------------------------------------------------
>>> Printing last 11 lines from file: rcf
CSTEP = " 1584",
CTIME = "00161200 ",
NSTEPLPP = 201, 202, 49, 228226, 228227, 2*0, 1572, 2*0,
IPRGPNSRES = 1,
IPRGPEWRES = 1,
IPRTRWRES = 1,
IPRTRVRES = 1,
GMASS0 = 0.000000000000000E+000,
GMASSI = 0.000000000000000E+000
/
------------------------------------------------
>>> Printing last 8 lines from file: /var/lib/boinc-client/slots/14/progress_file_12290336.xml
<running_values>
<last_cpu_time>39127.550000</last_cpu_time>
<upload_file_number>14</upload_file_number>
<last_iter>1585</last_iter>
<last_upload>1383300</last_upload>
<model_completed>0</model_completed>
</running_values>
------------------------------------------------
..Failed, model did not complete successfully
(argv0) ../../projects/climateprediction.net/oifs_43r3_bl_1.13_x86_64-pc-linux-gnu
(argv1) start_date: 2016092300
(argv2) exptid: h7zg
(argv3) unique_member_id: a1jt
(argv4) batchid: 1018
(argv5) wuid: 12290336
(argv6) fclen: 20
(argv7) app_name: oifs_43r3_bl
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/14
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_bl
version: 1.13
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290336
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/14/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/14/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_a50ff084338dcfb54bca575dfc1ba98c to: /var/lib/boinc-client/slots/14/oifs_43r3_bl_a1jt_2016092300_20_1018_12290336.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/14/oifs_43r3_bl_a1jt_2016092300_20_1018_12290336.zip
ic_ancil_file: ic_ancil_12290336
ifsdata_file: ifsdata_12290336
climate_data_file: clim_data_12290336
horiz_resolution: 159
vert_resolution: 91
grid_type: l_2
upload_interval: 96
utstep: 900
nfrres: restart dump frequency (steps) 48
Copying IC ancils from: ../../projects/climateprediction.net/jf_f7ecfd27aa1e68bbe7e8d19800e25c73 to: /var/lib/boinc-client/slots/14/ic_ancil_12290336.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/14/ic_ancil_12290336.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_6f701c91b9e1c1c8eb599adf929bfc6c to: /var/lib/boinc-client/slots/14/ifsdata/ifsdata_12290336.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/14/ifsdata/ifsdata_12290336.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/14/159l_2/clim_data_12290336.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/14/159l_2/clim_data_12290336.zip
Checking for progress XML file: /var/lib/boinc-client/slots/14/progress_file_12290336.xml
Opened progress file ok : /var/lib/boinc-client/slots/14/progress_file_12290336.xml
-- Model is restarting --
Adjusting last_iter, 1585, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/14/progress_file_12290336.xml
last_cpu_time: 39127
upload_file_number: 14
last_iter: 1585
last_upload: 1383300
model_completed: 0
total_length_of_simulation: 1728000
result_base_name: oifs_43r3_bl_a1jt_2016092300_20_1018_12290336_0_r1764552379
The child process has been launched with process id: 903556
Executing the command: /var/lib/boinc-client/slots/14/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:box.zenlime.org:1:1:903556:903556] [20240615:095233:1718445153:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2091] fp = 0x314f440
[EC_DRHOOK:box.zenlime.org:1:1:903556:903556] [20240615:095233:1718445153:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:box.zenlime.org:1:1:903556:903556] [20240615:095233:1718445153:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:box.zenlime.org:1:1:903556:903556] [20240615:095233:1718445153:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:box.zenlime.org:1:1:903556:903556] [20240615:095233:1718445153:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:box.zenlime.org:1:1:903556:903556] [20240615:095233:1718445153:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647)
[EC_DRHOOK:box.zenlime.org:1:1:903556:903556] [20240615:095233:1718445153:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:box.zenlime.org:1:1:903556:903556] [20240615:095233:1718445153:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:box.zenlime.org:1:1:903556:903556] [20240615:095233:1718445153:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:box.zenlime.org:1:1:903556:903556] [20240615:095233:1718445153:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:box.zenlime.org:1:1:903556:903556] [20240615:095233:1718445153:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:box.zenlime.org:1:1:903556:903556] [20240615:095233:1718445153:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:box.zenlime.org:1:1:903556:903556] [20240615:095233:1718445153:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:box.zenlime.org:1:1:903556:903556] [20240615:095233:1718445153:0.000] [ignore_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:box.zenlime.org:1:1:903556:903556] [20240615:095233:1718445153:0.000] [restore_default_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:box.zenlime.org:1:1:903556:903556] [20240615:095233:1718445153:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:box.zenlime.org:1:1:903556:903556] [20240615:095233:1718445153:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:box.zenlime.org:1:1:903556:903556] [20240615:095233:1718445153:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:box.zenlime.org:1:1:903556:903556] [20240615:095233:1718445153:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x820cf0 (old at (nil))
[EC_DRHOOK:box.zenlime.org:1:1:903556:903556] [20240615:095233:1718445153:0.000] [trapfpe_treatment@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:box.zenlime.org:1:1:903556:903556] [20240615:095233:1718445153:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:box.zenlime.org:1:1:903556:903556] [20240615:095233:1718445153:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:box.zenlime.org:1:1:903556:903556] [20240615:095233:1718445153:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x820cf0 (old at (nil))
[EC_DRHOOK:box.zenlime.org:1:1:903556:903556] [20240615:095233:1718445153:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:box.zenlime.org:1:1:903556:903556] [20240615:095233:1718445153:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:box.zenlime.org:1:1:903556:903556] [20240615:095233:1718445153:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:box.zenlime.org:1:1:903556:903556] [20240615:095233:1718445153:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x820cf0 (old at (nil))
[EC_DRHOOK:box.zenlime.org:1:1:903556:903556] [20240615:095233:1718445153:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x820cf0 (old at (nil))
[EC_DRHOOK:box.zenlime.org:1:1:903556:903556] [20240615:095233:1718445153:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:box.zenlime.org:1:1:903556:903556] [20240615:095233:1718445153:0.000] [catch_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
09:53:05 STEP 1584 H= 396:00 +CPU= 28.049
09:53:30 STEP 1585 H= 396:15 +CPU= 24.529
09:53:55 STEP 1586 H= 396:30 +CPU= 24.544
09:54:20 STEP 1587 H= 396:45 +CPU= 25.410
09:54:47 STEP 1588 H= 397:00 +CPU= 26.855
09:55:17 STEP 1589 H= 397:15 +CPU= 29.139
09:55:46 STEP 1590 H= 397:30 +CPU= 29.605
09:56:16 STEP 1591 H= 397:45 +CPU= 29.078
09:56:46 STEP 1592 H= 398:00 +CPU= 30.230
09:57:15 STEP 1593 H= 398:15 +CPU= 28.740
09:57:45 STEP 1594 H= 398:30 +CPU= 29.173
09:58:11 STEP 1595 H= 398:45 +CPU= 26.711
09:58:43 STEP 1596 H= 399:00 +CPU= 31.291
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGh7zg+001596
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHh7zg+001596
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAh7zg+001596
09:59:09 STEP 1597 H= 399:15 +CPU= 26.175
09:59:34 STEP 1598 H= 399:30 +CPU= 24.950
10:00:00 STEP 1599 H= 399:45 +CPU= 25.872
10:00:26 STEP 1600 H= 400:00 +CPU= 25.493
10:00:50 STEP 1601 H= 400:15 +CPU= 24.450
10:01:19 STEP 1602 H= 400:30 +CPU= 28.570
10:01:49 STEP 1603 H= 400:45 +CPU= 29.365
10:02:18 STEP 1604 H= 401:00 +CPU= 29.153
10:02:48 STEP 1605 H= 401:15 +CPU= 29.116
10:03:18 STEP 1606 H= 401:30 +CPU= 29.385
10:03:44 STEP 1607 H= 401:45 +CPU= 25.895
10:04:12 STEP 1608 H= 402:00 +CPU= 28.264
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGh7zg+001608
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHh7zg+001608
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAh7zg+001608
10:04:33 STEP 1609 H= 402:15 +CPU= 21.407
10:04:58 STEP 1610 H= 402:30 +CPU= 24.896
10:05:22 STEP 1611 H= 402:45 +CPU= 23.496
10:05:48 STEP 1612 H= 403:00 +CPU= 25.919
10:06:14 STEP 1613 H= 403:15 +CPU= 26.097
10:06:40 STEP 1614 H= 403:30 +CPU= 26.114
10:07:09 STEP 1615 H= 403:45 +CPU= 28.391
10:07:39 STEP 1616 H= 404:00 +CPU= 29.568
10:08:08 STEP 1617 H= 404:15 +CPU= 29.169
10:08:37 STEP 1618 H= 404:30 +CPU= 28.952
10:09:07 STEP 1619 H= 404:45 +CPU= 29.300
10:09:43 STEP 1620 H= 405:00 +CPU= 35.770
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGh7zg+001620
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHh7zg+001620
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAh7zg+001620
10:10:06 STEP 1621 H= 405:15 +CPU= 22.469
10:10:26 STEP 1622 H= 405:30 +CPU= 19.827
10:10:47 STEP 1623 H= 405:45 +CPU= 20.873
10:11:13 STEP 1624 H= 406:00 +CPU= 26.229
10:11:39 STEP 1625 H= 406:15 +CPU= 25.948
10:12:05 STEP 1626 H= 406:30 +CPU= 25.827
10:12:31 STEP 1627 H= 406:45 +CPU= 25.605
10:12:57 STEP 1628 H= 407:00 +CPU= 26.271
10:13:27 STEP 1629 H= 407:15 +CPU= 28.948
10:13:57 STEP 1630 H= 407:30 +CPU= 29.232
10:14:27 STEP 1631 H= 407:45 +CPU= 29.467
10:15:06 STEP 1632 H= 408:00 +CPU= 38.622
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGh7zg+001632
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHh7zg+001632
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAh7zg+001632
10:15:36 STEP 1633 H= 408:15 +CPU= 29.335
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290336/ICMGGh7zg+001548
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290336/ICMSHh7zg+001548
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290336/ICMUAh7zg+001548
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290336/ICMGGh7zg+001560
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290336/ICMSHh7zg+001560
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290336/ICMUAh7zg+001560
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290336/ICMGGh7zg+001572
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290336/ICMSHh7zg+001572
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290336/ICMUAh7zg+001572
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290336/ICMGGh7zg+001584
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290336/ICMSHh7zg+001584
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290336/ICMUAh7zg+001584
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290336/ICMGGh7zg+001596
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290336/ICMSHh7zg+001596
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290336/ICMUAh7zg+001596
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290336/ICMGGh7zg+001608
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290336/ICMSHh7zg+001608
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290336/ICMUAh7zg+001608
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290336/ICMGGh7zg+001620
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290336/ICMSHh7zg+001620
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290336/ICMUAh7zg+001620
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290336/ICMGGh7zg+001632
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290336/ICMSHh7zg+001632
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290336/ICMUAh7zg+001632
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a1jt_2016092300_20_1018_12290336_0_r1764552379_14.zip
Uploading the intermediate file: upload_file_14.zip
10:16:06 STEP 1634 H= 408:30 +CPU= 28.993
10:16:35 STEP 1635 H= 408:45 +CPU= 29.008
10:17:06 STEP 1636 H= 409:00 +CPU= 29.968
10:17:35 STEP 1637 H= 409:15 +CPU= 29.412
10:18:05 STEP 1638 H= 409:30 +CPU= 29.557
10:18:35 STEP 1639 H= 409:45 +CPU= 29.063
10:19:05 STEP 1640 H= 410:00 +CPU= 29.554
10:19:34 STEP 1641 H= 410:15 +CPU= 29.514
10:20:05 STEP 1642 H= 410:30 +CPU= 29.397
10:20:34 STEP 1643 H= 410:45 +CPU= 28.868
10:21:10 STEP 1644 H= 411:00 +CPU= 35.732
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGh7zg+001644
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHh7zg+001644
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAh7zg+001644
10:21:41 STEP 1645 H= 411:15 +CPU= 29.564
10:22:10 STEP 1646 H= 411:30 +CPU= 28.118
10:22:36 STEP 1647 H= 411:45 +CPU= 26.087
10:23:03 STEP 1648 H= 412:00 +CPU= 26.550
10:23:29 STEP 1649 H= 412:15 +CPU= 25.866
10:23:55 STEP 1650 H= 412:30 +CPU= 25.841
10:24:21 STEP 1651 H= 412:45 +CPU= 25.729
10:24:47 STEP 1652 H= 413:00 +CPU= 25.937
10:25:14 STEP 1653 H= 413:15 +CPU= 27.063
10:25:44 STEP 1654 H= 413:30 +CPU= 29.163
10:26:15 STEP 1655 H= 413:45 +CPU= 29.723
10:26:51 STEP 1656 H= 414:00 +CPU= 36.225
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGh7zg+001656
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHh7zg+001656
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAh7zg+001656
10:27:20 STEP 1657 H= 414:15 +CPU= 28.947
10:27:50 STEP 1658 H= 414:30 +CPU= 29.371
10:28:16 STEP 1659 H= 414:45 +CPU= 26.099
10:28:38 STEP 1660 H= 415:00 +CPU= 21.804
10:28:59 STEP 1661 H= 415:15 +CPU= 21.032
10:29:24 STEP 1662 H= 415:30 +CPU= 25.538
10:29:44 STEP 1663 H= 415:45 +CPU= 19.785
10:30:06 STEP 1664 H= 416:00 +CPU= 21.983
10:30:34 STEP 1665 H= 416:15 +CPU= 27.353
10:30:58 STEP 1666 H= 416:30 +CPU= 24.281
10:31:24 STEP 1667 H= 416:45 +CPU= 25.673
10:32:01 STEP 1668 H= 417:00 +CPU= 36.363
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGh7zg+001668
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHh7zg+001668
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAh7zg+001668
10:32:31 STEP 1669 H= 417:15 +CPU= 29.155
10:33:00 STEP 1670 H= 417:30 +CPU= 29.093
10:33:30 STEP 1671 H= 417:45 +CPU= 29.554
10:34:01 STEP 1672 H= 418:00 +CPU= 29.946
10:34:24 STEP 1673 H= 418:15 +CPU= 22.618
10:34:47 STEP 1674 H= 418:30 +CPU= 22.802
10:35:12 STEP 1675 H= 418:45 +CPU= 25.256
10:35:38 STEP 1676 H= 419:00 +CPU= 25.822
10:36:05 STEP 1677 H= 419:15 +CPU= 25.967
10:36:31 STEP 1678 H= 419:30 +CPU= 26.497
10:36:56 STEP 1679 H= 419:45 +CPU= 24.541
10:37:32 STEP 1680 H= 420:00 +CPU= 36.106
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGh7zg+001680
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHh7zg+001680
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAh7zg+001680
10:37:59 STEP 1681 H= 420:15 +CPU= 27.230
10:38:26 STEP 1682 H= 420:30 +CPU= 27.168
10:38:51 STEP 1683 H= 420:45 +CPU= 24.979
10:39:19 STEP 1684 H= 421:00 +CPU= 27.035
10:39:44 STEP 1685 H= 421:15 +CPU= 25.107
10:40:06 STEP 1686 H= 421:30 +CPU= 22.691
10:40:31 STEP 1687 H= 421:45 +CPU= 24.804
10:40:57 STEP 1688 H= 422:00 +CPU= 26.042
10:41:22 STEP 1689 H= 422:15 +CPU= 24.617
10:41:48 STEP 1690 H= 422:30 +CPU= 26.104
10:42:12 STEP 1691 H= 422:45 +CPU= 23.384
10:42:43 STEP 1692 H= 423:00 +CPU= 31.037
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGh7zg+001692
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHh7zg+001692
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAh7zg+001692
10:43:10 STEP 1693 H= 423:15 +CPU= 27.169
10:43:38 STEP 1694 H= 423:30 +CPU= 27.551
10:44:05 STEP 1695 H= 423:45 +CPU= 27.214
10:44:33 STEP 1696 H= 424:00 +CPU= 28.327
10:45:01 STEP 1697 H= 424:15 +CPU= 27.347
10:45:28 STEP 1698 H= 424:30 +CPU= 27.138
10:45:55 STEP 1699 H= 424:45 +CPU= 26.873
10:46:20 STEP 1700 H= 425:00 +CPU= 24.687
10:46:46 STEP 1701 H= 425:15 +CPU= 26.088
10:47:11 STEP 1702 H= 425:30 +CPU= 24.826
10:47:37 STEP 1703 H= 425:45 +CPU= 25.854
10:48:09 STEP 1704 H= 426:00 +CPU= 32.060
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGh7zg+001704
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHh7zg+001704
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAh7zg+001704
10:48:34 STEP 1705 H= 426:15 +CPU= 24.627
10:48:59 STEP 1706 H= 426:30 +CPU= 24.981
10:49:28 STEP 1707 H= 426:45 +CPU= 28.474
10:49:58 STEP 1708 H= 427:00 +CPU= 29.809
10:50:28 STEP 1709 H= 427:15 +CPU= 29.536
10:50:58 STEP 1710 H= 427:30 +CPU= 29.970
10:51:28 STEP 1711 H= 427:45 +CPU= 29.576
10:51:56 STEP 1712 H= 428:00 +CPU= 27.383
10:52:16 STEP 1713 H= 428:15 +CPU= 20.457
10:52:39 STEP 1714 H= 428:30 +CPU= 22.937
10:53:05 STEP 1715 H= 428:45 +CPU= 25.481
10:53:36 STEP 1716 H= 429:00 +CPU= 30.615
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGh7zg+001716
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHh7zg+001716
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAh7zg+001716
10:54:01 STEP 1717 H= 429:15 +CPU= 25.452
10:54:28 STEP 1718 H= 429:30 +CPU= 26.381
10:54:54 STEP 1719 H= 429:45 +CPU= 26.339
10:55:24 STEP 1720 H= 430:00 +CPU= 29.532
10:55:54 STEP 1721 H= 430:15 +CPU= 29.415
10:56:24 STEP 1722 H= 430:30 +CPU= 29.767
10:56:54 STEP 1723 H= 430:45 +CPU= 29.488
10:57:25 STEP 1724 H= 431:00 +CPU= 30.152
10:57:54 STEP 1725 H= 431:15 +CPU= 29.274
10:58:20 STEP 1726 H= 431:30 +CPU= 25.249
10:58:40 STEP 1727 H= 431:45 +CPU= 19.696
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_bl_1.13_x86_64-pc-linux-gnu
(argv1) start_date: 2016092300
(argv2) exptid: h7zg
(argv3) unique_member_id: a1jt
(argv4) batchid: 1018
(argv5) wuid: 12290336
(argv6) fclen: 20
(argv7) app_name: oifs_43r3_bl
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/14
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_bl
version: 1.13
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290336
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/14/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/14/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_a50ff084338dcfb54bca575dfc1ba98c to: /var/lib/boinc-client/slots/14/oifs_43r3_bl_a1jt_2016092300_20_1018_12290336.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/14/oifs_43r3_bl_a1jt_2016092300_20_1018_12290336.zip
ic_ancil_file: ic_ancil_12290336
ifsdata_file: ifsdata_12290336
climate_data_file: clim_data_12290336
horiz_resolution: 159
vert_resolution: 91
grid_type: l_2
upload_interval: 96
utstep: 900
nfrres: restart dump frequency (steps) 48
Copying IC ancils from: ../../projects/climateprediction.net/jf_f7ecfd27aa1e68bbe7e8d19800e25c73 to: /var/lib/boinc-client/slots/14/ic_ancil_12290336.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/14/ic_ancil_12290336.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_6f701c91b9e1c1c8eb599adf929bfc6c to: /var/lib/boinc-client/slots/14/ifsdata/ifsdata_12290336.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/14/ifsdata/ifsdata_12290336.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/14/159l_2/clim_data_12290336.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/14/159l_2/clim_data_12290336.zip
Checking for progress XML file: /var/lib/boinc-client/slots/14/progress_file_12290336.xml
Opened progress file ok : /var/lib/boinc-client/slots/14/progress_file_12290336.xml
-- Model is restarting --
Adjusting last_iter, 1728, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/14/progress_file_12290336.xml
last_cpu_time: 43086
upload_file_number: 15
last_iter: 1729
last_upload: 1469700
model_completed: 0
total_length_of_simulation: 1728000
result_base_name: oifs_43r3_bl_a1jt_2016092300_20_1018_12290336_0_r1764552379
The child process has been launched with process id: 906697
Executing the command: /var/lib/boinc-client/slots/14/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:box.zenlime.org:1:1:906697:906697] [20240615:110013:1718449213:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2091] fp = 0x314f440
[EC_DRHOOK:box.zenlime.org:1:1:906697:906697] [20240615:110013:1718449213:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:box.zenlime.org:1:1:906697:906697] [20240615:110013:1718449213:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:box.zenlime.org:1:1:906697:906697] [20240615:110013:1718449213:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:box.zenlime.org:1:1:906697:906697] [20240615:110013:1718449213:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:box.zenlime.org:1:1:906697:906697] [20240615:110013:1718449213:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647)
[EC_DRHOOK:box.zenlime.org:1:1:906697:906697] [20240615:110013:1718449213:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:box.zenlime.org:1:1:906697:906697] [20240615:110013:1718449213:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:box.zenlime.org:1:1:906697:906697] [20240615:110013:1718449213:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:box.zenlime.org:1:1:906697:906697] [20240615:110013:1718449213:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:box.zenlime.org:1:1:906697:906697] [20240615:110013:1718449213:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:box.zenlime.org:1:1:906697:906697] [20240615:110013:1718449213:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:box.zenlime.org:1:1:906697:906697] [20240615:110013:1718449213:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:box.zenlime.org:1:1:906697:906697] [20240615:110013:1718449213:0.000] [ignore_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:box.zenlime.org:1:1:906697:906697] [20240615:110013:1718449213:0.000] [restore_default_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:box.zenlime.org:1:1:906697:906697] [20240615:110013:1718449213:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:box.zenlime.org:1:1:906697:906697] [20240615:110013:1718449213:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:box.zenlime.org:1:1:906697:906697] [20240615:110013:1718449213:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:box.zenlime.org:1:1:906697:906697] [20240615:110013:1718449213:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x820cf0 (old at (nil))
[EC_DRHOOK:box.zenlime.org:1:1:906697:906697] [20240615:110013:1718449213:0.000] [trapfpe_treatment@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:box.zenlime.org:1:1:906697:906697] [20240615:110013:1718449213:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:box.zenlime.org:1:1:906697:906697] [20240615:110013:1718449213:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:box.zenlime.org:1:1:906697:906697] [20240615:110013:1718449213:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x820cf0 (old at (nil))
[EC_DRHOOK:box.zenlime.org:1:1:906697:906697] [20240615:110013:1718449213:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:box.zenlime.org:1:1:906697:906697] [20240615:110013:1718449213:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:box.zenlime.org:1:1:906697:906697] [20240615:110013:1718449213:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:box.zenlime.org:1:1:906697:906697] [20240615:110013:1718449213:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x820cf0 (old at (nil))
[EC_DRHOOK:box.zenlime.org:1:1:906697:906697] [20240615:110013:1718449213:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x820cf0 (old at (nil))
[EC_DRHOOK:box.zenlime.org:1:1:906697:906697] [20240615:110013:1718449213:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:box.zenlime.org:1:1:906697:906697] [20240615:110013:1718449213:0.000] [catch_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290336/ICMGGh7zg+001644
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290336/ICMSHh7zg+001644
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290336/ICMUAh7zg+001644
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290336/ICMGGh7zg+001656
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290336/ICMSHh7zg+001656
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290336/ICMUAh7zg+001656
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290336/ICMGGh7zg+001668
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290336/ICMSHh7zg+001668
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290336/ICMUAh7zg+001668
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290336/ICMGGh7zg+001680
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290336/ICMSHh7zg+001680
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290336/ICMUAh7zg+001680
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290336/ICMGGh7zg+001692
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290336/ICMSHh7zg+001692
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290336/ICMUAh7zg+001692
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290336/ICMGGh7zg+001704
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290336/ICMSHh7zg+001704
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290336/ICMUAh7zg+001704
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290336/ICMGGh7zg+001716
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290336/ICMSHh7zg+001716
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290336/ICMUAh7zg+001716
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a1jt_2016092300_20_1018_12290336_0_r1764552379_15.zip
Uploading the intermediate file: upload_file_15.zip
11:00:43 STEP 1728 H= 432:00 +CPU= 27.194
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGh7zg+001728
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHh7zg+001728
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAh7zg+001728
11:01:09 STEP 1729 H= 432:15 +CPU= 25.127
11:01:38 STEP 1730 H= 432:30 +CPU= 28.982
11:02:08 STEP 1731 H= 432:45 +CPU= 29.502
11:02:39 STEP 1732 H= 433:00 +CPU= 30.450
11:03:08 STEP 1733 H= 433:15 +CPU= 29.389
11:03:39 STEP 1734 H= 433:30 +CPU= 30.326
11:04:09 STEP 1735 H= 433:45 +CPU= 29.697
11:04:39 STEP 1736 H= 434:00 +CPU= 29.988
11:05:09 STEP 1737 H= 434:15 +CPU= 29.411
11:05:38 STEP 1738 H= 434:30 +CPU= 29.239
11:06:08 STEP 1739 H= 434:45 +CPU= 29.902
11:06:45 STEP 1740 H= 435:00 +CPU= 36.673
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGh7zg+001740
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHh7zg+001740
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAh7zg+001740
11:07:14 STEP 1741 H= 435:15 +CPU= 27.840
11:07:43 STEP 1742 H= 435:30 +CPU= 29.177
11:08:13 STEP 1743 H= 435:45 +CPU= 29.637
11:08:43 STEP 1744 H= 436:00 +CPU= 29.854
11:09:13 STEP 1745 H= 436:15 +CPU= 29.405
11:09:44 STEP 1746 H= 436:30 +CPU= 29.712
11:10:07 STEP 1747 H= 436:45 +CPU= 23.518
11:10:28 STEP 1748 H= 437:00 +CPU= 20.046
11:10:46 STEP 1749 H= 437:15 +CPU= 18.895
11:11:09 STEP 1750 H= 437:30 +CPU= 22.188
11:11:33 STEP 1751 H= 437:45 +CPU= 24.612
11:12:02 STEP 1752 H= 438:00 +CPU= 29.156
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGh7zg+001752
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHh7zg+001752
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAh7zg+001752
11:12:27 STEP 1753 H= 438:15 +CPU= 24.133
11:12:50 STEP 1754 H= 438:30 +CPU= 23.107
11:13:17 STEP 1755 H= 438:45 +CPU= 27.011
11:13:48 STEP 1756 H= 439:00 +CPU= 29.943
11:14:17 STEP 1757 H= 439:15 +CPU= 29.138
11:14:47 STEP 1758 H= 439:30 +CPU= 29.643
11:15:17 STEP 1759 H= 439:45 +CPU= 29.502
11:15:47 STEP 1760 H= 440:00 +CPU= 29.758
11:16:17 STEP 1761 H= 440:15 +CPU= 28.925
11:16:39 STEP 1762 H= 440:30 +CPU= 21.725
11:17:00 STEP 1763 H= 440:45 +CPU= 21.300
11:17:29 STEP 1764 H= 441:00 +CPU= 29.251
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGh7zg+001764
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHh7zg+001764
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAh7zg+001764
11:17:56 STEP 1765 H= 441:15 +CPU= 25.855
11:18:19 STEP 1766 H= 441:30 +CPU= 23.597
11:18:46 STEP 1767 H= 441:45 +CPU= 26.715
11:19:14 STEP 1768 H= 442:00 +CPU= 26.544
11:19:44 STEP 1769 H= 442:15 +CPU= 29.396
11:20:13 STEP 1770 H= 442:30 +CPU= 29.224
11:20:43 STEP 1771 H= 442:45 +CPU= 29.351
11:21:14 STEP 1772 H= 443:00 +CPU= 30.212
11:21:45 STEP 1773 H= 443:15 +CPU= 29.648
11:22:14 STEP 1774 H= 443:30 +CPU= 28.533
11:22:44 STEP 1775 H= 443:45 +CPU= 29.671
11:23:25 STEP 1776 H= 444:00 +CPU= 39.544
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGh7zg+001776
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHh7zg+001776
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAh7zg+001776
11:23:55 STEP 1777 H= 444:15 +CPU= 29.341
11:24:25 STEP 1778 H= 444:30 +CPU= 29.417
11:24:54 STEP 1779 H= 444:45 +CPU= 29.414
11:25:24 STEP 1780 H= 445:00 +CPU= 30.043
11:25:55 STEP 1781 H= 445:15 +CPU= 29.934
11:26:25 STEP 1782 H= 445:30 +CPU= 29.557
11:26:55 STEP 1783 H= 445:45 +CPU= 29.431
11:27:26 STEP 1784 H= 446:00 +CPU= 30.347
11:27:56 STEP 1785 H= 446:15 +CPU= 29.282
11:28:26 STEP 1786 H= 446:30 +CPU= 29.523
11:28:57 STEP 1787 H= 446:45 +CPU= 28.989
11:29:33 STEP 1788 H= 447:00 +CPU= 34.820
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGh7zg+001788
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHh7zg+001788
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAh7zg+001788
11:29:54 STEP 1789 H= 447:15 +CPU= 20.446
11:30:15 STEP 1790 H= 447:30 +CPU= 21.264
11:30:40 STEP 1791 H= 447:45 +CPU= 24.537
11:31:05 STEP 1792 H= 448:00 +CPU= 25.445
11:31:29 STEP 1793 H= 448:15 +CPU= 23.821
11:31:54 STEP 1794 H= 448:30 +CPU= 25.306
11:32:19 STEP 1795 H= 448:45 +CPU= 24.918
11:32:48 STEP 1796 H= 449:00 +CPU= 28.088
11:33:17 STEP 1797 H= 449:15 +CPU= 29.387
11:33:47 STEP 1798 H= 449:30 +CPU= 29.749
11:34:17 STEP 1799 H= 449:45 +CPU= 29.315
11:34:54 STEP 1800 H= 450:00 +CPU= 36.607
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGh7zg+001800
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHh7zg+001800
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAh7zg+001800
11:35:23 STEP 1801 H= 450:15 +CPU= 29.259
11:35:53 STEP 1802 H= 450:30 +CPU= 29.774
11:36:22 STEP 1803 H= 450:45 +CPU= 29.132
11:36:51 STEP 1804 H= 451:00 +CPU= 28.625
11:37:16 STEP 1805 H= 451:15 +CPU= 25.080
11:37:42 STEP 1806 H= 451:30 +CPU= 25.546
11:38:06 STEP 1807 H= 451:45 +CPU= 24.554
11:38:31 STEP 1808 H= 452:00 +CPU= 24.986
11:38:57 STEP 1809 H= 452:15 +CPU= 25.981
11:39:23 STEP 1810 H= 452:30 +CPU= 24.928
11:39:50 STEP 1811 H= 452:45 +CPU= 26.939
11:40:25 STEP 1812 H= 453:00 +CPU= 34.245
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGh7zg+001812
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHh7zg+001812
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAh7zg+001812
11:40:53 STEP 1813 H= 453:15 +CPU= 28.012
11:41:20 STEP 1814 H= 453:30 +CPU= 26.920
11:41:47 STEP 1815 H= 453:45 +CPU= 26.602
11:42:15 STEP 1816 H= 454:00 +CPU= 27.457
11:42:42 STEP 1817 H= 454:15 +CPU= 27.055
11:43:10 STEP 1818 H= 454:30 +CPU= 27.293
11:43:37 STEP 1819 H= 454:45 +CPU= 27.169
11:44:05 STEP 1820 H= 455:00 +CPU= 27.851
11:44:32 STEP 1821 H= 455:15 +CPU= 27.472
11:45:02 STEP 1822 H= 455:30 +CPU= 29.312
11:45:32 STEP 1823 H= 455:45 +CPU= 29.496
11:46:11 STEP 1824 H= 456:00 +CPU= 39.299
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGh7zg+001824
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHh7zg+001824
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAh7zg+001824
11:46:41 STEP 1825 H= 456:15 +CPU= 29.607
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290336/ICMGGh7zg+001740
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290336/ICMSHh7zg+001740
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290336/ICMUAh7zg+001740
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290336/ICMGGh7zg+001752
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290336/ICMSHh7zg+001752
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290336/ICMUAh7zg+001752
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290336/ICMGGh7zg+001764
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290336/ICMSHh7zg+001764
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290336/ICMUAh7zg+001764
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290336/ICMGGh7zg+001776
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290336/ICMSHh7zg+001776
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290336/ICMUAh7zg+001776
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290336/ICMGGh7zg+001788
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290336/ICMSHh7zg+001788
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290336/ICMUAh7zg+001788
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290336/ICMGGh7zg+001800
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290336/ICMSHh7zg+001800
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290336/ICMUAh7zg+001800
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290336/ICMGGh7zg+001812
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290336/ICMSHh7zg+001812
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290336/ICMUAh7zg+001812
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290336/ICMGGh7zg+001824
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290336/ICMSHh7zg+001824
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290336/ICMUAh7zg+001824
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a1jt_2016092300_20_1018_12290336_0_r1764552379_16.zip
Uploading the intermediate file: upload_file_16.zip
11:47:12 STEP 1826 H= 456:30 +CPU= 29.240
11:47:42 STEP 1827 H= 456:45 +CPU= 29.534
11:48:13 STEP 1828 H= 457:00 +CPU= 30.399
11:48:43 STEP 1829 H= 457:15 +CPU= 29.653
11:49:13 STEP 1830 H= 457:30 +CPU= 29.368
11:49:43 STEP 1831 H= 457:45 +CPU= 29.648
11:50:13 STEP 1832 H= 458:00 +CPU= 29.954
11:50:43 STEP 1833 H= 458:15 +CPU= 29.334
11:51:12 STEP 1834 H= 458:30 +CPU= 29.065
11:51:42 STEP 1835 H= 458:45 +CPU= 29.976
11:52:20 STEP 1836 H= 459:00 +CPU= 36.570
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGh7zg+001836
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHh7zg+001836
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAh7zg+001836
11:52:49 STEP 1837 H= 459:15 +CPU= 29.392
11:53:19 STEP 1838 H= 459:30 +CPU= 29.178
11:53:49 STEP 1839 H= 459:45 +CPU= 29.533
11:54:19 STEP 1840 H= 460:00 +CPU= 29.297
11:54:49 STEP 1841 H= 460:15 +CPU= 29.594
11:55:18 STEP 1842 H= 460:30 +CPU= 29.303
11:55:48 STEP 1843 H= 460:45 +CPU= 29.332
11:56:18 STEP 1844 H= 461:00 +CPU= 29.714
11:56:48 STEP 1845 H= 461:15 +CPU= 29.554
11:57:17 STEP 1846 H= 461:30 +CPU= 29.183
11:57:47 STEP 1847 H= 461:45 +CPU= 29.286
11:58:23 STEP 1848 H= 462:00 +CPU= 35.534
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGh7zg+001848
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHh7zg+001848
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAh7zg+001848
11:58:53 STEP 1849 H= 462:15 +CPU= 29.891
11:59:22 STEP 1850 H= 462:30 +CPU= 29.669
11:59:51 STEP 1851 H= 462:45 +CPU= 28.976
12:00:22 STEP 1852 H= 463:00 +CPU= 30.449
12:00:52 STEP 1853 H= 463:15 +CPU= 29.333
12:01:22 STEP 1854 H= 463:30 +CPU= 29.234
12:01:51 STEP 1855 H= 463:45 +CPU= 29.099
12:02:21 STEP 1856 H= 464:00 +CPU= 29.717
12:02:51 STEP 1857 H= 464:15 +CPU= 29.515
12:03:22 STEP 1858 H= 464:30 +CPU= 29.536
12:03:53 STEP 1859 H= 464:45 +CPU= 28.933
12:04:30 STEP 1860 H= 465:00 +CPU= 35.975
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGh7zg+001860
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHh7zg+001860
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAh7zg+001860
12:05:00 STEP 1861 H= 465:15 +CPU= 28.857
12:05:30 STEP 1862 H= 465:30 +CPU= 28.485
12:05:59 STEP 1863 H= 465:45 +CPU= 29.246
12:06:30 STEP 1864 H= 466:00 +CPU= 30.576
12:07:01 STEP 1865 H= 466:15 +CPU= 29.464
12:07:31 STEP 1866 H= 466:30 +CPU= 30.011
12:08:02 STEP 1867 H= 466:45 +CPU= 29.440
12:08:32 STEP 1868 H= 467:00 +CPU= 30.251
12:09:02 STEP 1869 H= 467:15 +CPU= 29.286
12:09:31 STEP 1870 H= 467:30 +CPU= 29.268
12:10:01 STEP 1871 H= 467:45 +CPU= 29.365
12:10:41 STEP 1872 H= 468:00 +CPU= 39.121
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGh7zg+001872
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHh7zg+001872
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAh7zg+001872
12:11:09 STEP 1873 H= 468:15 +CPU= 28.764
12:11:39 STEP 1874 H= 468:30 +CPU= 29.574
12:12:09 STEP 1875 H= 468:45 +CPU= 29.879
12:12:39 STEP 1876 H= 469:00 +CPU= 29.930
12:13:08 STEP 1877 H= 469:15 +CPU= 29.195
12:13:38 STEP 1878 H= 469:30 +CPU= 29.469
12:14:08 STEP 1879 H= 469:45 +CPU= 29.395
12:14:38 STEP 1880 H= 470:00 +CPU= 29.751
12:15:06 STEP 1881 H= 470:15 +CPU= 28.647
12:15:36 STEP 1882 H= 470:30 +CPU= 30.107
12:16:06 STEP 1883 H= 470:45 +CPU= 29.041
12:16:43 STEP 1884 H= 471:00 +CPU= 36.666
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGh7zg+001884
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHh7zg+001884
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAh7zg+001884
12:17:13 STEP 1885 H= 471:15 +CPU= 29.064
12:17:44 STEP 1886 H= 471:30 +CPU= 29.883
12:18:14 STEP 1887 H= 471:45 +CPU= 29.954
12:18:45 STEP 1888 H= 472:00 +CPU= 29.919
12:19:14 STEP 1889 H= 472:15 +CPU= 28.833
12:19:43 STEP 1890 H= 472:30 +CPU= 28.872
12:20:13 STEP 1891 H= 472:45 +CPU= 29.355
12:20:43 STEP 1892 H= 473:00 +CPU= 29.947
12:21:13 STEP 1893 H= 473:15 +CPU= 28.921
12:21:42 STEP 1894 H= 473:30 +CPU= 29.253
12:22:12 STEP 1895 H= 473:45 +CPU= 29.706
12:22:48 STEP 1896 H= 474:00 +CPU= 35.260
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGh7zg+001896
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHh7zg+001896
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAh7zg+001896
12:23:18 STEP 1897 H= 474:15 +CPU= 29.473
12:23:48 STEP 1898 H= 474:30 +CPU= 29.292
12:24:18 STEP 1899 H= 474:45 +CPU= 29.382
12:24:48 STEP 1900 H= 475:00 +CPU= 29.902
12:25:17 STEP 1901 H= 475:15 +CPU= 29.365
12:25:47 STEP 1902 H= 475:30 +CPU= 29.500
12:26:17 STEP 1903 H= 475:45 +CPU= 28.780
12:26:47 STEP 1904 H= 476:00 +CPU= 30.354
12:27:17 STEP 1905 H= 476:15 +CPU= 29.056
12:27:47 STEP 1906 H= 476:30 +CPU= 29.215
12:28:16 STEP 1907 H= 476:45 +CPU= 28.974
12:28:53 STEP 1908 H= 477:00 +CPU= 36.210
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGh7zg+001908
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHh7zg+001908
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAh7zg+001908
12:29:23 STEP 1909 H= 477:15 +CPU= 29.664
12:29:54 STEP 1910 H= 477:30 +CPU= 30.090
12:30:24 STEP 1911 H= 477:45 +CPU= 29.349
12:30:55 STEP 1912 H= 478:00 +CPU= 30.036
12:31:25 STEP 1913 H= 478:15 +CPU= 29.637
12:31:55 STEP 1914 H= 478:30 +CPU= 29.492
12:32:24 STEP 1915 H= 478:45 +CPU= 29.364
12:32:54 STEP 1916 H= 479:00 +CPU= 30.002
12:33:25 STEP 1917 H= 479:15 +CPU= 29.976
12:33:54 STEP 1918 H= 479:30 +CPU= 29.359
12:34:24 STEP 1919 H= 479:45 +CPU= 29.164
12:35:01 STEP 1920 H= 480:00 +CPU= 37.620
..The child process terminated with status: 0
>>> Printing last 70 lines from file: NODE.001_01
DDH------------------------------------------------------------------------
I I
I TIME = 480.000H DDH * Diagnostic Var., AREA GLOBAL I
I I
I-------------------------------------------------------------------------I
I VARIABLE I Initial value I Current value I
I-------------------------------------------------------------------------I
I OMEGA (P/S) I -0.1128981122E-22 I 0.3609048673E-05 I
ICloud fractionI 0.0000000000E+00 I 0.3274999049E-01 I
I Relat. Hum. I 0.3423680658E+00 I 0.4010914769E+00 I
I PASS 01 I 0.0000000000E+00 I 0.5992883639E+03 I
DDH------------------------------------------------------------------------
MAXGPFV : MAX. VALUE = 0.000000000000000E+000
MAXGPFV : MAX. VALUE = 0.000000000000000E+000
NSTEP = 1920 SCAN2M_HPOS P
IO-STREAM SETUP - IOTYPE = 2 NUMIOPROCS = 1 CPATH =
ICMGGh7zg+001920 MODE=a
IO-STREAM CLOSED - ICMGGh7zg+001920
MPI-TASK: 1 - 302214196 BYTES IN 7 RECORDS TRANSFERRED IN 0.0005 SECONDS604428.4 Mbytes/s, TOTAL TIME= 0.0111( 4.5%)
12:35:01 STEP 1920 H= 480:00 +CPU= 37.620
END CNT3
NSTEP = 1920 CNT0 000000000 27.991 27.991 28.078 0.997 0 0 0 0 0
IO-STREAM STATISTICS, TOTAL NO OF REC/BYTES READ 46/ 1069125708 WRITTEN 5720/************
-TASK-OPENED-OPEN-RECS IN -KBYTE IN -RECS OUT-KBYTE OUT-WALL -WALL IN-WALL
OU-TOT IN -TOT OUT
1 103 0 46 1044068 5720 ************ 10.4 1.2 9.2 2.0 19.6
===-=== START OF TIMING STATISTICS ===-===
STATS FOR ALL TASKS
NUM ROUTINE CALLS MEAN(ms) MAX(ms) FRAC(%) UNBAL(%)
0 CNT0 - COMPLETE EXECUTION 1 5689008.9 5689008.9 100.00 0.00
1 CNT4 - FORWARD INTEGRATION 1 5685521.6 5685521.6 99.94 0.00
8 SCAN2M - GRID-POINT DYNAMICS 209 5479.6 5479.6 20.13 0.00
9 SPCM - SPECTRAL COMP. 192 947.6 947.6 3.20 0.00
10 SCAN2M - PHYSICS 193 12854.6 12854.6 43.61 0.00
11 IOPACK - OUTPUT P.P. RESULTS 16 859.3 859.3 0.24 0.00
12 SPNORM - SPECTRAL NORM COMP. 9 81.2 81.2 0.01 0.00
17 GRIDFPOS IN CNT4 16 29.2 29.2 0.01 0.00
27 RESTART FILES - WRITING 4 3437.3 3437.3 0.24 0.00
28 RESTART FILES - READING 1 2974.3 2974.3 0.05 0.00
29 SU4FPOS IN CNT4 16 0.1 0.1 0.00 0.00
30 DYNFPOS IN CNT4 16 5730.7 5730.7 1.61 0.00
31 POSDDH IN STEPO 1 10.4 10.4 0.00 0.00
37 CPGLAG - SL COMPUTATIONS 193 6111.7 6111.7 20.73 0.00
39 SU0YOMB 1 273.3 273.3 0.00 0.00
51 SCAN2M - SL COMM. PART 1 193 31.0 31.0 0.11 0.00
54 SPCM - M TO S/S TO M TRANSP. 192 569.2 569.2 1.92 0.00
55 SPCIMPF - S TO M/M TO S TRANSP. 192 70.7 70.7 0.24 0.00
56 SPNORM - SPECTRAL NORM COMM. 9 0.3 0.3 0.00 0.00
102 LTINV_CTL - INVERSE LEGENDRE TRANSFORM 420 407.6 407.6 3.01 0.00
103 LTDIR_CTL - DIRECT LEGENDRE TRANSFORM 400 235.6 235.6 1.66 0.00
106 FTDIR_CTL - DIRECT FOURIER TRANSFORM 400 78.8 78.8 0.55 0.00
107 FTINV_CTL - INVERSE FOURIER TRANSFORM 420 112.7 112.7 0.83 0.00
140 SULEG - COMP. OF LEGENDRE POL. 1 35.4 35.4 0.00 0.00
152 LTINV_CTL - M TO L TRANSPOSITION 420 64.8 64.8 0.48 0.00
153 LTDIR_CTL - L TO M TRANSPOSITION 400 36.0 36.0 0.25 0.00
157 FTINV_CTL - L TO G TRANSPOSITION 420 137.6 137.6 1.02 0.00
158 FTDIR_CTL - G TO L TRANSPOSITION 400 61.5 61.5 0.43 0.00
400 GSTATS 23311 0.0 0.0 0.00 0.00
401 GSTATS HOOK 21635 0.0 0.0 0.00 0.00
TOTAL MEASURED IMBALANCE = 0.0 SECONDS, 0.0 PERCENT
TOTAL WALLCLOCK TIME 5689.0 CPU TIME 5614.7 VECTOR TIME 5614.7
===-=== END OF TIMING STATISTICS ===-===
FORECAST DAYS PER DAY 303.9
*** END CNT0 ***
------------------------------------------------
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGh7zg+001920
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHh7zg+001920
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAh7zg+001920
Adding to the zip: /var/lib/boinc-client/slots/14/NODE.001_01
Adding to the zip: /var/lib/boinc-client/slots/14/ifs.stat
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290336/ICMGGh7zg+000864
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290336/ICMGGh7zg+001884
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290336/ICMSHh7zg+001920
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290336/ICMSHh7zg+001884
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290336/ICMUAh7zg+001884
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290336/ICMUAh7zg+001872
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290336/ICMGGh7zg+001440
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290336/ICMGGh7zg+001896
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290336/ICMUAh7zg+001896
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290336/ICMUAh7zg+001908
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290336/ICMUAh7zg+001860
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290336/ICMSHh7zg+001440
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290336/ICMUAh7zg+000864
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290336/ICMGGh7zg+001908
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290336/ICMSHh7zg+000864
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290336/ICMSHh7zg+001908
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290336/ICMSHh7zg+001728
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290336/ICMUAh7zg+001836
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290336/ICMSHh7zg+001848
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290336/ICMGGh7zg+001848
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290336/ICMGGh7zg+001920
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290336/ICMGGh7zg+001872
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290336/ICMGGh7zg+001728
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290336/ICMSHh7zg+001860
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290336/ICMSHh7zg+001872
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290336/ICMSHh7zg+001896
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290336/ICMUAh7zg+001920
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290336/ICMUAh7zg+001440
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290336/ICMGGh7zg+001836
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290336/ICMGGh7zg+001860
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290336/ICMSHh7zg+001836
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290336/ICMUAh7zg+001728
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290336/ICMUAh7zg+001848
Zipping up the final file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a1jt_2016092300_20_1018_12290336_0_r1764552379_17.zip
Uploading the final file: upload_file_17.zip
Uploading trickle at timestep: 1727100
12:38:32 (906694): called boinc_finish(0)
</stderr_txt>
<message>
upload failure: <file_xfer_error>
<file_name>oifs_43r3_bl_a1jt_2016092300_20_1018_12290336_0_r1764552379_18.zip</file_name>
<error_code>-161 (not found)</error_code>
</file_xfer_error>
<file_xfer_error>
<file_name>oifs_43r3_bl_a1jt_2016092300_20_1018_12290336_0_r1764552379_19.zip</file_name>
<error_code>-161 (not found)</error_code>
</file_xfer_error>
</message>
]]>
