<core_client_version>7.22.0</core_client_version>
<![CDATA[
<message>
process exited with code 9 (0x9, -247)</message>
<stderr_txt>
he child process
18:57:57 STEP 79 H= 79:00 +CPU= 7.598
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
18:58:13 STEP 80 H= 80:00 +CPU= 7.862
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
18:58:33 STEP 81 H= 81:00 +CPU= 12.119
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
18:58:46 STEP 82 H= 82:00 +CPU= 7.758
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
18:58:58 STEP 83 H= 83:00 +CPU= 7.579
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
18:59:21 STEP 84 H= 84:00 +CPU= 13.315
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
18:59:33 STEP 85 H= 85:00 +CPU= 7.554
Moving to projects directory: /home/stefan/slots/4/ICMGGhq0f+000084
Moving to projects directory: /home/stefan/slots/4/ICMSHhq0f+000084
Moving to projects directory: /home/stefan/slots/4/ICMUAhq0f+000084
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
18:59:49 STEP 86 H= 86:00 +CPU= 7.826
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
[EC_DRHOOK:stefan-Z690-UD:1:1:1122:1122] [20230124:190003:1674583203:1418.143] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1538] Received signal#15 (SIGTERM) :: 3992MB (heap), 4344MB (maxrss), 0MB (maxstack), 0 (paging), nsigs = 1
[EC_DRHOOK:stefan-Z690-UD:1:1:1122:1122] [20230124:190003:1674583203:1418.143] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1542] Also activating Harakiri-alarm (SIGALRM=14) to expire after 500s elapsed to prevent hangs, nsigs = 1
[EC_DRHOOK:stefan-Z690-UD:1:1:1122:1122] [20230124:190003:1674583203:1418.143] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1544] Harakiri signal handler 'signal_harakiri' for signal#14 (SIGALRM) installed at 0x81f3c0 (old at (nil))
[EC_DRHOOK:stefan-Z690-UD:1:1:1122:1122] [20230124:190003:1674583203:1418.143] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1617] Signal#15 was caused by unrecognized si_code [memaddr=0x3e800000000], nsigs = 1
[EC_DRHOOK:stefan-Z690-UD:1:1:1122:1122] [20230124:190003:1674583203:1418.143] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1686] Starting DrHook backtrace for signal#15, nsigs = 1
[EC_DRHOOK:stefan-Z690-UD:1:1:1122:1122] [20230124:190003:1674583203:1418.143] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3843] 3992 MB (maxheap), 4344 MB (maxrss), 0 MB (maxstack)
[EC_DRHOOK:stefan-Z690-UD:1:1:1122:1122] [20230124:190003:1674583203:1418.143] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : MASTER
[EC_DRHOOK:stefan-Z690-UD:1:1:1122:1122] [20230124:190003:1674583203:1418.143] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT0
[EC_DRHOOK:stefan-Z690-UD:1:1:1122:1122] [20230124:190003:1674583203:1418.144] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT1
[EC_DRHOOK:stefan-Z690-UD:1:1:1122:1122] [20230124:190003:1674583203:1418.144] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT2
[EC_DRHOOK:stefan-Z690-UD:1:1:1122:1122] [20230124:190003:1674583203:1418.144] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT3
[EC_DRHOOK:stefan-Z690-UD:1:1:1122:1122] [20230124:190003:1674583203:1418.144] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT4
[EC_DRHOOK:stefan-Z690-UD:1:1:1122:1122] [20230124:190003:1674583203:1418.144] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : STEPO
[EC_DRHOOK:stefan-Z690-UD:1:1:1122:1122] [20230124:190003:1674583203:1418.144] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : SCAN2M
[EC_DRHOOK:stefan-Z690-UD:1:1:1122:1122] [20230124:190003:1674583203:1418.144] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : GP_MODEL_HEAP
[EC_DRHOOK:stefan-Z690-UD:1:1:1122:1122] [20230124:190003:1674583203:1418.144] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : GP_MODEL
[EC_DRHOOK:stefan-Z690-UD:1:1:1122:1122] [20230124:190003:1674583203:1418.144] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : EC_PHYS_DRV
[EC_DRHOOK:stefan-Z690-UD:1:1:1122:1122] [20230124:190003:1674583203:1418.144] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : >OMP-PHYSICS CLDPP T/S (1002)
[EC_DRHOOK:stefan-Z690-UD:1:1:1122:1122] [20230124:190003:1674583203:1418.144] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : EC_PHYS
[EC_DRHOOK:stefan-Z690-UD:1:1:1122:1122] [20230124:190003:1674583203:1418.144] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CALLPAR
[EC_DRHOOK:stefan-Z690-UD:1:1:1122:1122] [20230124:190003:1674583203:1418.144] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : TURBULENCE_LAYER
[EC_DRHOOK:stefan-Z690-UD:1:1:1122:1122] [20230124:190003:1674583203:1418.144] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : VDFOUTER
[EC_DRHOOK:stefan-Z690-UD:1:1:1122:1122] [20230124:190003:1674583203:1418.144] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : VDFMAIN
[EC_DRHOOK:stefan-Z690-UD:1:1:1122:1122] [20230124:190004:1674583204:1419.144] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1734] DrHook backtrace done for signal#15, nsigs = 1
[EC_DRHOOK:stefan-Z690-UD:1:1:1122:1122] [20230124:190004:1674583204:1419.144] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1785] Calling previous signal handler at 0x1ce8bb0 for signal#15, nsigs = 1
forrtl: error (78): process killed (SIGTERM)
Image PC Routine Line Source
oifs_43r3_model.e 0000000001CE916B Unknown Unknown Unknown
oifs_43r3_model.e 000000000081FFF1 Unknown Unknown Unknown
oifs_43r3_model.e 0000000001DC9090 Unknown Unknown Unknown
oifs_43r3_model.e 0000000000FAA5EA vdfmain_ 874 vdfmain.F90
oifs_43r3_model.e 0000000000F92699 vdfouter_ 739 vdfouter.F90
oifs_43r3_model.e 0000000000F88512 turbulence_layer_ 158 turbulence_layer.F90
oifs_43r3_model.e 0000000000F0B683 callpar_ 869 callpar.F90
oifs_43r3_model.e 0000000000E4940F ec_phys_ 670 ec_phys.F90
oifs_43r3_model.e 0000000000E3610C ec_phys_drv_ 599 ec_phys_drv.F90
oifs_43r3_model.e 0000000000BFA3D3 gp_model_ 613 gp_model.F90
oifs_43r3_model.e 00000000012E9482 gp_model_heap_ 74 gp_model_heap.F90
oifs_43r3_model.e 0000000000BBC1CE scan2m_ 535 scan2m.F90
oifs_43r3_model.e 000000000057A664 stepo_ 327 stepo.F90
oifs_43r3_model.e 000000000055DEEC cnt4_ 1133 cnt4.F90
oifs_43r3_model.e 00000000005471C9 cnt3_ 267 cnt3.F90
oifs_43r3_model.e 0000000000546412 cnt2_ 88 cnt2.F90
oifs_43r3_model.e 0000000000545E28 cnt1_ 92 cnt1.F90
oifs_43r3_model.e 000000000040708D cnt0_ 146 cnt0.F90
oifs_43r3_model.e 000000000040220F MAIN__ 96 master.F90
oifs_43r3_model.e 00000000004021A2 Unknown Unknown Unknown
oifs_43r3_model.e 0000000001DCA390 Unknown Unknown Unknown
oifs_43r3_model.e 000000000040206E Unknown Unknown Unknown
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 2003050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0181
(argv4) batchid: 972
(argv5) wuid: 12188825
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /home/stefan/slots/4
Project directory is: /home/stefan/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_12188825
..mkdir for temp folder for results failed
Copying: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /home/stefan/slots/4/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /home/stefan/slots/4/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_943de426e6fe821666cd1e1c0f309465 to: /home/stefan/slots/4/oifs_43r3_ps_0181_2003050100_123_972_12188825.zip
Unzipping the namelist zip file: /home/stefan/slots/4/oifs_43r3_ps_0181_2003050100_123_972_12188825.zip
ic_ancil_file: ic_ancil_12188825
ifsdata_file: ifsdata_12188825
climate_data_file: clim_data_12188825
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_170d1551d97285d1db6fa8a53abf160c to: /home/stefan/slots/4/ic_ancil_12188825.zip
Unzipping the IC ancils zip file: /home/stefan/slots/4/ic_ancil_12188825.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_b440f6f96d74acf6a7feb76499bffa6b to: /home/stefan/slots/4/ifsdata/ifsdata_12188825.zip
Unzipping the ifsdata_zip file: /home/stefan/slots/4/ifsdata/ifsdata_12188825.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /home/stefan/slots/4/159l_2/clim_data_12188825.zip
Unzipping the climate data zip file: /home/stefan/slots/4/159l_2/clim_data_12188825.zip
Checking for progress XML file: /home/stefan/slots/4/progress_file_12188825.xml
Opened progress file ok : /home/stefan/slots/4/progress_file_12188825.xml
-- Model is restarting --
Adjusting last_iter, 87, to previous model restart step.
Creating progress file: /home/stefan/slots/4/progress_file_12188825.xml
last_cpu_time: 837
upload_file_number: 3
last_iter: 71
last_upload: 259200
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0181_2003050100_123_972_12188825_1_r51269030
The child process has been launched with process id: 95
Executing the command: /home/stefan/slots/4/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:stefan-Z690-UD:1:1:95:95] [20230125:153846:1674657526:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:stefan-Z690-UD:1:1:95:95] [20230125:153846:1674657526:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:stefan-Z690-UD:1:1:95:95] [20230125:153846:1674657526:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:stefan-Z690-UD:1:1:95:95] [20230125:153846:1674657526:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:stefan-Z690-UD:1:1:95:95] [20230125:153846:1674657526:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:stefan-Z690-UD:1:1:95:95] [20230125:153846:1674657526:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647)
[EC_DRHOOK:stefan-Z690-UD:1:1:95:95] [20230125:153846:1674657526:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:stefan-Z690-UD:1:1:95:95] [20230125:153846:1674657526:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:stefan-Z690-UD:1:1:95:95] [20230125:153846:1674657526:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:stefan-Z690-UD:1:1:95:95] [20230125:153846:1674657526:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:stefan-Z690-UD:1:1:95:95] [20230125:153846:1674657526:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:stefan-Z690-UD:1:1:95:95] [20230125:153846:1674657526:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:stefan-Z690-UD:1:1:95:95] [20230125:153846:1674657526:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:stefan-Z690-UD:1:1:95:95] [20230125:153846:1674657526:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:stefan-Z690-UD:1:1:95:95] [20230125:153846:1674657526:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:stefan-Z690-UD:1:1:95:95] [20230125:153846:1674657526:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:stefan-Z690-UD:1:1:95:95] [20230125:153846:1674657526:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:stefan-Z690-UD:1:1:95:95] [20230125:153846:1674657526:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:stefan-Z690-UD:1:1:95:95] [20230125:153846:1674657526:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:stefan-Z690-UD:1:1:95:95] [20230125:153846:1674657526:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:stefan-Z690-UD:1:1:95:95] [20230125:153846:1674657526:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:stefan-Z690-UD:1:1:95:95] [20230125:153846:1674657526:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:stefan-Z690-UD:1:1:95:95] [20230125:153846:1674657526:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:stefan-Z690-UD:1:1:95:95] [20230125:153846:1674657526:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:stefan-Z690-UD:1:1:95:95] [20230125:153846:1674657526:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:stefan-Z690-UD:1:1:95:95] [20230125:153846:1674657526:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:stefan-Z690-UD:1:1:95:95] [20230125:153846:1674657526:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:stefan-Z690-UD:1:1:95:95] [20230125:153846:1674657526:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at 0x1)
[EC_DRHOOK:stefan-Z690-UD:1:1:95:95] [20230125:153846:1674657526:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:stefan-Z690-UD:1:1:95:95] [20230125:153846:1674657526:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
15:39:01 STEP 72 H= 72:00 +CPU= 13.496
15:39:09 STEP 73 H= 73:00 +CPU= 7.590
15:39:16 STEP 74 H= 74:00 +CPU= 7.730
15:39:28 STEP 75 H= 75:00 +CPU= 12.080
15:39:36 STEP 76 H= 76:00 +CPU= 7.782
15:39:44 STEP 77 H= 77:00 +CPU= 7.505
15:39:56 STEP 78 H= 78:00 +CPU= 12.211
15:40:04 STEP 79 H= 79:00 +CPU= 7.885
15:40:12 STEP 80 H= 80:00 +CPU= 8.095
15:40:25 STEP 81 H= 81:00 +CPU= 12.578
15:40:33 STEP 82 H= 82:00 +CPU= 8.106
15:40:41 STEP 83 H= 83:00 +CPU= 7.912
15:40:55 STEP 84 H= 84:00 +CPU= 13.883
Moving to projects directory: /home/stefan/slots/4/ICMGGhq0f+000084
Moving to projects directory: /home/stefan/slots/4/ICMSHhq0f+000084
Moving to projects directory: /home/stefan/slots/4/ICMUAhq0f+000084
15:41:03 STEP 85 H= 85:00 +CPU= 7.915
15:41:11 STEP 86 H= 86:00 +CPU= 8.146
15:41:24 STEP 87 H= 87:00 +CPU= 12.567
15:41:32 STEP 88 H= 88:00 +CPU= 8.161
15:41:40 STEP 89 H= 89:00 +CPU= 7.971
15:41:53 STEP 90 H= 90:00 +CPU= 12.826
15:42:01 STEP 91 H= 91:00 +CPU= 7.967
15:42:09 STEP 92 H= 92:00 +CPU= 8.161
15:42:22 STEP 93 H= 93:00 +CPU= 12.613
15:42:30 STEP 94 H= 94:00 +CPU= 8.186
15:42:39 STEP 95 H= 95:00 +CPU= 8.193
15:42:54 STEP 96 H= 96:00 +CPU= 14.988
Moving to projects directory: /home/stefan/slots/4/ICMGGhq0f+000096
Moving to projects directory: /home/stefan/slots/4/ICMSHhq0f+000096
Moving to projects directory: /home/stefan/slots/4/ICMUAhq0f+000096
Adding to the zip: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_12188825/ICMGGhq0f+000072
Adding to the zip: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_12188825/ICMSHhq0f+000072
Adding to the zip: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_12188825/ICMUAhq0f+000072
Adding to the zip: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_12188825/ICMGGhq0f+000084
Adding to the zip: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_12188825/ICMSHhq0f+000084
Adding to the zip: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_12188825/ICMUAhq0f+000084
Zipping up the intermediate file: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_0181_2003050100_123_972_12188825_1_r51269030_3.zip
Uploading the intermediate file: upload_file_3.zip
15:43:02 STEP 97 H= 97:00 +CPU= 7.925
15:43:10 STEP 98 H= 98:00 +CPU= 8.121
15:43:23 STEP 99 H= 99:00 +CPU= 12.592
15:43:31 STEP 100 H= 100:00 +CPU= 8.139
15:43:39 STEP 101 H= 101:00 +CPU= 7.928
15:43:52 STEP 102 H= 102:00 +CPU= 12.871
15:44:00 STEP 103 H= 103:00 +CPU= 7.939
15:44:08 STEP 104 H= 104:00 +CPU= 8.136
15:44:21 STEP 105 H= 105:00 +CPU= 12.595
15:44:29 STEP 106 H= 106:00 +CPU= 8.142
15:44:37 STEP 107 H= 107:00 +CPU= 7.916
15:44:51 STEP 108 H= 108:00 +CPU= 13.950
Moving to projects directory: /home/stefan/slots/4/ICMGGhq0f+000108
Moving to projects directory: /home/stefan/slots/4/ICMSHhq0f+000108
Moving to projects directory: /home/stefan/slots/4/ICMUAhq0f+000108
15:44:59 STEP 109 H= 109:00 +CPU= 7.990
15:45:08 STEP 110 H= 110:00 +CPU= 8.202
15:45:20 STEP 111 H= 111:00 +CPU= 12.599
15:45:29 STEP 112 H= 112:00 +CPU= 8.160
15:45:37 STEP 113 H= 113:00 +CPU= 7.956
15:45:50 STEP 114 H= 114:00 +CPU= 12.870
15:45:58 STEP 115 H= 115:00 +CPU= 7.976
15:46:06 STEP 116 H= 116:00 +CPU= 8.156
15:46:19 STEP 117 H= 117:00 +CPU= 12.585
15:46:27 STEP 118 H= 118:00 +CPU= 8.155
15:46:35 STEP 119 H= 119:00 +CPU= 8.150
15:46:50 STEP 120 H= 120:00 +CPU= 14.992
Moving to projects directory: /home/stefan/slots/4/ICMGGhq0f+000120
Moving to projects directory: /home/stefan/slots/4/ICMSHhq0f+000120
Moving to projects directory: /home/stefan/slots/4/ICMUAhq0f+000120
Adding to the zip: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_12188825/ICMGGhq0f+000096
Adding to the zip: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_12188825/ICMSHhq0f+000096
Adding to the zip: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_12188825/ICMUAhq0f+000096
Adding to the zip: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_12188825/ICMGGhq0f+000108
Adding to the zip: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_12188825/ICMSHhq0f+000108
Adding to the zip: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_12188825/ICMUAhq0f+000108
Zipping up the intermediate file: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_0181_2003050100_123_972_12188825_1_r51269030_4.zip
Uploading the intermediate file: upload_file_4.zip
15:46:58 STEP 121 H= 121:00 +CPU= 7.932
15:47:06 STEP 122 H= 122:00 +CPU= 8.146
15:47:19 STEP 123 H= 123:00 +CPU= 12.627
15:47:27 STEP 124 H= 124:00 +CPU= 8.165
15:47:35 STEP 125 H= 125:00 +CPU= 7.953
15:47:48 STEP 126 H= 126:00 +CPU= 12.845
15:47:57 STEP 127 H= 127:00 +CPU= 8.029
15:48:05 STEP 128 H= 128:00 +CPU= 8.196
15:48:18 STEP 129 H= 129:00 +CPU= 12.610
15:48:26 STEP 130 H= 130:00 +CPU= 8.153
15:48:34 STEP 131 H= 131:00 +CPU= 7.979
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 2003050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0181
(argv4) batchid: 972
(argv5) wuid: 12188825
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /home/stefan/slots/4
Project directory is: /home/stefan/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_12188825
..mkdir for temp folder for results failed
Copying: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /home/stefan/slots/4/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /home/stefan/slots/4/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_943de426e6fe821666cd1e1c0f309465 to: /home/stefan/slots/4/oifs_43r3_ps_0181_2003050100_123_972_12188825.zip
Unzipping the namelist zip file: /home/stefan/slots/4/oifs_43r3_ps_0181_2003050100_123_972_12188825.zip
ic_ancil_file: ic_ancil_12188825
ifsdata_file: ifsdata_12188825
climate_data_file: clim_data_12188825
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_170d1551d97285d1db6fa8a53abf160c to: /home/stefan/slots/4/ic_ancil_12188825.zip
Unzipping the IC ancils zip file: /home/stefan/slots/4/ic_ancil_12188825.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_b440f6f96d74acf6a7feb76499bffa6b to: /home/stefan/slots/4/ifsdata/ifsdata_12188825.zip
Unzipping the ifsdata_zip file: /home/stefan/slots/4/ifsdata/ifsdata_12188825.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /home/stefan/slots/4/159l_2/clim_data_12188825.zip
Unzipping the climate data zip file: /home/stefan/slots/4/159l_2/clim_data_12188825.zip
Checking for progress XML file: /home/stefan/slots/4/progress_file_12188825.xml
Opened progress file ok : /home/stefan/slots/4/progress_file_12188825.xml
-- Model is restarting --
Adjusting last_iter, 132, to previous model restart step.
Creating progress file: /home/stefan/slots/4/progress_file_12188825.xml
last_cpu_time: 1422
upload_file_number: 5
last_iter: 119
last_upload: 432000
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0181_2003050100_123_972_12188825_1_r51269030
The child process has been launched with process id: 81
Executing the command: /home/stefan/slots/4/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:stefan-Z690-UD:1:1:81:81] [20230125:155016:1674658216:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:stefan-Z690-UD:1:1:81:81] [20230125:155016:1674658216:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:stefan-Z690-UD:1:1:81:81] [20230125:155016:1674658216:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:stefan-Z690-UD:1:1:81:81] [20230125:155016:1674658216:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:stefan-Z690-UD:1:1:81:81] [20230125:155016:1674658216:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:stefan-Z690-UD:1:1:81:81] [20230125:155016:1674658216:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647)
[EC_DRHOOK:stefan-Z690-UD:1:1:81:81] [20230125:155016:1674658216:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:stefan-Z690-UD:1:1:81:81] [20230125:155016:1674658216:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:stefan-Z690-UD:1:1:81:81] [20230125:155016:1674658216:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:stefan-Z690-UD:1:1:81:81] [20230125:155016:1674658216:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:stefan-Z690-UD:1:1:81:81] [20230125:155016:1674658216:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:stefan-Z690-UD:1:1:81:81] [20230125:155016:1674658216:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:stefan-Z690-UD:1:1:81:81] [20230125:155016:1674658216:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:stefan-Z690-UD:1:1:81:81] [20230125:155016:1674658216:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:stefan-Z690-UD:1:1:81:81] [20230125:155016:1674658216:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:stefan-Z690-UD:1:1:81:81] [20230125:155016:1674658216:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:stefan-Z690-UD:1:1:81:81] [20230125:155016:1674658216:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:stefan-Z690-UD:1:1:81:81] [20230125:155016:1674658216:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:stefan-Z690-UD:1:1:81:81] [20230125:155016:1674658216:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:stefan-Z690-UD:1:1:81:81] [20230125:155016:1674658216:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:stefan-Z690-UD:1:1:81:81] [20230125:155016:1674658216:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:stefan-Z690-UD:1:1:81:81] [20230125:155016:1674658216:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:stefan-Z690-UD:1:1:81:81] [20230125:155016:1674658216:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:stefan-Z690-UD:1:1:81:81] [20230125:155016:1674658216:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:stefan-Z690-UD:1:1:81:81] [20230125:155016:1674658216:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:stefan-Z690-UD:1:1:81:81] [20230125:155016:1674658216:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:stefan-Z690-UD:1:1:81:81] [20230125:155016:1674658216:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:stefan-Z690-UD:1:1:81:81] [20230125:155016:1674658216:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at 0x1)
[EC_DRHOOK:stefan-Z690-UD:1:1:81:81] [20230125:155016:1674658216:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:stefan-Z690-UD:1:1:81:81] [20230125:155016:1674658216:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
15:50:32 STEP 120 H= 120:00 +CPU= 14.257
15:50:40 STEP 121 H= 121:00 +CPU= 8.040
15:50:48 STEP 122 H= 122:00 +CPU= 8.258
15:51:01 STEP 123 H= 123:00 +CPU= 12.756
15:51:09 STEP 124 H= 124:00 +CPU= 8.207
15:51:18 STEP 125 H= 125:00 +CPU= 8.076
15:51:31 STEP 126 H= 126:00 +CPU= 12.990
15:51:39 STEP 127 H= 127:00 +CPU= 8.035
15:51:47 STEP 128 H= 128:00 +CPU= 8.227
15:52:00 STEP 129 H= 129:00 +CPU= 12.711
15:52:08 STEP 130 H= 130:00 +CPU= 8.248
15:52:16 STEP 131 H= 131:00 +CPU= 8.053
15:52:31 STEP 132 H= 132:00 +CPU= 14.046
Moving to projects directory: /home/stefan/slots/4/ICMGGhq0f+000132
Moving to projects directory: /home/stefan/slots/4/ICMSHhq0f+000132
Moving to projects directory: /home/stefan/slots/4/ICMUAhq0f+000132
15:52:39 STEP 133 H= 133:00 +CPU= 8.050
15:52:47 STEP 134 H= 134:00 +CPU= 8.253
15:53:00 STEP 135 H= 135:00 +CPU= 12.741
15:53:08 STEP 136 H= 136:00 +CPU= 8.248
15:53:17 STEP 137 H= 137:00 +CPU= 8.039
15:53:30 STEP 138 H= 138:00 +CPU= 13.031
15:53:38 STEP 139 H= 139:00 +CPU= 8.063
15:53:46 STEP 140 H= 140:00 +CPU= 8.221
15:53:59 STEP 141 H= 141:00 +CPU= 12.694
15:54:07 STEP 142 H= 142:00 +CPU= 8.220
15:54:16 STEP 143 H= 143:00 +CPU= 8.233
15:54:31 STEP 144 H= 144:00 +CPU= 15.146
Moving to projects directory: /home/stefan/slots/4/ICMGGhq0f+000144
Moving to projects directory: /home/stefan/slots/4/ICMSHhq0f+000144
Moving to projects directory: /home/stefan/slots/4/ICMUAhq0f+000144
Adding to the zip: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_12188825/ICMGGhq0f+000120
Adding to the zip: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_12188825/ICMSHhq0f+000120
Adding to the zip: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_12188825/ICMUAhq0f+000120
Adding to the zip: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_12188825/ICMGGhq0f+000132
Adding to the zip: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_12188825/ICMSHhq0f+000132
Adding to the zip: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_12188825/ICMUAhq0f+000132
Zipping up the intermediate file: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_0181_2003050100_123_972_12188825_1_r51269030_5.zip
Uploading the intermediate file: upload_file_5.zip
15:54:39 STEP 145 H= 145:00 +CPU= 8.033
15:54:48 STEP 146 H= 146:00 +CPU= 8.254
15:55:01 STEP 147 H= 147:00 +CPU= 12.765
15:55:09 STEP 148 H= 148:00 +CPU= 8.240
15:55:17 STEP 149 H= 149:00 +CPU= 8.021
15:55:30 STEP 150 H= 150:00 +CPU= 12.935
15:55:38 STEP 151 H= 151:00 +CPU= 8.032
15:55:47 STEP 152 H= 152:00 +CPU= 8.243
15:56:00 STEP 153 H= 153:00 +CPU= 12.793
15:56:08 STEP 154 H= 154:00 +CPU= 8.235
15:56:16 STEP 155 H= 155:00 +CPU= 8.034
15:56:30 STEP 156 H= 156:00 +CPU= 14.020
Moving to projects directory: /home/stefan/slots/4/ICMGGhq0f+000156
Moving to projects directory: /home/stefan/slots/4/ICMSHhq0f+000156
Moving to projects directory: /home/stefan/slots/4/ICMUAhq0f+000156
15:56:38 STEP 157 H= 157:00 +CPU= 8.039
15:56:47 STEP 158 H= 158:00 +CPU= 8.204
15:57:00 STEP 159 H= 159:00 +CPU= 12.720
15:57:08 STEP 160 H= 160:00 +CPU= 8.280
15:57:16 STEP 161 H= 161:00 +CPU= 8.055
15:57:29 STEP 162 H= 162:00 +CPU= 12.977
15:57:37 STEP 163 H= 163:00 +CPU= 8.065
15:57:46 STEP 164 H= 164:00 +CPU= 8.267
15:57:59 STEP 165 H= 165:00 +CPU= 12.781
15:58:07 STEP 166 H= 166:00 +CPU= 8.231
15:58:15 STEP 167 H= 167:00 +CPU= 8.224
15:58:31 STEP 168 H= 168:00 +CPU= 15.229
Moving to projects directory: /home/stefan/slots/4/ICMGGhq0f+000168
Moving to projects directory: /home/stefan/slots/4/ICMSHhq0f+000168
Moving to projects directory: /home/stefan/slots/4/ICMUAhq0f+000168
Adding to the zip: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_12188825/ICMGGhq0f+000144
Adding to the zip: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_12188825/ICMSHhq0f+000144
Adding to the zip: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_12188825/ICMUAhq0f+000144
Adding to the zip: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_12188825/ICMGGhq0f+000156
Adding to the zip: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_12188825/ICMSHhq0f+000156
Adding to the zip: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_12188825/ICMUAhq0f+000156
Zipping up the intermediate file: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_0181_2003050100_123_972_12188825_1_r51269030_6.zip
Uploading the intermediate file: upload_file_6.zip
15:58:39 STEP 169 H= 169:00 +CPU= 7.997
15:58:47 STEP 170 H= 170:00 +CPU= 8.198
15:59:00 STEP 171 H= 171:00 +CPU= 12.697
15:59:08 STEP 172 H= 172:00 +CPU= 8.211
15:59:16 STEP 173 H= 173:00 +CPU= 7.716
15:59:29 STEP 174 H= 174:00 +CPU= 12.443
15:59:36 STEP 175 H= 175:00 +CPU= 7.699
15:59:44 STEP 176 H= 176:00 +CPU= 7.844
15:59:56 STEP 177 H= 177:00 +CPU= 12.192
16:00:04 STEP 178 H= 178:00 +CPU= 7.855
16:00:12 STEP 179 H= 179:00 +CPU= 7.660
16:00:25 STEP 180 H= 180:00 +CPU= 13.364
Moving to projects directory: /home/stefan/slots/4/ICMGGhq0f+000180
Moving to projects directory: /home/stefan/slots/4/ICMSHhq0f+000180
Moving to projects directory: /home/stefan/slots/4/ICMUAhq0f+000180
16:00:33 STEP 181 H= 181:00 +CPU= 7.621
16:00:41 STEP 182 H= 182:00 +CPU= 7.791
16:00:53 STEP 183 H= 183:00 +CPU= 12.039
16:01:01 STEP 184 H= 184:00 +CPU= 7.828
16:01:08 STEP 185 H= 185:00 +CPU= 7.618
16:01:21 STEP 186 H= 186:00 +CPU= 12.408
16:01:28 STEP 187 H= 187:00 +CPU= 7.715
16:01:36 STEP 188 H= 188:00 +CPU= 7.798
16:01:48 STEP 189 H= 189:00 +CPU= 12.069
16:01:56 STEP 190 H= 190:00 +CPU= 7.766
16:02:04 STEP 191 H= 191:00 +CPU= 7.799
16:02:18 STEP 192 H= 192:00 +CPU= 14.598
16:02:26 STEP 193 H= 193:00 +CPU= 7.687
Moving to projects directory: /home/stefan/slots/4/ICMGGhq0f+000192
Moving to projects directory: /home/stefan/slots/4/ICMSHhq0f+000192
Moving to projects directory: /home/stefan/slots/4/ICMUAhq0f+000192
Adding to the zip: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_12188825/ICMGGhq0f+000168
Adding to the zip: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_12188825/ICMSHhq0f+000168
Adding to the zip: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_12188825/ICMUAhq0f+000168
Adding to the zip: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_12188825/ICMGGhq0f+000180
Adding to the zip: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_12188825/ICMSHhq0f+000180
Adding to the zip: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_12188825/ICMUAhq0f+000180
Zipping up the intermediate file: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_0181_2003050100_123_972_12188825_1_r51269030_7.zip
Uploading the intermediate file: upload_file_7.zip
16:02:34 STEP 194 H= 194:00 +CPU= 7.927
16:02:46 STEP 195 H= 195:00 +CPU= 12.413
16:02:54 STEP 196 H= 196:00 +CPU= 7.917
16:03:02 STEP 197 H= 197:00 +CPU= 7.663
16:03:14 STEP 198 H= 198:00 +CPU= 12.522
16:03:22 STEP 199 H= 199:00 +CPU= 7.585
16:03:30 STEP 200 H= 200:00 +CPU= 7.797
16:03:42 STEP 201 H= 201:00 +CPU= 12.176
16:03:50 STEP 202 H= 202:00 +CPU= 7.929
16:03:58 STEP 203 H= 203:00 +CPU= 7.781
16:04:11 STEP 204 H= 204:00 +CPU= 13.555
Moving to projects directory: /home/stefan/slots/4/ICMGGhq0f+000204
Moving to projects directory: /home/stefan/slots/4/ICMSHhq0f+000204
Moving to projects directory: /home/stefan/slots/4/ICMUAhq0f+000204
16:04:19 STEP 205 H= 205:00 +CPU= 7.695
16:04:27 STEP 206 H= 206:00 +CPU= 8.034
16:04:40 STEP 207 H= 207:00 +CPU= 12.762
16:04:48 STEP 208 H= 208:00 +CPU= 8.276
16:04:57 STEP 209 H= 209:00 +CPU= 8.032
16:05:10 STEP 210 H= 210:00 +CPU= 13.006
16:05:18 STEP 211 H= 211:00 +CPU= 8.056
16:05:26 STEP 212 H= 212:00 +CPU= 8.286
16:05:39 STEP 213 H= 213:00 +CPU= 12.743
16:05:47 STEP 214 H= 214:00 +CPU= 8.259
16:05:56 STEP 215 H= 215:00 +CPU= 8.196
16:06:11 STEP 216 H= 216:00 +CPU= 15.279
Moving to projects directory: /home/stefan/slots/4/ICMGGhq0f+000216
Moving to projects directory: /home/stefan/slots/4/ICMSHhq0f+000216
Moving to projects directory: /home/stefan/slots/4/ICMUAhq0f+000216
Adding to the zip: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_12188825/ICMGGhq0f+000192
Adding to the zip: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_12188825/ICMSHhq0f+000192
Adding to the zip: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_12188825/ICMUAhq0f+000192
Adding to the zip: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_12188825/ICMGGhq0f+000204
Adding to the zip: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_12188825/ICMSHhq0f+000204
Adding to the zip: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_12188825/ICMUAhq0f+000204
Zipping up the intermediate file: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_0181_2003050100_123_972_12188825_1_r51269030_8.zip
Uploading the intermediate file: upload_file_8.zip
16:06:19 STEP 217 H= 217:00 +CPU= 8.069
16:06:28 STEP 218 H= 218:00 +CPU= 8.264
16:06:41 STEP 219 H= 219:00 +CPU= 12.778
16:06:49 STEP 220 H= 220:00 +CPU= 8.244
16:06:57 STEP 221 H= 221:00 +CPU= 8.044
16:07:10 STEP 222 H= 222:00 +CPU= 13.013
16:07:19 STEP 223 H= 223:00 +CPU= 8.074
16:07:27 STEP 224 H= 224:00 +CPU= 8.255
16:07:40 STEP 225 H= 225:00 +CPU= 12.744
16:07:48 STEP 226 H= 226:00 +CPU= 8.280
16:07:56 STEP 227 H= 227:00 +CPU= 8.075
16:08:11 STEP 228 H= 228:00 +CPU= 14.073
16:08:19 STEP 229 H= 229:00 +CPU= 8.054
Moving to projects directory: /home/stefan/slots/4/ICMGGhq0f+000228
Moving to projects directory: /home/stefan/slots/4/ICMSHhq0f+000228
Moving to projects directory: /home/stefan/slots/4/ICMUAhq0f+000228
16:08:27 STEP 230 H= 230:00 +CPU= 8.249
16:08:40 STEP 231 H= 231:00 +CPU= 12.765
16:08:48 STEP 232 H= 232:00 +CPU= 8.281
16:08:56 STEP 233 H= 233:00 +CPU= 8.032
16:09:10 STEP 234 H= 234:00 +CPU= 12.957
16:09:18 STEP 235 H= 235:00 +CPU= 8.049
16:09:26 STEP 236 H= 236:00 +CPU= 8.211
16:09:39 STEP 237 H= 237:00 +CPU= 12.733
16:09:47 STEP 238 H= 238:00 +CPU= 8.261
16:09:55 STEP 239 H= 239:00 +CPU= 8.117
16:10:10 STEP 240 H= 240:00 +CPU= 14.846
16:10:18 STEP 241 H= 241:00 +CPU= 7.861
Moving to projects directory: /home/stefan/slots/4/ICMGGhq0f+000240
Moving to projects directory: /home/stefan/slots/4/ICMSHhq0f+000240
Moving to projects directory: /home/stefan/slots/4/ICMUAhq0f+000240
Adding to the zip: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_12188825/ICMGGhq0f+000216
Adding to the zip: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_12188825/ICMSHhq0f+000216
Adding to the zip: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_12188825/ICMUAhq0f+000216
Adding to the zip: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_12188825/ICMGGhq0f+000228
Adding to the zip: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_12188825/ICMSHhq0f+000228
Adding to the zip: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_12188825/ICMUAhq0f+000228
Zipping up the intermediate file: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_0181_2003050100_123_972_12188825_1_r51269030_9.zip
Uploading the intermediate file: upload_file_9.zip
16:10:26 STEP 242 H= 242:00 +CPU= 8.055
16:10:39 STEP 243 H= 243:00 +CPU= 12.414
16:10:47 STEP 244 H= 244:00 +CPU= 8.000
16:10:55 STEP 245 H= 245:00 +CPU= 7.783
16:11:07 STEP 246 H= 246:00 +CPU= 12.666
16:11:15 STEP 247 H= 247:00 +CPU= 7.813
16:11:23 STEP 248 H= 248:00 +CPU= 8.076
16:11:36 STEP 249 H= 249:00 +CPU= 12.407
16:11:44 STEP 250 H= 250:00 +CPU= 8.017
16:11:52 STEP 251 H= 251:00 +CPU= 7.829
16:12:05 STEP 252 H= 252:00 +CPU= 13.735
Moving to projects directory: /home/stefan/slots/4/ICMGGhq0f+000252
Moving to projects directory: /home/stefan/slots/4/ICMSHhq0f+000252
Moving to projects directory: /home/stefan/slots/4/ICMUAhq0f+000252
16:12:13 STEP 253 H= 253:00 +CPU= 7.904
16:12:21 STEP 254 H= 254:00 +CPU= 8.049
16:12:34 STEP 255 H= 255:00 +CPU= 12.434
16:12:42 STEP 256 H= 256:00 +CPU= 8.031
16:12:50 STEP 257 H= 257:00 +CPU= 7.846
16:13:02 STEP 258 H= 258:00 +CPU= 12.591
16:13:10 STEP 259 H= 259:00 +CPU= 7.835
16:13:18 STEP 260 H= 260:00 +CPU= 7.997
16:13:31 STEP 261 H= 261:00 +CPU= 12.382
16:13:39 STEP 262 H= 262:00 +CPU= 8.073
16:13:47 STEP 263 H= 263:00 +CPU= 8.004
16:14:02 STEP 264 H= 264:00 +CPU= 14.824
Moving to projects directory: /home/stefan/slots/4/ICMGGhq0f+000264
Moving to projects directory: /home/stefan/slots/4/ICMSHhq0f+000264
Moving to projects directory: /home/stefan/slots/4/ICMUAhq0f+000264
Adding to the zip: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_12188825/ICMGGhq0f+000240
Adding to the zip: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_12188825/ICMSHhq0f+000240
Adding to the zip: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_12188825/ICMUAhq0f+000240
Adding to the zip: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_12188825/ICMGGhq0f+000252
Adding to the zip: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_12188825/ICMSHhq0f+000252
Adding to the zip: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_12188825/ICMUAhq0f+000252
Zipping up the intermediate file: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_0181_2003050100_123_972_12188825_1_r51269030_10.zip
16:14:10 STEP 265 H= 265:00 +CPU= 7.823
Uploading the intermediate file: upload_file_10.zip
16:14:18 STEP 266 H= 266:00 +CPU= 8.047
16:14:30 STEP 267 H= 267:00 +CPU= 12.437
16:14:38 STEP 268 H= 268:00 +CPU= 8.085
16:14:46 STEP 269 H= 269:00 +CPU= 7.874
16:14:59 STEP 270 H= 270:00 +CPU= 12.672
16:15:07 STEP 271 H= 271:00 +CPU= 7.834
16:15:15 STEP 272 H= 272:00 +CPU= 8.068
16:15:27 STEP 273 H= 273:00 +CPU= 12.478
16:15:35 STEP 274 H= 274:00 +CPU= 8.037
16:15:43 STEP 275 H= 275:00 +CPU= 7.868
16:15:57 STEP 276 H= 276:00 +CPU= 13.734
Moving to projects directory: /home/stefan/slots/4/ICMGGhq0f+000276
Moving to projects directory: /home/stefan/slots/4/ICMSHhq0f+000276
Moving to projects directory: /home/stefan/slots/4/ICMUAhq0f+000276
16:16:05 STEP 277 H= 277:00 +CPU= 7.853
16:16:13 STEP 278 H= 278:00 +CPU= 8.068
16:16:26 STEP 279 H= 279:00 +CPU= 12.408
16:16:34 STEP 280 H= 280:00 +CPU= 8.035
16:16:41 STEP 281 H= 281:00 +CPU= 7.826
16:16:54 STEP 282 H= 282:00 +CPU= 12.589
16:17:02 STEP 283 H= 283:00 +CPU= 7.909
16:17:10 STEP 284 H= 284:00 +CPU= 8.055
16:17:23 STEP 285 H= 285:00 +CPU= 12.365
16:17:31 STEP 286 H= 286:00 +CPU= 7.974
16:17:39 STEP 287 H= 287:00 +CPU= 7.979
16:17:53 STEP 288 H= 288:00 +CPU= 14.905
Moving to projects directory: /home/stefan/slots/4/ICMGGhq0f+000288
Moving to projects directory: /home/stefan/slots/4/ICMSHhq0f+000288
Moving to projects directory: /home/stefan/slots/4/ICMUAhq0f+000288
Adding to the zip: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_12188825/ICMGGhq0f+000264
Adding to the zip: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_12188825/ICMSHhq0f+000264
Adding to the zip: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_12188825/ICMUAhq0f+000264
Adding to the zip: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_12188825/ICMGGhq0f+000276
Adding to the zip: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_12188825/ICMSHhq0f+000276
Adding to the zip: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_12188825/ICMUAhq0f+000276
Zipping up the intermediate file: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_0181_2003050100_123_972_12188825_1_r51269030_11.zip
Uploading the intermediate file: upload_file_11.zip
16:18:01 STEP 289 H= 289:00 +CPU= 7.765
16:18:09 STEP 290 H= 290:00 +CPU= 7.975
16:18:22 STEP 291 H= 291:00 +CPU= 12.368
16:18:30 STEP 292 H= 292:00 +CPU= 7.988
16:18:37 STEP 293 H= 293:00 +CPU= 7.737
16:18:50 STEP 294 H= 294:00 +CPU= 12.621
16:18:58 STEP 295 H= 295:00 +CPU= 7.876
16:19:06 STEP 296 H= 296:00 +CPU= 7.982
16:19:19 STEP 297 H= 297:00 +CPU= 12.436
16:19:27 STEP 298 H= 298:00 +CPU= 7.970
16:19:34 STEP 299 H= 299:00 +CPU= 7.777
16:19:48 STEP 300 H= 300:00 +CPU= 13.702
Moving to projects directory: /home/stefan/slots/4/ICMGGhq0f+000300
Moving to projects directory: /home/stefan/slots/4/ICMSHhq0f+000300
Moving to projects directory: /home/stefan/slots/4/ICMUAhq0f+000300
16:19:56 STEP 301 H= 301:00 +CPU= 7.778
16:20:04 STEP 302 H= 302:00 +CPU= 8.086
16:20:16 STEP 303 H= 303:00 +CPU= 12.343
16:20:24 STEP 304 H= 304:00 +CPU= 8.072
16:20:32 STEP 305 H= 305:00 +CPU= 7.854
16:20:45 STEP 306 H= 306:00 +CPU= 12.632
16:20:53 STEP 307 H= 307:00 +CPU= 7.881
16:21:01 STEP 308 H= 308:00 +CPU= 8.050
16:21:13 STEP 309 H= 309:00 +CPU= 12.401
16:21:21 STEP 310 H= 310:00 +CPU= 8.060
16:21:30 STEP 311 H= 311:00 +CPU= 8.039
16:21:44 STEP 312 H= 312:00 +CPU= 14.816
Moving to projects directory: /home/stefan/slots/4/ICMGGhq0f+000312
Moving to projects directory: /home/stefan/slots/4/ICMSHhq0f+000312
Moving to projects directory: /home/stefan/slots/4/ICMUAhq0f+000312
Adding to the zip: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_12188825/ICMGGhq0f+000288
Adding to the zip: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_12188825/ICMSHhq0f+000288
Adding to the zip: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_12188825/ICMUAhq0f+000288
Adding to the zip: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_12188825/ICMGGhq0f+000300
Adding to the zip: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_12188825/ICMSHhq0f+000300
Adding to the zip: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_12188825/ICMUAhq0f+000300
Zipping up the intermediate file: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_0181_2003050100_123_972_12188825_1_r51269030_12.zip
Uploading the intermediate file: upload_file_12.zip
16:21:52 STEP 313 H= 313:00 +CPU= 7.835
16:22:00 STEP 314 H= 314:00 +CPU= 8.050
16:22:13 STEP 315 H= 315:00 +CPU= 12.420
16:22:21 STEP 316 H= 316:00 +CPU= 7.992
16:22:29 STEP 317 H= 317:00 +CPU= 7.859
16:22:41 STEP 318 H= 318:00 +CPU= 12.622
16:22:49 STEP 319 H= 319:00 +CPU= 7.773
16:22:57 STEP 320 H= 320:00 +CPU= 8.064
16:23:10 STEP 321 H= 321:00 +CPU= 12.385
16:23:18 STEP 322 H= 322:00 +CPU= 8.031
16:23:26 STEP 323 H= 323:00 +CPU= 7.770
16:23:39 STEP 324 H= 324:00 +CPU= 13.685
Moving to projects directory: /home/stefan/slots/4/ICMGGhq0f+000324
Moving to projects directory: /home/stefan/slots/4/ICMSHhq0f+000324
Moving to projects directory: /home/stefan/slots/4/ICMUAhq0f+000324
16:23:47 STEP 325 H= 325:00 +CPU= 7.865
16:23:55 STEP 326 H= 326:00 +CPU= 8.090
16:24:08 STEP 327 H= 327:00 +CPU= 12.427
16:24:16 STEP 328 H= 328:00 +CPU= 7.990
16:24:24 STEP 329 H= 329:00 +CPU= 7.819
16:24:36 STEP 330 H= 330:00 +CPU= 12.695
16:24:44 STEP 331 H= 331:00 +CPU= 7.857
16:24:52 STEP 332 H= 332:00 +CPU= 7.952
16:25:05 STEP 333 H= 333:00 +CPU= 12.381
16:25:13 STEP 334 H= 334:00 +CPU= 7.998
16:25:21 STEP 335 H= 335:00 +CPU= 8.077
16:25:36 STEP 336 H= 336:00 +CPU= 14.835
Moving to projects directory: /home/stefan/slots/4/ICMGGhq0f+000336
Moving to projects directory: /home/stefan/slots/4/ICMSHhq0f+000336
Moving to projects directory: /home/stefan/slots/4/ICMUAhq0f+000336
Adding to the zip: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_12188825/ICMGGhq0f+000312
Adding to the zip: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_12188825/ICMSHhq0f+000312
Adding to the zip: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_12188825/ICMUAhq0f+000312
Adding to the zip: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_12188825/ICMGGhq0f+000324
Adding to the zip: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_12188825/ICMSHhq0f+000324
Adding to the zip: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_12188825/ICMUAhq0f+000324
Zipping up the intermediate file: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_0181_2003050100_123_972_12188825_1_r51269030_13.zip
Uploading the intermediate file: upload_file_13.zip
16:25:43 STEP 337 H= 337:00 +CPU= 7.848
16:25:52 STEP 338 H= 338:00 +CPU= 8.031
16:26:04 STEP 339 H= 339:00 +CPU= 12.432
16:26:12 STEP 340 H= 340:00 +CPU= 8.003
16:26:20 STEP 341 H= 341:00 +CPU= 7.841
16:26:33 STEP 342 H= 342:00 +CPU= 12.682
16:26:40 STEP 343 H= 343:00 +CPU= 7.829
16:26:49 STEP 344 H= 344:00 +CPU= 8.048
16:27:01 STEP 345 H= 345:00 +CPU= 12.456
16:27:09 STEP 346 H= 346:00 +CPU= 8.032
16:27:17 STEP 347 H= 347:00 +CPU= 7.814
16:27:31 STEP 348 H= 348:00 +CPU= 13.776
Moving to projects directory: /home/stefan/slots/4/ICMGGhq0f+000348
Moving to projects directory: /home/stefan/slots/4/ICMSHhq0f+000348
Moving to projects directory: /home/stefan/slots/4/ICMUAhq0f+000348
16:27:39 STEP 349 H= 349:00 +CPU= 7.846
16:27:47 STEP 350 H= 350:00 +CPU= 7.961
16:27:59 STEP 351 H= 351:00 +CPU= 12.433
16:28:07 STEP 352 H= 352:00 +CPU= 8.000
16:28:15 STEP 353 H= 353:00 +CPU= 7.842
16:28:28 STEP 354 H= 354:00 +CPU= 12.604
16:28:35 STEP 355 H= 355:00 +CPU= 7.801
16:28:44 STEP 356 H= 356:00 +CPU= 8.010
16:28:56 STEP 357 H= 357:00 +CPU= 12.372
16:29:04 STEP 358 H= 358:00 +CPU= 8.035
16:29:12 STEP 359 H= 359:00 +CPU= 7.985
16:29:27 STEP 360 H= 360:00 +CPU= 14.682
Moving to projects directory: /home/stefan/slots/4/ICMGGhq0f+000360
Moving to projects directory: /home/stefan/slots/4/ICMSHhq0f+000360
Moving to projects directory: /home/stefan/slots/4/ICMUAhq0f+000360
Adding to the zip: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_12188825/ICMGGhq0f+000336
Adding to the zip: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_12188825/ICMSHhq0f+000336
Adding to the zip: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_12188825/ICMUAhq0f+000336
Adding to the zip: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_12188825/ICMGGhq0f+000348
Adding to the zip: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_12188825/ICMSHhq0f+000348
Adding to the zip: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_12188825/ICMUAhq0f+000348
Zipping up the intermediate file: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_0181_2003050100_123_972_12188825_1_r51269030_14.zip
Uploading the intermediate file: upload_file_14.zip
16:29:35 STEP 361 H= 361:00 +CPU= 7.786
16:29:43 STEP 362 H= 362:00 +CPU= 8.005
Uploading trickle at timestep: 1296000
16:29:55 STEP 363 H= 363:00 +CPU= 12.437
16:30:03 STEP 364 H= 364:00 +CPU= 7.988
16:30:11 STEP 365 H= 365:00 +CPU= 7.773
16:30:24 STEP 366 H= 366:00 +CPU= 12.631
16:30:31 STEP 367 H= 367:00 +CPU= 7.779
16:30:39 STEP 368 H= 368:00 +CPU= 8.051
16:30:52 STEP 369 H= 369:00 +CPU= 12.378
16:31:00 STEP 370 H= 370:00 +CPU= 7.965
16:31:08 STEP 371 H= 371:00 +CPU= 7.860
16:31:21 STEP 372 H= 372:00 +CPU= 13.670
Moving to projects directory: /home/stefan/slots/4/ICMGGhq0f+000372
Moving to projects directory: /home/stefan/slots/4/ICMSHhq0f+000372
Moving to projects directory: /home/stefan/slots/4/ICMUAhq0f+000372
16:31:29 STEP 373 H= 373:00 +CPU= 7.821
16:31:37 STEP 374 H= 374:00 +CPU= 7.978
16:31:50 STEP 375 H= 375:00 +CPU= 12.351
16:31:58 STEP 376 H= 376:00 +CPU= 8.005
16:32:06 STEP 377 H= 377:00 +CPU= 7.764
16:32:18 STEP 378 H= 378:00 +CPU= 12.653
16:32:26 STEP 379 H= 379:00 +CPU= 7.817
16:32:34 STEP 380 H= 380:00 +CPU= 8.029
16:32:47 STEP 381 H= 381:00 +CPU= 12.360
16:32:55 STEP 382 H= 382:00 +CPU= 8.020
16:33:03 STEP 383 H= 383:00 +CPU= 8.037
16:33:17 STEP 384 H= 384:00 +CPU= 14.706
Moving to projects directory: /home/stefan/slots/4/ICMGGhq0f+000384
Moving to projects directory: /home/stefan/slots/4/ICMSHhq0f+000384
Moving to projects directory: /home/stefan/slots/4/ICMUAhq0f+000384
Adding to the zip: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_12188825/ICMGGhq0f+000360
Adding to the zip: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_12188825/ICMSHhq0f+000360
Adding to the zip: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_12188825/ICMUAhq0f+000360
Adding to the zip: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_12188825/ICMGGhq0f+000372
Adding to the zip: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_12188825/ICMSHhq0f+000372
Adding to the zip: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_12188825/ICMUAhq0f+000372
Zipping up the intermediate file: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_0181_2003050100_123_972_12188825_1_r51269030_15.zip
Uploading the intermediate file: upload_file_15.zip
16:33:25 STEP 385 H= 385:00 +CPU= 7.809
16:33:33 STEP 386 H= 386:00 +CPU= 8.024
16:33:46 STEP 387 H= 387:00 +CPU= 12.365
16:33:54 STEP 388 H= 388:00 +CPU= 7.990
16:34:02 STEP 389 H= 389:00 +CPU= 7.820
16:34:14 STEP 390 H= 390:00 +CPU= 12.458
16:34:22 STEP 391 H= 391:00 +CPU= 7.795
16:34:30 STEP 392 H= 392:00 +CPU= 8.000
16:34:42 STEP 393 H= 393:00 +CPU= 12.380
16:34:50 STEP 394 H= 394:00 +CPU= 8.048
16:34:58 STEP 395 H= 395:00 +CPU= 7.882
16:35:12 STEP 396 H= 396:00 +CPU= 13.722
Moving to projects directory: /home/stefan/slots/4/ICMGGhq0f+000396
Moving to projects directory: /home/stefan/slots/4/ICMSHhq0f+000396
Moving to projects directory: /home/stefan/slots/4/ICMUAhq0f+000396
16:35:20 STEP 397 H= 397:00 +CPU= 7.830
16:35:28 STEP 398 H= 398:00 +CPU= 8.077
16:35:40 STEP 399 H= 399:00 +CPU= 12.479
16:35:49 STEP 400 H= 400:00 +CPU= 8.060
16:35:56 STEP 401 H= 401:00 +CPU= 7.836
16:36:09 STEP 402 H= 402:00 +CPU= 12.618
16:36:17 STEP 403 H= 403:00 +CPU= 7.856
16:36:25 STEP 404 H= 404:00 +CPU= 8.086
16:36:37 STEP 405 H= 405:00 +CPU= 12.291
16:36:45 STEP 406 H= 406:00 +CPU= 8.016
16:36:53 STEP 407 H= 407:00 +CPU= 8.012
16:37:08 STEP 408 H= 408:00 +CPU= 14.797
Moving to projects directory: /home/stefan/slots/4/ICMGGhq0f+000408
Moving to projects directory: /home/stefan/slots/4/ICMSHhq0f+000408
Moving to projects directory: /home/stefan/slots/4/ICMUAhq0f+000408
Adding to the zip: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_12188825/ICMGGhq0f+000384
Adding to the zip: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_12188825/ICMSHhq0f+000384
Adding to the zip: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_12188825/ICMUAhq0f+000384
Adding to the zip: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_12188825/ICMGGhq0f+000396
Adding to the zip: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_12188825/ICMSHhq0f+000396
Adding to the zip: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_12188825/ICMUAhq0f+000396
Zipping up the intermediate file: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_0181_2003050100_123_972_12188825_1_r51269030_16.zip
Uploading the intermediate file: upload_file_16.zip
16:37:16 STEP 409 H= 409:00 +CPU= 7.789
16:37:24 STEP 410 H= 410:00 +CPU= 8.027
16:37:37 STEP 411 H= 411:00 +CPU= 12.446
16:37:45 STEP 412 H= 412:00 +CPU= 8.023
16:37:53 STEP 413 H= 413:00 +CPU= 7.786
16:38:05 STEP 414 H= 414:00 +CPU= 12.699
16:38:13 STEP 415 H= 415:00 +CPU= 7.805
16:38:21 STEP 416 H= 416:00 +CPU= 8.003
16:38:34 STEP 417 H= 417:00 +CPU= 12.415
16:38:42 STEP 418 H= 418:00 +CPU= 8.031
16:38:50 STEP 419 H= 419:00 +CPU= 7.788
16:39:03 STEP 420 H= 420:00 +CPU= 13.647
Moving to projects directory: /home/stefan/slots/4/ICMGGhq0f+000420
Moving to projects directory: /home/stefan/slots/4/ICMSHhq0f+000420
Moving to projects directory: /home/stefan/slots/4/ICMUAhq0f+000420
16:39:11 STEP 421 H= 421:00 +CPU= 7.818
16:39:19 STEP 422 H= 422:00 +CPU= 8.003
16:39:32 STEP 423 H= 423:00 +CPU= 12.379
16:39:40 STEP 424 H= 424:00 +CPU= 7.987
16:39:47 STEP 425 H= 425:00 +CPU= 7.834
16:40:00 STEP 426 H= 426:00 +CPU= 12.622
16:40:08 STEP 427 H= 427:00 +CPU= 7.745
16:40:16 STEP 428 H= 428:00 +CPU= 7.986
16:40:28 STEP 429 H= 429:00 +CPU= 12.404
16:40:36 STEP 430 H= 430:00 +CPU= 7.992
16:40:44 STEP 431 H= 431:00 +CPU= 7.992
16:40:59 STEP 432 H= 432:00 +CPU= 14.773
Moving to projects directory: /home/stefan/slots/4/ICMGGhq0f+000432
Moving to projects directory: /home/stefan/slots/4/ICMSHhq0f+000432
Moving to projects directory: /home/stefan/slots/4/ICMUAhq0f+000432
Adding to the zip: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_12188825/ICMGGhq0f+000408
Adding to the zip: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_12188825/ICMSHhq0f+000408
Adding to the zip: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_12188825/ICMUAhq0f+000408
Adding to the zip: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_12188825/ICMGGhq0f+000420
Adding to the zip: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_12188825/ICMSHhq0f+000420
Adding to the zip: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_12188825/ICMUAhq0f+000420
Zipping up the intermediate file: /home/stefan/projects/climateprediction.net/oifs_43r3_ps_0181_2003050100_123_972_12188825_1_r51269030_17.zip
Uploading the intermediate file: upload_file_17.zip
16:41:07 STEP 433 H= 433:00 +CPU= 7.745
16:41:15 STEP 434 H= 434:00 +CPU= 7.950
16:41:27 STEP 435 H= 435:00 +CPU= 12.324
16:41:35 STEP 436 H= 436:00 +CPU= 8.039
16:41:43 STEP 437 H= 437:00 +CPU= 7.833
16:41:56 STEP 438 H= 438:00 +CPU= 12.541
16:42:04 STEP 439 H= 439:00 +CPU= 7.794
16:42:12 STEP 440 H= 440:00 +CPU= 7.985
16:42:24 STEP 441 H= 441:00 +CPU= 12.406
16:42:32 STEP 442 H= 442:00 +CPU= 8.022
16:42:40 STEP 443 H= 443:00 +CPU= 7.812
double free or corruption (out)
</stderr_txt>
]]>
