Name | oifs_43r3_bl_a3t1_2016092300_15_991_12211720_0 |
Workunit | 12211720 |
Created | 13 Feb 2023, 21:05:23 UTC |
Sent | 13 Feb 2023, 21:09:12 UTC |
Report deadline | 14 Apr 2023, 21:09:12 UTC |
Received | 16 Feb 2023, 1:06:21 UTC |
Server state | Over |
Outcome | Computation error |
Client state | Compute error |
Exit status | 5 (0x00000005) Unknown error code |
Computer ID | 1537658 |
Run time | 10 hours 10 min 53 sec |
CPU time | 10 hours 10 min 53 sec |
Validate state | Invalid |
Credit | 0.00 |
Device peak FLOPS | 2.62 GFLOPS |
Application version | OpenIFS 43r3 Baroclinic Lifecycle v1.11 x86_64-pc-linux-gnu |
Peak working set size | 4,783.64 MB |
Peak swap size | 5,930.79 MB |
Peak disk usage | 2,230.85 MB |
Stderr | <core_client_version>7.20.5</core_client_version> <![CDATA[ <message> process exited with code 5 (0x5, -251)</message> <stderr_txt> .000000000000000E+00 0.100000000000000E+01 GPNORM CLW AVERAGE MINIMUM MAXIMUM AVE 0.000000000000000E+00 0.000000000000000E+00 0.000000000000000E+00 GPNORM CIW AVERAGE MINIMUM MAXIMUM AVE 0.198189194057345E-07 0.000000000000000E+00 0.273309426807427E-05 GPNORM CC AVERAGE MINIMUM MAXIMUM AVE 0.884863620141587E-01 0.000000000000000E+00 0.100000000000000E+01 GPNORM RFL AVERAGE MINIMUM MAXIMUM AVE 0.000000000000000E+00 0.000000000000000E+00 0.000000000000000E+00 GPNORM SFL AVERAGE MINIMUM MAXIMUM AVE 0.457431400828819E-09 0.000000000000000E+00 0.154399557857648E-06 GPNORM CRFL AVERAGE MINIMUM MAXIMUM AVE 0.000000000000000E+00 0.000000000000000E+00 0.000000000000000E+00 GPNORM CSFL AVERAGE MINIMUM MAXIMUM AVE 0.000000000000000E+00 0.000000000000000E+00 0.000000000000000E+00 GPNORM PFRC AVERAGE MINIMUM MAXIMUM AVE 0.233683459758845E-01 0.000000000000000E+00 0.100000000000000E+01 NSTEP = 288 STEPO 0AAA00AAA 20:57:57 STEP 288 H= 72:00 +CPU= 48.468 NSTEP = 289 STEPO 0AAA00AAA 20:58:42 STEP 289 H= 72:15 +CPU= 44.388 ISTEP= 290 RSOLINC= 1361.4482 RI0= 1365.0000 ZI0= 1361.4482 RIP0= 1361.4482 ISTEP= 290 ZI0= 1361.4482 ZSEASON= 1.000000000 REA=149597870000.00 RDEASO=149597870000.00 NSTEP = 290 STEPO 0AAA00AAA 20:59:26 STEP 290 H= 72:30 +CPU= 43.949 NSTEP = 291 STEPO 0AAA00AAA 21:00:09 STEP 291 H= 72:45 +CPU= 43.568 NSTEP = 292 STEPO 0AAA00AAA 21:00:54 STEP 292 H= 73:00 +CPU= 44.657 NSTEP = 293 STEPO 0AAA00AAA 21:01:38 STEP 293 H= 73:15 +CPU= 44.144 NSTEP = 294 STEPO 0AAA00AAA 21:02:23 STEP 294 H= 73:30 +CPU= 44.494 ISTEP= 295 RSOLINC= 1361.4482 RI0= 1365.0000 ZI0= 1361.4482 RIP0= 1361.4482 ISTEP= 295 ZI0= 1361.4482 ZSEASON= 1.000000000 REA=149597870000.00 RDEASO=149597870000.00 NSTEP = 295 STEPO 0AAA00AAA ------------------------------------------------ CNT0 not found; string returned was: 'STEPO' >>> Printing last 8 lines from file: ifs.stat 20:57:57 0AAA00AAA STEPO 289 48.212 48.212 48.521 0:54 1:01 0.10446467725561E-05 2GB 0MB 20:58:42 0AAA00AAA STEPO 290 44.388 44.388 44.549 1:39 1:46 0.10446467725561E-05 2GB 0MB 20:59:26 0AAA00AAA STEPO 291 43.950 43.950 44.086 2:23 2:30 0.10446467725561E-05 2GB 0MB 21:00:09 0AAA00AAA STEPO 292 43.568 43.568 43.729 3:06 3:14 0.10446467725561E-05 2GB 0MB 21:00:54 0AAA00AAA STEPO 293 44.657 44.657 44.702 3:51 3:58 0.10446467725561E-05 2GB 0MB 21:01:38 0AAA00AAA STEPO 294 44.144 44.144 44.185 4:35 4:43 0.10446467725561E-05 2GB 0MB 21:02:23 0AAA00AAA STEPO 295 44.494 44.494 44.603 5:20 5:27 0.10446467725561E-05 2GB 0MB ------------------------------------------------ >>> Printing last 11 lines from file: rcf CSTEP = " 288", CTIME = "00030000 ", NSTEPLPP = 201, 202, 49, 228226, 228227, 2*0, 276, 2*0, IPRGPNSRES = 1, IPRGPEWRES = 1, IPRTRWRES = 1, IPRTRVRES = 1, GMASS0 = 0.000000000000000E+000, GMASSI = 0.000000000000000E+000 / ------------------------------------------------ >>> Printing last 8 lines from file: /var/lib/boinc-client/slots/39/progress_file_12211720.xml <running_values> <last_cpu_time>27059.300000</last_cpu_time> <upload_file_number>3</upload_file_number> <last_iter>294</last_iter> <last_upload>259200</last_upload> <model_completed>0</model_completed> </running_values> ------------------------------------------------ ..Failed, model did not complete successfully (argv0) ../../projects/climateprediction.net/oifs_43r3_bl_1.11_x86_64-pc-linux-gnu (argv1) start_date: 2016092300 (argv2) exptid: h7zg (argv3) unique_member_id: a3t1 (argv4) batchid: 991 (argv5) wuid: 12211720 (argv6) fclen: 15 (argv7) app_name: oifs_43r3_bl (argv8) nthreads: 1 Working directory is: /var/lib/boinc-client/slots/39 Project directory is: /var/lib/boinc-client/projects/climateprediction.net/ app name: oifs_43r3_bl version: 1.11 Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211720 ..mkdir for temp folder for results failed Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_app_1.11_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/39/oifs_43r3_bl_app_1.11_x86_64-pc-linux-gnu.zip Unzipping the app zip file: /var/lib/boinc-client/slots/39/oifs_43r3_bl_app_1.11_x86_64-pc-linux-gnu.zip Copying the namelist files from: ../../projects/climateprediction.net/jf_8a660c23fc0cf7a33fe79a39cb63b992 to: /var/lib/boinc-client/slots/39/oifs_43r3_bl_a3t1_2016092300_15_991_12211720.zip Unzipping the namelist zip file: /var/lib/boinc-client/slots/39/oifs_43r3_bl_a3t1_2016092300_15_991_12211720.zip ic_ancil_file: ic_ancil_12211720 ifsdata_file: ifsdata_12211720 climate_data_file: clim_data_12211720 horiz_resolution: 159 vert_resolution: 91 grid_type: l_2 upload_interval: 96 utstep: 900 nfrres: restart dump frequency (steps) 48 Copying IC ancils from: ../../projects/climateprediction.net/jf_f7ecfd27aa1e68bbe7e8d19800e25c73 to: /var/lib/boinc-client/slots/39/ic_ancil_12211720.zip Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/39/ic_ancil_12211720.zip ..mkdir for ifsdata folder failed Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_c442047aac50823c47ff0af11442c9b4 to: /var/lib/boinc-client/slots/39/ifsdata/ifsdata_12211720.zip Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/39/ifsdata/ifsdata_12211720.zip ..mkdir for the climate data folder failed Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/39/159l_2/clim_data_12211720.zip Unzipping the climate data zip file: /var/lib/boinc-client/slots/39/159l_2/clim_data_12211720.zip Checking for progress XML file: /var/lib/boinc-client/slots/39/progress_file_12211720.xml Opened progress file ok : /var/lib/boinc-client/slots/39/progress_file_12211720.xml -- Model is restarting -- Adjusting last_iter, 294, to previous model restart step. Creating progress file: /var/lib/boinc-client/slots/39/progress_file_12211720.xml last_cpu_time: 27059 upload_file_number: 3 last_iter: 289 last_upload: 259200 model_completed: 0 total_length_of_simulation: 1296000 result_base_name: oifs_43r3_bl_a3t1_2016092300_15_991_12211720_0_r2123028782 The child process has been launched with process id: 72779 Executing the command: /var/lib/boinc-client/slots/39/oifs_43r3_model.exe [EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1 [EC_DRHOOK:boinc5:1:1:72779:72779] [20230215:210351:1676495031:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2091] fp = 0x30e4a00 [EC_DRHOOK:boinc5:1:1:72779:72779] [20230215:210351:1676495031:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1 [EC_DRHOOK:boinc5:1:1:72779:72779] [20230215:210351:1676495031:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0) [EC_DRHOOK:boinc5:1:1:72779:72779] [20230215:210351:1676495031:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1 [EC_DRHOOK:boinc5:1:1:72779:72779] [20230215:210351:1676495031:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000 [EC_DRHOOK:boinc5:1:1:72779:72779] [20230215:210351:1676495031:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647) [EC_DRHOOK:boinc5:1:1:72779:72779] [20230215:210351:1676495031:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16 [EC_DRHOOK:boinc5:1:1:72779:72779] [20230215:210351:1676495031:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0 [EC_DRHOOK:boinc5:1:1:72779:72779] [20230215:210351:1676495031:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500 [EC_DRHOOK:boinc5:1:1:72779:72779] [20230215:210351:1676495031:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1 [EC_DRHOOK:boinc5:1:1:72779:72779] [20230215:210351:1676495031:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1 [EC_DRHOOK:boinc5:1:1:72779:72779] [20230215:210351:1676495031:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1 [EC_DRHOOK:boinc5:1:1:72779:72779] [20230215:210351:1676495031:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1 [EC_DRHOOK:boinc5:1:1:72779:72779] [20230215:210351:1676495031:0.000] [ignore_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef> [EC_DRHOOK:boinc5:1:1:72779:72779] [20230215:210351:1676495031:0.001] [restore_default_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef> [EC_DRHOOK:boinc5:1:1:72779:72779] [20230215:210351:1676495031:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f520 (old at 0x401140) [EC_DRHOOK:boinc5:1:1:72779:72779] [20230215:210351:1676495031:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f520 (old at (nil)) [EC_DRHOOK:boinc5:1:1:72779:72779] [20230215:210351:1676495031:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f520 (old at 0x401140) [EC_DRHOOK:boinc5:1:1:72779:72779] [20230215:210351:1676495031:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f520 (old at (nil)) [EC_DRHOOK:boinc5:1:1:72779:72779] [20230215:210351:1676495031:0.001] [trapfpe_treatment@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1 [EC_DRHOOK:boinc5:1:1:72779:72779] [20230215:210351:1676495031:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f520 (old at 0x401140) [EC_DRHOOK:boinc5:1:1:72779:72779] [20230215:210351:1676495031:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f520 (old at 0x401140) [EC_DRHOOK:boinc5:1:1:72779:72779] [20230215:210351:1676495031:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f520 (old at (nil)) [EC_DRHOOK:boinc5:1:1:72779:72779] [20230215:210351:1676495031:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f520 (old at 0x401140) [EC_DRHOOK:boinc5:1:1:72779:72779] [20230215:210351:1676495031:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f520 (old at 0x401140) [EC_DRHOOK:boinc5:1:1:72779:72779] [20230215:210351:1676495031:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f520 (old at 0x401140) [EC_DRHOOK:boinc5:1:1:72779:72779] [20230215:210351:1676495031:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f520 (old at (nil)) [EC_DRHOOK:boinc5:1:1:72779:72779] [20230215:210351:1676495031:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f520 (old at (nil)) [EC_DRHOOK:boinc5:1:1:72779:72779] [20230215:210351:1676495031:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f520 (old at (nil)) [EC_DRHOOK:boinc5:1:1:72779:72779] [20230215:210351:1676495031:0.001] [catch_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef> Quit request received from BOINC client, ending the child process (argv0) ../../projects/climateprediction.net/oifs_43r3_bl_1.11_x86_64-pc-linux-gnu (argv1) start_date: 2016092300 (argv2) exptid: h7zg (argv3) unique_member_id: a3t1 (argv4) batchid: 991 (argv5) wuid: 12211720 (argv6) fclen: 15 (argv7) app_name: oifs_43r3_bl (argv8) nthreads: 1 Working directory is: /var/lib/boinc-client/slots/39 Project directory is: /var/lib/boinc-client/projects/climateprediction.net/ app name: oifs_43r3_bl version: 1.11 Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211720 ..mkdir for temp folder for results failed Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_app_1.11_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/39/oifs_43r3_bl_app_1.11_x86_64-pc-linux-gnu.zip Unzipping the app zip file: /var/lib/boinc-client/slots/39/oifs_43r3_bl_app_1.11_x86_64-pc-linux-gnu.zip Copying the namelist files from: ../../projects/climateprediction.net/jf_8a660c23fc0cf7a33fe79a39cb63b992 to: /var/lib/boinc-client/slots/39/oifs_43r3_bl_a3t1_2016092300_15_991_12211720.zip Unzipping the namelist zip file: /var/lib/boinc-client/slots/39/oifs_43r3_bl_a3t1_2016092300_15_991_12211720.zip ic_ancil_file: ic_ancil_12211720 ifsdata_file: ifsdata_12211720 climate_data_file: clim_data_12211720 horiz_resolution: 159 vert_resolution: 91 grid_type: l_2 upload_interval: 96 utstep: 900 nfrres: restart dump frequency (steps) 48 Copying IC ancils from: ../../projects/climateprediction.net/jf_f7ecfd27aa1e68bbe7e8d19800e25c73 to: /var/lib/boinc-client/slots/39/ic_ancil_12211720.zip Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/39/ic_ancil_12211720.zip ..mkdir for ifsdata folder failed Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_c442047aac50823c47ff0af11442c9b4 to: /var/lib/boinc-client/slots/39/ifsdata/ifsdata_12211720.zip Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/39/ifsdata/ifsdata_12211720.zip ..mkdir for the climate data folder failed Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/39/159l_2/clim_data_12211720.zip Unzipping the climate data zip file: /var/lib/boinc-client/slots/39/159l_2/clim_data_12211720.zip Checking for progress XML file: /var/lib/boinc-client/slots/39/progress_file_12211720.xml Opened progress file ok : /var/lib/boinc-client/slots/39/progress_file_12211720.xml -- Model is restarting -- Adjusting last_iter, 288, to previous model restart step. Creating progress file: /var/lib/boinc-client/slots/39/progress_file_12211720.xml last_cpu_time: 27077 upload_file_number: 3 last_iter: 289 last_upload: 259200 model_completed: 0 total_length_of_simulation: 1296000 result_base_name: oifs_43r3_bl_a3t1_2016092300_15_991_12211720_0_r2123028782 The child process has been launched with process id: 72846 Executing the command: /var/lib/boinc-client/slots/39/oifs_43r3_model.exe [EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1 [EC_DRHOOK:boinc5:1:1:72846:72846] [20230215:210523:1676495123:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2091] fp = 0x30e4a00 [EC_DRHOOK:boinc5:1:1:72846:72846] [20230215:210523:1676495123:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1 [EC_DRHOOK:boinc5:1:1:72846:72846] [20230215:210523:1676495123:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0) [EC_DRHOOK:boinc5:1:1:72846:72846] [20230215:210523:1676495123:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1 [EC_DRHOOK:boinc5:1:1:72846:72846] [20230215:210523:1676495123:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000 [EC_DRHOOK:boinc5:1:1:72846:72846] [20230215:210523:1676495123:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647) [EC_DRHOOK:boinc5:1:1:72846:72846] [20230215:210523:1676495123:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16 [EC_DRHOOK:boinc5:1:1:72846:72846] [20230215:210523:1676495123:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0 [EC_DRHOOK:boinc5:1:1:72846:72846] [20230215:210523:1676495123:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500 [EC_DRHOOK:boinc5:1:1:72846:72846] [20230215:210523:1676495123:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1 [EC_DRHOOK:boinc5:1:1:72846:72846] [20230215:210523:1676495123:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1 [EC_DRHOOK:boinc5:1:1:72846:72846] [20230215:210523:1676495123:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1 [EC_DRHOOK:boinc5:1:1:72846:72846] [20230215:210523:1676495123:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1 [EC_DRHOOK:boinc5:1:1:72846:72846] [20230215:210523:1676495123:0.000] [ignore_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef> [EC_DRHOOK:boinc5:1:1:72846:72846] [20230215:210523:1676495123:0.000] [restore_default_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef> [EC_DRHOOK:boinc5:1:1:72846:72846] [20230215:210523:1676495123:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f520 (old at 0x401140) [EC_DRHOOK:boinc5:1:1:72846:72846] [20230215:210523:1676495123:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f520 (old at (nil)) [EC_DRHOOK:boinc5:1:1:72846:72846] [20230215:210523:1676495123:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f520 (old at 0x401140) [EC_DRHOOK:boinc5:1:1:72846:72846] [20230215:210523:1676495123:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f520 (old at (nil)) [EC_DRHOOK:boinc5:1:1:72846:72846] [20230215:210523:1676495123:0.001] [trapfpe_treatment@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1 [EC_DRHOOK:boinc5:1:1:72846:72846] [20230215:210523:1676495123:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f520 (old at 0x401140) [EC_DRHOOK:boinc5:1:1:72846:72846] [20230215:210523:1676495123:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f520 (old at 0x401140) [EC_DRHOOK:boinc5:1:1:72846:72846] [20230215:210523:1676495123:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f520 (old at (nil)) [EC_DRHOOK:boinc5:1:1:72846:72846] [20230215:210523:1676495123:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f520 (old at 0x401140) [EC_DRHOOK:boinc5:1:1:72846:72846] [20230215:210523:1676495123:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f520 (old at 0x401140) [EC_DRHOOK:boinc5:1:1:72846:72846] [20230215:210523:1676495123:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f520 (old at 0x401140) [EC_DRHOOK:boinc5:1:1:72846:72846] [20230215:210523:1676495123:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f520 (old at (nil)) [EC_DRHOOK:boinc5:1:1:72846:72846] [20230215:210523:1676495123:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f520 (old at (nil)) [EC_DRHOOK:boinc5:1:1:72846:72846] [20230215:210523:1676495123:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f520 (old at (nil)) [EC_DRHOOK:boinc5:1:1:72846:72846] [20230215:210523:1676495123:0.001] [catch_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef> 21:06:54 STEP 288 H= 72:00 +CPU= 53.277 21:07:38 STEP 289 H= 72:15 +CPU= 44.072 21:08:22 STEP 290 H= 72:30 +CPU= 43.415 21:09:05 STEP 291 H= 72:45 +CPU= 42.950 21:09:58 STEP 292 H= 73:00 +CPU= 47.496 21:10:44 STEP 293 H= 73:15 +CPU= 45.304 21:11:31 STEP 294 H= 73:30 +CPU= 47.650 21:12:20 STEP 295 H= 73:45 +CPU= 45.464 21:13:04 STEP 296 H= 74:00 +CPU= 44.050 21:14:00 STEP 297 H= 74:15 +CPU= 47.634 21:14:46 STEP 298 H= 74:30 +CPU= 45.011 21:15:31 STEP 299 H= 74:45 +CPU= 45.014 21:16:28 STEP 300 H= 75:00 +CPU= 54.712 Moving to projects directory: /var/lib/boinc-client/slots/39/ICMGGh7zg+000300 Moving to projects directory: /var/lib/boinc-client/slots/39/ICMSHh7zg+000300 Moving to projects directory: /var/lib/boinc-client/slots/39/ICMUAh7zg+000300 21:17:13 STEP 301 H= 75:15 +CPU= 44.392 21:17:56 STEP 302 H= 75:30 +CPU= 42.687 21:18:46 STEP 303 H= 75:45 +CPU= 45.370 21:19:31 STEP 304 H= 76:00 +CPU= 44.832 21:20:18 STEP 305 H= 76:15 +CPU= 45.012 21:21:09 STEP 306 H= 76:30 +CPU= 44.630 21:21:55 STEP 307 H= 76:45 +CPU= 45.514 21:22:41 STEP 308 H= 77:00 +CPU= 46.471 21:23:28 STEP 309 H= 77:15 +CPU= 45.959 21:24:15 STEP 310 H= 77:30 +CPU= 46.981 21:25:06 STEP 311 H= 77:45 +CPU= 47.867 21:26:06 STEP 312 H= 78:00 +CPU= 58.256 Moving to projects directory: /var/lib/boinc-client/slots/39/ICMGGh7zg+000312 Moving to projects directory: /var/lib/boinc-client/slots/39/ICMSHh7zg+000312 Moving to projects directory: /var/lib/boinc-client/slots/39/ICMUAh7zg+000312 21:26:55 STEP 313 H= 78:15 +CPU= 47.826 21:27:41 STEP 314 H= 78:30 +CPU= 45.446 21:28:29 STEP 315 H= 78:45 +CPU= 46.850 21:29:17 STEP 316 H= 79:00 +CPU= 47.860 21:30:01 STEP 317 H= 79:15 +CPU= 43.349 21:30:45 STEP 318 H= 79:30 +CPU= 43.745 21:31:32 STEP 319 H= 79:45 +CPU= 46.794 21:32:22 STEP 320 H= 80:00 +CPU= 49.705 21:33:09 STEP 321 H= 80:15 +CPU= 46.229 21:33:56 STEP 322 H= 80:30 +CPU= 47.040 21:34:46 STEP 323 H= 80:45 +CPU= 48.094 21:35:45 STEP 324 H= 81:00 +CPU= 58.761 Moving to projects directory: /var/lib/boinc-client/slots/39/ICMGGh7zg+000324 Moving to projects directory: /var/lib/boinc-client/slots/39/ICMSHh7zg+000324 Moving to projects directory: /var/lib/boinc-client/slots/39/ICMUAh7zg+000324 21:36:35 STEP 325 H= 81:15 +CPU= 45.984 21:37:17 STEP 326 H= 81:30 +CPU= 42.023 21:38:11 STEP 327 H= 81:45 +CPU= 43.810 21:38:55 STEP 328 H= 82:00 +CPU= 43.921 21:39:40 STEP 329 H= 82:15 +CPU= 44.794 21:40:32 STEP 330 H= 82:30 +CPU= 45.257 21:41:15 STEP 331 H= 82:45 +CPU= 42.780 21:42:01 STEP 332 H= 83:00 +CPU= 45.740 21:43:05 STEP 333 H= 83:15 +CPU= 50.916 21:43:54 STEP 334 H= 83:30 +CPU= 47.366 21:44:42 STEP 335 H= 83:45 +CPU= 47.065 21:45:58 STEP 336 H= 84:00 +CPU= 67.046 Moving to projects directory: /var/lib/boinc-client/slots/39/ICMGGh7zg+000336 Moving to projects directory: /var/lib/boinc-client/slots/39/ICMSHh7zg+000336 Moving to projects directory: /var/lib/boinc-client/slots/39/ICMUAh7zg+000336 21:46:54 STEP 337 H= 84:15 +CPU= 46.617 21:47:39 STEP 338 H= 84:30 +CPU= 44.710 21:48:32 STEP 339 H= 84:45 +CPU= 51.681 21:49:20 STEP 340 H= 85:00 +CPU= 47.315 21:50:05 STEP 341 H= 85:15 +CPU= 44.376 21:50:57 STEP 342 H= 85:30 +CPU= 48.368 21:51:50 STEP 343 H= 85:45 +CPU= 47.109 21:52:36 STEP 344 H= 86:00 +CPU= 45.763 21:53:21 STEP 345 H= 86:15 +CPU= 44.993 21:54:09 STEP 346 H= 86:30 +CPU= 47.154 21:55:00 STEP 347 H= 86:45 +CPU= 47.730 21:55:59 STEP 348 H= 87:00 +CPU= 56.620 Moving to projects directory: /var/lib/boinc-client/slots/39/ICMGGh7zg+000348 Moving to projects directory: /var/lib/boinc-client/slots/39/ICMSHh7zg+000348 Moving to projects directory: /var/lib/boinc-client/slots/39/ICMUAh7zg+000348 21:56:52 STEP 349 H= 87:15 +CPU= 48.647 21:57:46 STEP 350 H= 87:30 +CPU= 47.822 21:58:35 STEP 351 H= 87:45 +CPU= 47.436 21:59:26 STEP 352 H= 88:00 +CPU= 49.964 22:00:15 STEP 353 H= 88:15 +CPU= 47.927 22:01:02 STEP 354 H= 88:30 +CPU= 46.702 22:01:50 STEP 355 H= 88:45 +CPU= 47.479 22:02:38 STEP 356 H= 89:00 +CPU= 48.514 22:03:29 STEP 357 H= 89:15 +CPU= 49.712 22:04:19 STEP 358 H= 89:30 +CPU= 47.984 22:05:13 STEP 359 H= 89:45 +CPU= 53.008 22:06:19 STEP 360 H= 90:00 +CPU= 64.425 Moving to projects directory: /var/lib/boinc-client/slots/39/ICMGGh7zg+000360 Moving to projects directory: /var/lib/boinc-client/slots/39/ICMSHh7zg+000360 Moving to projects directory: /var/lib/boinc-client/slots/39/ICMUAh7zg+000360 22:07:11 STEP 361 H= 90:15 +CPU= 51.029 22:07:59 STEP 362 H= 90:30 +CPU= 47.723 22:08:48 STEP 363 H= 90:45 +CPU= 48.204 22:09:37 STEP 364 H= 91:00 +CPU= 48.793 22:10:25 STEP 365 H= 91:15 +CPU= 47.195 22:11:17 STEP 366 H= 91:30 +CPU= 49.818 22:12:08 STEP 367 H= 91:45 +CPU= 49.852 22:12:56 STEP 368 H= 92:00 +CPU= 47.283 22:13:45 STEP 369 H= 92:15 +CPU= 49.006 22:14:35 STEP 370 H= 92:30 +CPU= 46.895 22:15:19 STEP 371 H= 92:45 +CPU= 44.221 22:16:23 STEP 372 H= 93:00 +CPU= 55.908 Moving to projects directory: /var/lib/boinc-client/slots/39/ICMGGh7zg+000372 Moving to projects directory: /var/lib/boinc-client/slots/39/ICMSHh7zg+000372 Moving to projects directory: /var/lib/boinc-client/slots/39/ICMUAh7zg+000372 22:17:13 STEP 373 H= 93:15 +CPU= 46.330 22:18:16 STEP 374 H= 93:30 +CPU= 44.849 22:19:01 STEP 375 H= 93:45 +CPU= 43.067 22:19:51 STEP 376 H= 94:00 +CPU= 44.874 22:20:37 STEP 377 H= 94:15 +CPU= 45.287 22:21:34 STEP 378 H= 94:30 +CPU= 46.986 22:22:19 STEP 379 H= 94:45 +CPU= 44.591 22:23:09 STEP 380 H= 95:00 +CPU= 49.678 22:23:57 STEP 381 H= 95:15 +CPU= 45.933 22:24:47 STEP 382 H= 95:30 +CPU= 47.173 22:25:37 STEP 383 H= 95:45 +CPU= 48.743 22:26:54 STEP 384 H= 96:00 +CPU= 66.593 Moving to projects directory: /var/lib/boinc-client/slots/39/ICMGGh7zg+000384 Moving to projects directory: /var/lib/boinc-client/slots/39/ICMSHh7zg+000384 Moving to projects directory: /var/lib/boinc-client/slots/39/ICMUAh7zg+000384 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211720/ICMGGh7zg+000288 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211720/ICMSHh7zg+000288 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211720/ICMUAh7zg+000288 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211720/ICMGGh7zg+000300 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211720/ICMSHh7zg+000300 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211720/ICMUAh7zg+000300 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211720/ICMGGh7zg+000312 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211720/ICMSHh7zg+000312 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211720/ICMUAh7zg+000312 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211720/ICMGGh7zg+000324 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211720/ICMSHh7zg+000324 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211720/ICMUAh7zg+000324 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211720/ICMGGh7zg+000336 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211720/ICMSHh7zg+000336 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211720/ICMUAh7zg+000336 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211720/ICMGGh7zg+000348 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211720/ICMSHh7zg+000348 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211720/ICMUAh7zg+000348 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211720/ICMGGh7zg+000360 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211720/ICMSHh7zg+000360 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211720/ICMUAh7zg+000360 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211720/ICMGGh7zg+000372 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211720/ICMSHh7zg+000372 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211720/ICMUAh7zg+000372 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a3t1_2016092300_15_991_12211720_0_r2123028782_3.zip Uploading the intermediate file: upload_file_3.zip 22:27:44 STEP 385 H= 96:15 +CPU= 47.207 22:28:29 STEP 386 H= 96:30 +CPU= 45.216 22:29:46 STEP 387 H= 96:45 +CPU= 49.728 22:30:36 STEP 388 H= 97:00 +CPU= 48.277 Quit request received from BOINC client, ending the child process (argv0) ../../projects/climateprediction.net/oifs_43r3_bl_1.11_x86_64-pc-linux-gnu (argv1) start_date: 2016092300 (argv2) exptid: h7zg (argv3) unique_member_id: a3t1 (argv4) batchid: 991 (argv5) wuid: 12211720 (argv6) fclen: 15 (argv7) app_name: oifs_43r3_bl (argv8) nthreads: 1 Working directory is: /var/lib/boinc-client/slots/39 Project directory is: /var/lib/boinc-client/projects/climateprediction.net/ app name: oifs_43r3_bl version: 1.11 Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211720 ..mkdir for temp folder for results failed Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_app_1.11_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/39/oifs_43r3_bl_app_1.11_x86_64-pc-linux-gnu.zip Unzipping the app zip file: /var/lib/boinc-client/slots/39/oifs_43r3_bl_app_1.11_x86_64-pc-linux-gnu.zip Copying the namelist files from: ../../projects/climateprediction.net/jf_8a660c23fc0cf7a33fe79a39cb63b992 to: /var/lib/boinc-client/slots/39/oifs_43r3_bl_a3t1_2016092300_15_991_12211720.zip Unzipping the namelist zip file: /var/lib/boinc-client/slots/39/oifs_43r3_bl_a3t1_2016092300_15_991_12211720.zip ic_ancil_file: ic_ancil_12211720 ifsdata_file: ifsdata_12211720 climate_data_file: clim_data_12211720 horiz_resolution: 159 vert_resolution: 91 grid_type: l_2 upload_interval: 96 utstep: 900 nfrres: restart dump frequency (steps) 48 Copying IC ancils from: ../../projects/climateprediction.net/jf_f7ecfd27aa1e68bbe7e8d19800e25c73 to: /var/lib/boinc-client/slots/39/ic_ancil_12211720.zip Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/39/ic_ancil_12211720.zip ..mkdir for ifsdata folder failed Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_c442047aac50823c47ff0af11442c9b4 to: /var/lib/boinc-client/slots/39/ifsdata/ifsdata_12211720.zip Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/39/ifsdata/ifsdata_12211720.zip ..mkdir for the climate data folder failed Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/39/159l_2/clim_data_12211720.zip Unzipping the climate data zip file: /var/lib/boinc-client/slots/39/159l_2/clim_data_12211720.zip Checking for progress XML file: /var/lib/boinc-client/slots/39/progress_file_12211720.xml Opened progress file ok : /var/lib/boinc-client/slots/39/progress_file_12211720.xml -- Model is restarting -- Adjusting last_iter, 389, to previous model restart step. Creating progress file: /var/lib/boinc-client/slots/39/progress_file_12211720.xml last_cpu_time: 31928 upload_file_number: 4 last_iter: 385 last_upload: 345600 model_completed: 0 total_length_of_simulation: 1296000 result_base_name: oifs_43r3_bl_a3t1_2016092300_15_991_12211720_0_r2123028782 The child process has been launched with process id: 75403 Executing the command: /var/lib/boinc-client/slots/39/oifs_43r3_model.exe [EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1 [EC_DRHOOK:boinc5:1:1:75403:75403] [20230215:223429:1676500469:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2091] fp = 0x30e4a00 [EC_DRHOOK:boinc5:1:1:75403:75403] [20230215:223429:1676500469:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1 [EC_DRHOOK:boinc5:1:1:75403:75403] [20230215:223429:1676500469:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0) [EC_DRHOOK:boinc5:1:1:75403:75403] [20230215:223429:1676500469:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1 [EC_DRHOOK:boinc5:1:1:75403:75403] [20230215:223429:1676500469:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000 [EC_DRHOOK:boinc5:1:1:75403:75403] [20230215:223429:1676500469:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647) [EC_DRHOOK:boinc5:1:1:75403:75403] [20230215:223429:1676500469:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16 [EC_DRHOOK:boinc5:1:1:75403:75403] [20230215:223429:1676500469:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0 [EC_DRHOOK:boinc5:1:1:75403:75403] [20230215:223429:1676500469:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500 [EC_DRHOOK:boinc5:1:1:75403:75403] [20230215:223429:1676500469:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1 [EC_DRHOOK:boinc5:1:1:75403:75403] [20230215:223429:1676500469:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1 [EC_DRHOOK:boinc5:1:1:75403:75403] [20230215:223429:1676500469:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1 [EC_DRHOOK:boinc5:1:1:75403:75403] [20230215:223429:1676500469:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1 [EC_DRHOOK:boinc5:1:1:75403:75403] [20230215:223429:1676500469:0.000] [ignore_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef> [EC_DRHOOK:boinc5:1:1:75403:75403] [20230215:223429:1676500469:0.000] [restore_default_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef> [EC_DRHOOK:boinc5:1:1:75403:75403] [20230215:223429:1676500469:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f520 (old at 0x401140) [EC_DRHOOK:boinc5:1:1:75403:75403] [20230215:223429:1676500469:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f520 (old at (nil)) [EC_DRHOOK:boinc5:1:1:75403:75403] [20230215:223429:1676500469:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f520 (old at 0x401140) [EC_DRHOOK:boinc5:1:1:75403:75403] [20230215:223429:1676500469:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f520 (old at (nil)) [EC_DRHOOK:boinc5:1:1:75403:75403] [20230215:223429:1676500469:0.001] [trapfpe_treatment@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1 [EC_DRHOOK:boinc5:1:1:75403:75403] [20230215:223429:1676500469:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f520 (old at 0x401140) [EC_DRHOOK:boinc5:1:1:75403:75403] [20230215:223429:1676500469:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f520 (old at 0x401140) [EC_DRHOOK:boinc5:1:1:75403:75403] [20230215:223429:1676500469:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f520 (old at (nil)) [EC_DRHOOK:boinc5:1:1:75403:75403] [20230215:223429:1676500469:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f520 (old at 0x401140) [EC_DRHOOK:boinc5:1:1:75403:75403] [20230215:223429:1676500469:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f520 (old at 0x401140) [EC_DRHOOK:boinc5:1:1:75403:75403] [20230215:223429:1676500469:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f520 (old at 0x401140) [EC_DRHOOK:boinc5:1:1:75403:75403] [20230215:223429:1676500469:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f520 (old at (nil)) [EC_DRHOOK:boinc5:1:1:75403:75403] [20230215:223429:1676500469:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f520 (old at (nil)) [EC_DRHOOK:boinc5:1:1:75403:75403] [20230215:223429:1676500469:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f520 (old at (nil)) [EC_DRHOOK:boinc5:1:1:75403:75403] [20230215:223429:1676500469:0.001] [catch_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef> 22:35:32 STEP 384 H= 96:00 +CPU= 48.317 22:36:13 STEP 385 H= 96:15 +CPU= 41.136 22:37:01 STEP 386 H= 96:30 +CPU= 47.389 22:37:49 STEP 387 H= 96:45 +CPU= 46.468 22:38:34 STEP 388 H= 97:00 +CPU= 45.532 22:39:24 STEP 389 H= 97:15 +CPU= 47.016 22:40:09 STEP 390 H= 97:30 +CPU= 43.969 22:40:53 STEP 391 H= 97:45 +CPU= 43.990 22:41:41 STEP 392 H= 98:00 +CPU= 47.558 22:42:32 STEP 393 H= 98:15 +CPU= 47.051 22:43:24 STEP 394 H= 98:30 +CPU= 50.726 22:44:18 STEP 395 H= 98:45 +CPU= 51.821 22:45:22 STEP 396 H= 99:00 +CPU= 62.065 Moving to projects directory: /var/lib/boinc-client/slots/39/ICMGGh7zg+000396 Moving to projects directory: /var/lib/boinc-client/slots/39/ICMSHh7zg+000396 Moving to projects directory: /var/lib/boinc-client/slots/39/ICMUAh7zg+000396 22:46:09 STEP 397 H= 99:15 +CPU= 47.246 22:46:56 STEP 398 H= 99:30 +CPU= 46.096 22:47:43 STEP 399 H= 99:45 +CPU= 47.592 22:48:33 STEP 400 H= 100:00 +CPU= 48.349 22:49:19 STEP 401 H= 100:15 +CPU= 46.110 22:50:06 STEP 402 H= 100:30 +CPU= 46.067 22:50:53 STEP 403 H= 100:45 +CPU= 46.727 22:51:41 STEP 404 H= 101:00 +CPU= 47.419 22:52:27 STEP 405 H= 101:15 +CPU= 45.435 22:53:11 STEP 406 H= 101:30 +CPU= 44.174 22:53:56 STEP 407 H= 101:45 +CPU= 44.614 22:54:54 STEP 408 H= 102:00 +CPU= 56.617 Moving to projects directory: /var/lib/boinc-client/slots/39/ICMGGh7zg+000408 Moving to projects directory: /var/lib/boinc-client/slots/39/ICMSHh7zg+000408 Moving to projects directory: /var/lib/boinc-client/slots/39/ICMUAh7zg+000408 22:55:41 STEP 409 H= 102:15 +CPU= 46.948 22:56:28 STEP 410 H= 102:30 +CPU= 46.196 22:57:14 STEP 411 H= 102:45 +CPU= 44.785 22:58:02 STEP 412 H= 103:00 +CPU= 48.086 22:58:45 STEP 413 H= 103:15 +CPU= 42.313 22:59:29 STEP 414 H= 103:30 +CPU= 43.973 23:00:19 STEP 415 H= 103:45 +CPU= 47.917 23:01:04 STEP 416 H= 104:00 +CPU= 45.529 23:01:52 STEP 417 H= 104:15 +CPU= 46.022 23:02:44 STEP 418 H= 104:30 +CPU= 48.554 23:03:31 STEP 419 H= 104:45 +CPU= 46.073 23:04:31 STEP 420 H= 105:00 +CPU= 56.532 Moving to projects directory: /var/lib/boinc-client/slots/39/ICMGGh7zg+000420 Moving to projects directory: /var/lib/boinc-client/slots/39/ICMSHh7zg+000420 Moving to projects directory: /var/lib/boinc-client/slots/39/ICMUAh7zg+000420 23:05:18 STEP 421 H= 105:15 +CPU= 46.011 23:06:07 STEP 422 H= 105:30 +CPU= 47.954 23:06:56 STEP 423 H= 105:45 +CPU= 49.405 23:07:52 STEP 424 H= 106:00 +CPU= 51.392 23:08:38 STEP 425 H= 106:15 +CPU= 45.317 23:09:25 STEP 426 H= 106:30 +CPU= 46.160 23:10:15 STEP 427 H= 106:45 +CPU= 46.066 23:11:03 STEP 428 H= 107:00 +CPU= 47.007 23:11:52 STEP 429 H= 107:15 +CPU= 48.087 23:12:40 STEP 430 H= 107:30 +CPU= 44.132 23:13:24 STEP 431 H= 107:45 +CPU= 43.709 23:14:34 STEP 432 H= 108:00 +CPU= 61.558 Moving to projects directory: /var/lib/boinc-client/slots/39/ICMGGh7zg+000432 Moving to projects directory: /var/lib/boinc-client/slots/39/ICMSHh7zg+000432 Moving to projects directory: /var/lib/boinc-client/slots/39/ICMUAh7zg+000432 23:15:19 STEP 433 H= 108:15 +CPU= 44.632 23:16:13 STEP 434 H= 108:30 +CPU= 51.466 23:16:58 STEP 435 H= 108:45 +CPU= 44.886 23:17:44 STEP 436 H= 109:00 +CPU= 45.553 23:18:32 STEP 437 H= 109:15 +CPU= 46.249 23:19:18 STEP 438 H= 109:30 +CPU= 45.225 23:20:06 STEP 439 H= 109:45 +CPU= 47.674 23:20:53 STEP 440 H= 110:00 +CPU= 46.505 23:21:41 STEP 441 H= 110:15 +CPU= 47.358 23:22:29 STEP 442 H= 110:30 +CPU= 45.287 23:23:16 STEP 443 H= 110:45 +CPU= 46.758 23:24:26 STEP 444 H= 111:00 +CPU= 63.118 Moving to projects directory: /var/lib/boinc-client/slots/39/ICMGGh7zg+000444 Moving to projects directory: /var/lib/boinc-client/slots/39/ICMSHh7zg+000444 Moving to projects directory: /var/lib/boinc-client/slots/39/ICMUAh7zg+000444 23:25:17 STEP 445 H= 111:15 +CPU= 46.227 23:26:03 STEP 446 H= 111:30 +CPU= 46.130 23:26:50 STEP 447 H= 111:45 +CPU= 45.496 23:27:38 STEP 448 H= 112:00 +CPU= 47.437 23:28:25 STEP 449 H= 112:15 +CPU= 45.955 23:29:11 STEP 450 H= 112:30 +CPU= 45.573 23:30:03 STEP 451 H= 112:45 +CPU= 47.004 23:30:57 STEP 452 H= 113:00 +CPU= 46.632 23:31:44 STEP 453 H= 113:15 +CPU= 45.942 23:32:34 STEP 454 H= 113:30 +CPU= 47.270 23:33:20 STEP 455 H= 113:45 +CPU= 45.407 23:34:28 STEP 456 H= 114:00 +CPU= 62.512 Moving to projects directory: /var/lib/boinc-client/slots/39/ICMGGh7zg+000456 Moving to projects directory: /var/lib/boinc-client/slots/39/ICMSHh7zg+000456 Moving to projects directory: /var/lib/boinc-client/slots/39/ICMUAh7zg+000456 23:35:15 STEP 457 H= 114:15 +CPU= 46.305 23:36:15 STEP 458 H= 114:30 +CPU= 52.158 23:37:02 STEP 459 H= 114:45 +CPU= 45.255 23:37:59 STEP 460 H= 115:00 +CPU= 48.709 23:38:47 STEP 461 H= 115:15 +CPU= 47.587 23:39:41 STEP 462 H= 115:30 +CPU= 47.557 23:40:28 STEP 463 H= 115:45 +CPU= 45.504 23:41:40 STEP 464 H= 116:00 +CPU= 55.167 23:43:01 STEP 465 H= 116:15 +CPU= 54.453 23:44:48 STEP 466 H= 116:30 +CPU= 52.416 23:46:05 STEP 467 H= 116:45 +CPU= 48.199 23:47:34 STEP 468 H= 117:00 +CPU= 63.039 Moving to projects directory: /var/lib/boinc-client/slots/39/ICMGGh7zg+000468 Moving to projects directory: /var/lib/boinc-client/slots/39/ICMSHh7zg+000468 Moving to projects directory: /var/lib/boinc-client/slots/39/ICMUAh7zg+000468 23:48:59 STEP 469 H= 117:15 +CPU= 46.339 23:50:02 STEP 470 H= 117:30 +CPU= 46.313 23:50:50 STEP 471 H= 117:45 +CPU= 46.981 23:51:47 STEP 472 H= 118:00 +CPU= 48.720 23:52:40 STEP 473 H= 118:15 +CPU= 46.966 23:53:30 STEP 474 H= 118:30 +CPU= 46.417 23:54:40 STEP 475 H= 118:45 +CPU= 47.196 23:56:28 STEP 476 H= 119:00 +CPU= 47.844 23:57:33 STEP 477 H= 119:15 +CPU= 46.523 23:58:28 STEP 478 H= 119:30 +CPU= 45.311 23:59:46 STEP 479 H= 119:45 +CPU= 46.980 Quit request received from BOINC client, ending the child process (argv0) ../../projects/climateprediction.net/oifs_43r3_bl_1.11_x86_64-pc-linux-gnu (argv1) start_date: 2016092300 (argv2) exptid: h7zg (argv3) unique_member_id: a3t1 (argv4) batchid: 991 (argv5) wuid: 12211720 (argv6) fclen: 15 (argv7) app_name: oifs_43r3_bl (argv8) nthreads: 1 Working directory is: /var/lib/boinc-client/slots/39 Project directory is: /var/lib/boinc-client/projects/climateprediction.net/ app name: oifs_43r3_bl version: 1.11 Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211720 ..mkdir for temp folder for results failed Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_app_1.11_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/39/oifs_43r3_bl_app_1.11_x86_64-pc-linux-gnu.zip Unzipping the app zip file: /var/lib/boinc-client/slots/39/oifs_43r3_bl_app_1.11_x86_64-pc-linux-gnu.zip Copying the namelist files from: ../../projects/climateprediction.net/jf_8a660c23fc0cf7a33fe79a39cb63b992 to: /var/lib/boinc-client/slots/39/oifs_43r3_bl_a3t1_2016092300_15_991_12211720.zip Unzipping the namelist zip file: /var/lib/boinc-client/slots/39/oifs_43r3_bl_a3t1_2016092300_15_991_12211720.zip ic_ancil_file: ic_ancil_12211720 ifsdata_file: ifsdata_12211720 climate_data_file: clim_data_12211720 horiz_resolution: 159 vert_resolution: 91 grid_type: l_2 upload_interval: 96 utstep: 900 nfrres: restart dump frequency (steps) 48 Copying IC ancils from: ../../projects/climateprediction.net/jf_f7ecfd27aa1e68bbe7e8d19800e25c73 to: /var/lib/boinc-client/slots/39/ic_ancil_12211720.zip Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/39/ic_ancil_12211720.zip ..mkdir for ifsdata folder failed Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_c442047aac50823c47ff0af11442c9b4 to: /var/lib/boinc-client/slots/39/ifsdata/ifsdata_12211720.zip Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/39/ifsdata/ifsdata_12211720.zip ..mkdir for the climate data folder failed Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/39/159l_2/clim_data_12211720.zip Unzipping the climate data zip file: /var/lib/boinc-client/slots/39/159l_2/clim_data_12211720.zip Checking for progress XML file: /var/lib/boinc-client/slots/39/progress_file_12211720.xml Opened progress file ok : /var/lib/boinc-client/slots/39/progress_file_12211720.xml -- Model is restarting -- Adjusting last_iter, 480, to previous model restart step. Creating progress file: /var/lib/boinc-client/slots/39/progress_file_12211720.xml last_cpu_time: 36560 upload_file_number: 4 last_iter: 481 last_upload: 345600 model_completed: 0 total_length_of_simulation: 1296000 result_base_name: oifs_43r3_bl_a3t1_2016092300_15_991_12211720_0_r2123028782 The child process has been launched with process id: 78177 Executing the command: /var/lib/boinc-client/slots/39/oifs_43r3_model.exe [EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1 [EC_DRHOOK:boinc5:1:1:78177:78177] [20230216:000249:1676505769:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2091] fp = 0x30e4a00 [EC_DRHOOK:boinc5:1:1:78177:78177] [20230216:000249:1676505769:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1 [EC_DRHOOK:boinc5:1:1:78177:78177] [20230216:000249:1676505769:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0) [EC_DRHOOK:boinc5:1:1:78177:78177] [20230216:000249:1676505769:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1 [EC_DRHOOK:boinc5:1:1:78177:78177] [20230216:000249:1676505769:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000 [EC_DRHOOK:boinc5:1:1:78177:78177] [20230216:000249:1676505769:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647) [EC_DRHOOK:boinc5:1:1:78177:78177] [20230216:000249:1676505769:0.001] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16 [EC_DRHOOK:boinc5:1:1:78177:78177] [20230216:000249:1676505769:0.001] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0 [EC_DRHOOK:boinc5:1:1:78177:78177] [20230216:000249:1676505769:0.001] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500 [EC_DRHOOK:boinc5:1:1:78177:78177] [20230216:000249:1676505769:0.001] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1 [EC_DRHOOK:boinc5:1:1:78177:78177] [20230216:000249:1676505769:0.001] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1 [EC_DRHOOK:boinc5:1:1:78177:78177] [20230216:000249:1676505769:0.001] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1 [EC_DRHOOK:boinc5:1:1:78177:78177] [20230216:000249:1676505769:0.001] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1 [EC_DRHOOK:boinc5:1:1:78177:78177] [20230216:000249:1676505769:0.001] [ignore_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef> [EC_DRHOOK:boinc5:1:1:78177:78177] [20230216:000249:1676505769:0.001] [restore_default_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef> [EC_DRHOOK:boinc5:1:1:78177:78177] [20230216:000249:1676505769:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f520 (old at 0x401140) [EC_DRHOOK:boinc5:1:1:78177:78177] [20230216:000249:1676505769:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f520 (old at (nil)) [EC_DRHOOK:boinc5:1:1:78177:78177] [20230216:000249:1676505769:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f520 (old at 0x401140) [EC_DRHOOK:boinc5:1:1:78177:78177] [20230216:000249:1676505769:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f520 (old at (nil)) [EC_DRHOOK:boinc5:1:1:78177:78177] [20230216:000249:1676505769:0.001] [trapfpe_treatment@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1 [EC_DRHOOK:boinc5:1:1:78177:78177] [20230216:000249:1676505769:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f520 (old at 0x401140) [EC_DRHOOK:boinc5:1:1:78177:78177] [20230216:000249:1676505769:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f520 (old at 0x401140) [EC_DRHOOK:boinc5:1:1:78177:78177] [20230216:000249:1676505769:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f520 (old at (nil)) [EC_DRHOOK:boinc5:1:1:78177:78177] [20230216:000249:1676505769:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f520 (old at 0x401140) [EC_DRHOOK:boinc5:1:1:78177:78177] [20230216:000249:1676505769:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f520 (old at 0x401140) [EC_DRHOOK:boinc5:1:1:78177:78177] [20230216:000249:1676505769:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f520 (old at 0x401140) [EC_DRHOOK:boinc5:1:1:78177:78177] [20230216:000249:1676505769:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f520 (old at (nil)) [EC_DRHOOK:boinc5:1:1:78177:78177] [20230216:000249:1676505769:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f520 (old at (nil)) [EC_DRHOOK:boinc5:1:1:78177:78177] [20230216:000249:1676505769:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f520 (old at (nil)) [EC_DRHOOK:boinc5:1:1:78177:78177] [20230216:000249:1676505769:0.001] [catch_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef> Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211720/ICMGGh7zg+000384 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211720/ICMSHh7zg+000384 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211720/ICMUAh7zg+000384 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211720/ICMGGh7zg+000396 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211720/ICMSHh7zg+000396 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211720/ICMUAh7zg+000396 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211720/ICMGGh7zg+000408 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211720/ICMSHh7zg+000408 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211720/ICMUAh7zg+000408 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211720/ICMGGh7zg+000420 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211720/ICMSHh7zg+000420 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211720/ICMUAh7zg+000420 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211720/ICMGGh7zg+000432 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211720/ICMSHh7zg+000432 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211720/ICMUAh7zg+000432 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211720/ICMGGh7zg+000444 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211720/ICMSHh7zg+000444 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211720/ICMUAh7zg+000444 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211720/ICMGGh7zg+000456 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211720/ICMSHh7zg+000456 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211720/ICMUAh7zg+000456 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211720/ICMGGh7zg+000468 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211720/ICMSHh7zg+000468 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211720/ICMUAh7zg+000468 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a3t1_2016092300_15_991_12211720_0_r2123028782_4.zip Uploading the intermediate file: upload_file_4.zip 00:04:02 STEP 480 H= 120:00 +CPU= 52.987 Moving to projects directory: /var/lib/boinc-client/slots/39/ICMGGh7zg+000480 Moving to projects directory: /var/lib/boinc-client/slots/39/ICMSHh7zg+000480 Moving to projects directory: /var/lib/boinc-client/slots/39/ICMUAh7zg+000480 free(): invalid pointer double free or corruption (out) </stderr_txt> ]]> |
No trickles! |
---|
©2024 climateprediction.net