Name | oifs_43r3_ps_0333_1999050100_123_968_12184977_0 |
Workunit | 12184977 |
Created | 22 Dec 2022, 18:33:02 UTC |
Sent | 19 Jan 2023, 0:52:35 UTC |
Report deadline | 20 Mar 2023, 0:52:35 UTC |
Received | 5 Feb 2023, 15:43:13 UTC |
Server state | Over |
Outcome | Computation error |
Client state | Compute error |
Exit status | 196 (0x000000C4) EXIT_DISK_LIMIT_EXCEEDED |
Computer ID | 1509518 |
Run time | 11 hours 16 min 16 sec |
CPU time | 9 hours 51 min 10 sec |
Validate state | Invalid |
Credit | 3,507.84 |
Device peak FLOPS | 4.11 GFLOPS |
Application version | OpenIFS 43r3 Perturbed Surface v1.05 x86_64-pc-linux-gnu |
Peak working set size | 4,418.24 MB |
Peak swap size | 4,978.07 MB |
Peak disk usage | 7,690.38 MB |
Stderr | <core_client_version>7.16.6</core_client_version> <![CDATA[ <message> Disk usage limit exceeded</message> <stderr_txt> l#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:ruly-Inspiron-5558:1:1:1931:1931] [20230204:213603:1675575363:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:ruly-Inspiron-5558:1:1:1931:1931] [20230204:213603:1675575363:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil)) [EC_DRHOOK:ruly-Inspiron-5558:1:1:1931:1931] [20230204:213603:1675575363:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:ruly-Inspiron-5558:1:1:1931:1931] [20230204:213603:1675575363:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:ruly-Inspiron-5558:1:1:1931:1931] [20230204:213603:1675575363:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:ruly-Inspiron-5558:1:1:1931:1931] [20230204:213603:1675575363:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil)) [EC_DRHOOK:ruly-Inspiron-5558:1:1:1931:1931] [20230204:213603:1675575363:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil)) [EC_DRHOOK:ruly-Inspiron-5558:1:1:1931:1931] [20230204:213603:1675575363:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil)) [EC_DRHOOK:ruly-Inspiron-5558:1:1:1931:1931] [20230204:213603:1675575363:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef> Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 05:37:49 STEP 744 H= 744:00 +CPU= 62.728 05:38:29 STEP 745 H= 745:00 +CPU= 37.715 Quit request received from BOINC client, ending the child process ..Failed, model did not complete successfully (argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu (argv1) start_date: 1999050100 (argv2) exptid: hq0f (argv3) unique_member_id: 0333 (argv4) batchid: 968 (argv5) wuid: 12184977 (argv6) fclen: 123 (argv7) app_name: oifs_43r3_ps (argv8) nthreads: 1 Working directory is: /var/lib/boinc-client/slots/1 Project directory is: /var/lib/boinc-client/projects/climateprediction.net/ app name: oifs_43r3_ps version: 1.05 Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184977 ..mkdir for temp folder for results failed Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/1/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip Unzipping the app zip file: /var/lib/boinc-client/slots/1/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip Copying the namelist files from: ../../projects/climateprediction.net/jf_0a2293733ba4099fbe133f2b8f3de008 to: /var/lib/boinc-client/slots/1/oifs_43r3_ps_0333_1999050100_123_968_12184977.zip Unzipping the namelist zip file: /var/lib/boinc-client/slots/1/oifs_43r3_ps_0333_1999050100_123_968_12184977.zip ic_ancil_file: ic_ancil_12184977 ifsdata_file: ifsdata_12184977 climate_data_file: clim_data_12184977 horiz_resolution: 159 vert_resolution: 60 grid_type: l_2 upload_interval: 24 utstep: 3600 nfrres: restart dump frequency (steps) 24 Copying IC ancils from: ../../projects/climateprediction.net/jf_300ad98e3e727469f31e1eaa7b8b5640 to: /var/lib/boinc-client/slots/1/ic_ancil_12184977.zip Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/1/ic_ancil_12184977.zip ..mkdir for ifsdata folder failed Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_422ffaf33a1d0a51301a373adc594014 to: /var/lib/boinc-client/slots/1/ifsdata/ifsdata_12184977.zip Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/1/ifsdata/ifsdata_12184977.zip ..mkdir for the climate data folder failed Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/1/159l_2/clim_data_12184977.zip Unzipping the climate data zip file: /var/lib/boinc-client/slots/1/159l_2/clim_data_12184977.zip Checking for progress XML file: /var/lib/boinc-client/slots/1/progress_file_12184977.xml Opened progress file ok : /var/lib/boinc-client/slots/1/progress_file_12184977.xml -- Model is restarting -- Adjusting last_iter, 746, to previous model restart step. Creating progress file: /var/lib/boinc-client/slots/1/progress_file_12184977.xml last_cpu_time: 29984 upload_file_number: 31 last_iter: 743 last_upload: 2678400 model_completed: 0 total_length_of_simulation: 10627200 result_base_name: oifs_43r3_ps_0333_1999050100_123_968_12184977_0_r2137061363 The child process has been launched with process id: 4901 Executing the command: /var/lib/boinc-client/slots/1/oifs_43r3_model.exe [EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1 [EC_DRHOOK:ruly-Inspiron-5558:1:1:4901:4901] [20230205:054025:1675604425:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0 [EC_DRHOOK:ruly-Inspiron-5558:1:1:4901:4901] [20230205:054025:1675604425:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1 [EC_DRHOOK:ruly-Inspiron-5558:1:1:4901:4901] [20230205:054025:1675604425:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0) [EC_DRHOOK:ruly-Inspiron-5558:1:1:4901:4901] [20230205:054025:1675604425:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1 [EC_DRHOOK:ruly-Inspiron-5558:1:1:4901:4901] [20230205:054025:1675604425:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000 [EC_DRHOOK:ruly-Inspiron-5558:1:1:4901:4901] [20230205:054025:1675604425:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647) [EC_DRHOOK:ruly-Inspiron-5558:1:1:4901:4901] [20230205:054025:1675604425:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16 [EC_DRHOOK:ruly-Inspiron-5558:1:1:4901:4901] [20230205:054025:1675604425:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0 [EC_DRHOOK:ruly-Inspiron-5558:1:1:4901:4901] [20230205:054025:1675604425:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500 [EC_DRHOOK:ruly-Inspiron-5558:1:1:4901:4901] [20230205:054025:1675604425:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1 [EC_DRHOOK:ruly-Inspiron-5558:1:1:4901:4901] [20230205:054025:1675604425:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1 [EC_DRHOOK:ruly-Inspiron-5558:1:1:4901:4901] [20230205:054025:1675604425:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1 [EC_DRHOOK:ruly-Inspiron-5558:1:1:4901:4901] [20230205:054025:1675604425:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1 [EC_DRHOOK:ruly-Inspiron-5558:1:1:4901:4901] [20230205:054025:1675604425:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef> [EC_DRHOOK:ruly-Inspiron-5558:1:1:4901:4901] [20230205:054025:1675604425:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef> [EC_DRHOOK:ruly-Inspiron-5558:1:1:4901:4901] [20230205:054025:1675604425:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:ruly-Inspiron-5558:1:1:4901:4901] [20230205:054025:1675604425:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil)) [EC_DRHOOK:ruly-Inspiron-5558:1:1:4901:4901] [20230205:054025:1675604425:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:ruly-Inspiron-5558:1:1:4901:4901] [20230205:054025:1675604425:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil)) [EC_DRHOOK:ruly-Inspiron-5558:1:1:4901:4901] [20230205:054025:1675604425:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1 [EC_DRHOOK:ruly-Inspiron-5558:1:1:4901:4901] [20230205:054025:1675604425:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:ruly-Inspiron-5558:1:1:4901:4901] [20230205:054025:1675604425:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:ruly-Inspiron-5558:1:1:4901:4901] [20230205:054025:1675604425:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil)) [EC_DRHOOK:ruly-Inspiron-5558:1:1:4901:4901] [20230205:054025:1675604425:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:ruly-Inspiron-5558:1:1:4901:4901] [20230205:054025:1675604425:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:ruly-Inspiron-5558:1:1:4901:4901] [20230205:054025:1675604425:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:ruly-Inspiron-5558:1:1:4901:4901] [20230205:054025:1675604425:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil)) [EC_DRHOOK:ruly-Inspiron-5558:1:1:4901:4901] [20230205:054025:1675604425:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil)) [EC_DRHOOK:ruly-Inspiron-5558:1:1:4901:4901] [20230205:054025:1675604425:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil)) [EC_DRHOOK:ruly-Inspiron-5558:1:1:4901:4901] [20230205:054025:1675604425:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef> Suspend request received from the BOINC client, suspending the child process Resuming the child process 05:42:18 STEP 744 H= 744:00 +CPU= 50.316 05:42:48 STEP 745 H= 745:00 +CPU= 27.517 Suspend request received from the BOINC client, suspending the child process Resuming the child process 05:44:02 STEP 746 H= 746:00 +CPU= 34.760 Suspend request received from the BOINC client, suspending the child process Resuming the child process 05:45:15 STEP 747 H= 747:00 +CPU= 53.486 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 05:47:20 STEP 748 H= 748:00 +CPU= 35.932 Suspend request received from the BOINC client, suspending the child process Resuming the child process 05:48:01 STEP 749 H= 749:00 +CPU= 27.430 Suspend request received from the BOINC client, suspending the child process Resuming the child process 05:49:02 STEP 750 H= 750:00 +CPU= 45.400 05:49:25 STEP 751 H= 751:00 +CPU= 22.645 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 05:50:40 STEP 752 H= 752:00 +CPU= 36.033 Suspend request received from the BOINC client, suspending the child process Resuming the child process 05:52:24 STEP 753 H= 753:00 +CPU= 52.850 05:53:00 STEP 754 H= 754:00 +CPU= 31.912 05:53:28 STEP 755 H= 755:00 +CPU= 26.608 05:54:24 STEP 756 H= 756:00 +CPU= 51.716 Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000756 Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000756 Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000756 05:54:50 STEP 757 H= 757:00 +CPU= 24.870 05:55:25 STEP 758 H= 758:00 +CPU= 32.383 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 05:57:32 STEP 759 H= 759:00 +CPU= 48.395 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 05:58:45 STEP 760 H= 760:00 +CPU= 35.204 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 05:59:46 STEP 761 H= 761:00 +CPU= 34.361 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 06:01:43 STEP 762 H= 762:00 +CPU= 46.963 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 06:03:37 STEP 763 H= 763:00 +CPU= 34.771 Suspend request received from the BOINC client, suspending the child process Resuming the child process 06:04:22 STEP 764 H= 764:00 +CPU= 31.183 Suspend request received from the BOINC client, suspending the child process Resuming the child process 06:05:27 STEP 765 H= 765:00 +CPU= 49.461 06:05:58 STEP 766 H= 766:00 +CPU= 28.495 06:06:31 STEP 767 H= 767:00 +CPU= 31.505 Suspend request received from the BOINC client, suspending the child process Resuming the child process 06:07:55 STEP 768 H= 768:00 +CPU= 56.121 Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000768 Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000768 Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000768 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184977/ICMGGhq0f+000744 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184977/ICMSHhq0f+000744 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184977/ICMUAhq0f+000744 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184977/ICMGGhq0f+000756 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184977/ICMSHhq0f+000756 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184977/ICMUAhq0f+000756 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0333_1999050100_123_968_12184977_0_r2137061363_31.zip Uploading the intermediate file: upload_file_31.zip 06:08:32 STEP 769 H= 769:00 +CPU= 31.311 Suspend request received from the BOINC client, suspending the child process Resuming the child process 06:09:23 STEP 770 H= 770:00 +CPU= 28.366 Suspend request received from the BOINC client, suspending the child process Resuming the child process Quit request received from BOINC client, ending the child process (argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu (argv1) start_date: 1999050100 (argv2) exptid: hq0f (argv3) unique_member_id: 0333 (argv4) batchid: 968 (argv5) wuid: 12184977 (argv6) fclen: 123 (argv7) app_name: oifs_43r3_ps (argv8) nthreads: 1 Working directory is: /var/lib/boinc-client/slots/1 Project directory is: /var/lib/boinc-client/projects/climateprediction.net/ app name: oifs_43r3_ps version: 1.05 Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184977 ..mkdir for temp folder for results failed Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/1/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip Unzipping the app zip file: /var/lib/boinc-client/slots/1/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip Copying the namelist files from: ../../projects/climateprediction.net/jf_0a2293733ba4099fbe133f2b8f3de008 to: /var/lib/boinc-client/slots/1/oifs_43r3_ps_0333_1999050100_123_968_12184977.zip Unzipping the namelist zip file: /var/lib/boinc-client/slots/1/oifs_43r3_ps_0333_1999050100_123_968_12184977.zip ic_ancil_file: ic_ancil_12184977 ifsdata_file: ifsdata_12184977 climate_data_file: clim_data_12184977 horiz_resolution: 159 vert_resolution: 60 grid_type: l_2 upload_interval: 24 utstep: 3600 nfrres: restart dump frequency (steps) 24 Copying IC ancils from: ../../projects/climateprediction.net/jf_300ad98e3e727469f31e1eaa7b8b5640 to: /var/lib/boinc-client/slots/1/ic_ancil_12184977.zip Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/1/ic_ancil_12184977.zip ..mkdir for ifsdata folder failed Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_422ffaf33a1d0a51301a373adc594014 to: /var/lib/boinc-client/slots/1/ifsdata/ifsdata_12184977.zip Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/1/ifsdata/ifsdata_12184977.zip ..mkdir for the climate data folder failed Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/1/159l_2/clim_data_12184977.zip Unzipping the climate data zip file: /var/lib/boinc-client/slots/1/159l_2/clim_data_12184977.zip Checking for progress XML file: /var/lib/boinc-client/slots/1/progress_file_12184977.xml Opened progress file ok : /var/lib/boinc-client/slots/1/progress_file_12184977.xml -- Model is restarting -- Adjusting last_iter, 771, to previous model restart step. Creating progress file: /var/lib/boinc-client/slots/1/progress_file_12184977.xml last_cpu_time: 31006 upload_file_number: 32 last_iter: 767 last_upload: 2764800 model_completed: 0 total_length_of_simulation: 10627200 result_base_name: oifs_43r3_ps_0333_1999050100_123_968_12184977_0_r2137061363 The child process has been launched with process id: 8656 Executing the command: /var/lib/boinc-client/slots/1/oifs_43r3_model.exe [EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1 [EC_DRHOOK:ruly-Inspiron-5558:1:1:8656:8656] [20230205:061113:1675606273:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0 [EC_DRHOOK:ruly-Inspiron-5558:1:1:8656:8656] [20230205:061113:1675606273:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1 [EC_DRHOOK:ruly-Inspiron-5558:1:1:8656:8656] [20230205:061113:1675606273:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0) [EC_DRHOOK:ruly-Inspiron-5558:1:1:8656:8656] [20230205:061113:1675606273:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1 [EC_DRHOOK:ruly-Inspiron-5558:1:1:8656:8656] [20230205:061113:1675606273:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000 [EC_DRHOOK:ruly-Inspiron-5558:1:1:8656:8656] [20230205:061113:1675606273:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647) [EC_DRHOOK:ruly-Inspiron-5558:1:1:8656:8656] [20230205:061113:1675606273:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16 [EC_DRHOOK:ruly-Inspiron-5558:1:1:8656:8656] [20230205:061113:1675606273:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0 [EC_DRHOOK:ruly-Inspiron-5558:1:1:8656:8656] [20230205:061113:1675606273:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500 [EC_DRHOOK:ruly-Inspiron-5558:1:1:8656:8656] [20230205:061113:1675606273:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1 [EC_DRHOOK:ruly-Inspiron-5558:1:1:8656:8656] [20230205:061113:1675606273:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1 [EC_DRHOOK:ruly-Inspiron-5558:1:1:8656:8656] [20230205:061113:1675606273:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1 [EC_DRHOOK:ruly-Inspiron-5558:1:1:8656:8656] [20230205:061113:1675606273:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1 [EC_DRHOOK:ruly-Inspiron-5558:1:1:8656:8656] [20230205:061113:1675606273:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef> [EC_DRHOOK:ruly-Inspiron-5558:1:1:8656:8656] [20230205:061113:1675606273:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef> [EC_DRHOOK:ruly-Inspiron-5558:1:1:8656:8656] [20230205:061113:1675606273:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:ruly-Inspiron-5558:1:1:8656:8656] [20230205:061113:1675606273:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil)) [EC_DRHOOK:ruly-Inspiron-5558:1:1:8656:8656] [20230205:061113:1675606273:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:ruly-Inspiron-5558:1:1:8656:8656] [20230205:061113:1675606273:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil)) [EC_DRHOOK:ruly-Inspiron-5558:1:1:8656:8656] [20230205:061113:1675606273:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1 [EC_DRHOOK:ruly-Inspiron-5558:1:1:8656:8656] [20230205:061113:1675606273:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:ruly-Inspiron-5558:1:1:8656:8656] [20230205:061113:1675606273:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:ruly-Inspiron-5558:1:1:8656:8656] [20230205:061113:1675606273:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil)) [EC_DRHOOK:ruly-Inspiron-5558:1:1:8656:8656] [20230205:061113:1675606273:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:ruly-Inspiron-5558:1:1:8656:8656] [20230205:061113:1675606273:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:ruly-Inspiron-5558:1:1:8656:8656] [20230205:061113:1675606273:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:ruly-Inspiron-5558:1:1:8656:8656] [20230205:061113:1675606273:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil)) [EC_DRHOOK:ruly-Inspiron-5558:1:1:8656:8656] [20230205:061113:1675606273:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil)) [EC_DRHOOK:ruly-Inspiron-5558:1:1:8656:8656] [20230205:061113:1675606273:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil)) [EC_DRHOOK:ruly-Inspiron-5558:1:1:8656:8656] [20230205:061113:1675606273:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef> Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 06:13:59 STEP 768 H= 768:00 +CPU= 63.159 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 06:15:24 STEP 769 H= 769:00 +CPU= 35.104 Suspend request received from the BOINC client, suspending the child process Resuming the child process 06:16:09 STEP 770 H= 770:00 +CPU= 30.857 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 06:18:30 STEP 771 H= 771:00 +CPU= 50.590 Suspend request received from the BOINC client, suspending the child process Resuming the child process 06:19:30 STEP 772 H= 772:00 +CPU= 33.413 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Quit request received from BOINC client, ending the child process (argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu (argv1) start_date: 1999050100 (argv2) exptid: hq0f (argv3) unique_member_id: 0333 (argv4) batchid: 968 (argv5) wuid: 12184977 (argv6) fclen: 123 (argv7) app_name: oifs_43r3_ps (argv8) nthreads: 1 Working directory is: /var/lib/boinc-client/slots/1 Project directory is: /var/lib/boinc-client/projects/climateprediction.net/ app name: oifs_43r3_ps version: 1.05 Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184977 ..mkdir for temp folder for results failed Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/1/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip Unzipping the app zip file: /var/lib/boinc-client/slots/1/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip Copying the namelist files from: ../../projects/climateprediction.net/jf_0a2293733ba4099fbe133f2b8f3de008 to: /var/lib/boinc-client/slots/1/oifs_43r3_ps_0333_1999050100_123_968_12184977.zip Unzipping the namelist zip file: /var/lib/boinc-client/slots/1/oifs_43r3_ps_0333_1999050100_123_968_12184977.zip ic_ancil_file: ic_ancil_12184977 ifsdata_file: ifsdata_12184977 climate_data_file: clim_data_12184977 horiz_resolution: 159 vert_resolution: 60 grid_type: l_2 upload_interval: 24 utstep: 3600 nfrres: restart dump frequency (steps) 24 Copying IC ancils from: ../../projects/climateprediction.net/jf_300ad98e3e727469f31e1eaa7b8b5640 to: /var/lib/boinc-client/slots/1/ic_ancil_12184977.zip Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/1/ic_ancil_12184977.zip ..mkdir for ifsdata folder failed Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_422ffaf33a1d0a51301a373adc594014 to: /var/lib/boinc-client/slots/1/ifsdata/ifsdata_12184977.zip Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/1/ifsdata/ifsdata_12184977.zip ..mkdir for the climate data folder failed Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/1/159l_2/clim_data_12184977.zip Unzipping the climate data zip file: /var/lib/boinc-client/slots/1/159l_2/clim_data_12184977.zip Checking for progress XML file: /var/lib/boinc-client/slots/1/progress_file_12184977.xml Opened progress file ok : /var/lib/boinc-client/slots/1/progress_file_12184977.xml -- Model is restarting -- Adjusting last_iter, 773, to previous model restart step. Creating progress file: /var/lib/boinc-client/slots/1/progress_file_12184977.xml last_cpu_time: 31223 upload_file_number: 32 last_iter: 767 last_upload: 2764800 model_completed: 0 total_length_of_simulation: 10627200 result_base_name: oifs_43r3_ps_0333_1999050100_123_968_12184977_0_r2137061363 The child process has been launched with process id: 10678 Executing the command: /var/lib/boinc-client/slots/1/oifs_43r3_model.exe [EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1 [EC_DRHOOK:ruly-Inspiron-5558:1:1:10678:10678] [20230205:062138:1675606898:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0 [EC_DRHOOK:ruly-Inspiron-5558:1:1:10678:10678] [20230205:062138:1675606898:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1 [EC_DRHOOK:ruly-Inspiron-5558:1:1:10678:10678] [20230205:062138:1675606898:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0) [EC_DRHOOK:ruly-Inspiron-5558:1:1:10678:10678] [20230205:062138:1675606898:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1 [EC_DRHOOK:ruly-Inspiron-5558:1:1:10678:10678] [20230205:062138:1675606898:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000 [EC_DRHOOK:ruly-Inspiron-5558:1:1:10678:10678] [20230205:062138:1675606898:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647) [EC_DRHOOK:ruly-Inspiron-5558:1:1:10678:10678] [20230205:062138:1675606898:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16 [EC_DRHOOK:ruly-Inspiron-5558:1:1:10678:10678] [20230205:062138:1675606898:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0 [EC_DRHOOK:ruly-Inspiron-5558:1:1:10678:10678] [20230205:062138:1675606898:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500 [EC_DRHOOK:ruly-Inspiron-5558:1:1:10678:10678] [20230205:062138:1675606898:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1 [EC_DRHOOK:ruly-Inspiron-5558:1:1:10678:10678] [20230205:062138:1675606898:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1 [EC_DRHOOK:ruly-Inspiron-5558:1:1:10678:10678] [20230205:062138:1675606898:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1 [EC_DRHOOK:ruly-Inspiron-5558:1:1:10678:10678] [20230205:062138:1675606898:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1 [EC_DRHOOK:ruly-Inspiron-5558:1:1:10678:10678] [20230205:062138:1675606898:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef> [EC_DRHOOK:ruly-Inspiron-5558:1:1:10678:10678] [20230205:062138:1675606898:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef> [EC_DRHOOK:ruly-Inspiron-5558:1:1:10678:10678] [20230205:062138:1675606898:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:ruly-Inspiron-5558:1:1:10678:10678] [20230205:062138:1675606898:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil)) [EC_DRHOOK:ruly-Inspiron-5558:1:1:10678:10678] [20230205:062138:1675606898:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:ruly-Inspiron-5558:1:1:10678:10678] [20230205:062138:1675606898:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil)) [EC_DRHOOK:ruly-Inspiron-5558:1:1:10678:10678] [20230205:062138:1675606898:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1 [EC_DRHOOK:ruly-Inspiron-5558:1:1:10678:10678] [20230205:062138:1675606898:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:ruly-Inspiron-5558:1:1:10678:10678] [20230205:062138:1675606898:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:ruly-Inspiron-5558:1:1:10678:10678] [20230205:062138:1675606898:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil)) [EC_DRHOOK:ruly-Inspiron-5558:1:1:10678:10678] [20230205:062138:1675606898:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:ruly-Inspiron-5558:1:1:10678:10678] [20230205:062138:1675606898:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:ruly-Inspiron-5558:1:1:10678:10678] [20230205:062138:1675606898:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:ruly-Inspiron-5558:1:1:10678:10678] [20230205:062138:1675606898:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil)) [EC_DRHOOK:ruly-Inspiron-5558:1:1:10678:10678] [20230205:062138:1675606898:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil)) [EC_DRHOOK:ruly-Inspiron-5558:1:1:10678:10678] [20230205:062138:1675606898:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil)) [EC_DRHOOK:ruly-Inspiron-5558:1:1:10678:10678] [20230205:062138:1675606898:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef> Suspend request received from the BOINC client, suspending the child process Resuming the child process 06:23:23 STEP 768 H= 768:00 +CPU= 59.613 06:23:52 STEP 769 H= 769:00 +CPU= 27.281 06:24:22 STEP 770 H= 770:00 +CPU= 28.609 06:25:11 STEP 771 H= 771:00 +CPU= 44.506 Suspend request received from the BOINC client, suspending the child process Resuming the child process 06:25:55 STEP 772 H= 772:00 +CPU= 30.928 06:26:23 STEP 773 H= 773:00 +CPU= 26.871 06:27:10 STEP 774 H= 774:00 +CPU= 44.234 06:27:41 STEP 775 H= 775:00 +CPU= 29.172 06:28:09 STEP 776 H= 776:00 +CPU= 26.987 06:28:57 STEP 777 H= 777:00 +CPU= 43.859 Suspend request received from the BOINC client, suspending the child process Resuming the child process 06:29:44 STEP 778 H= 778:00 +CPU= 32.647 06:30:10 STEP 779 H= 779:00 +CPU= 25.824 06:30:52 STEP 780 H= 780:00 +CPU= 41.557 Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000780 Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000780 Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000780 06:31:20 STEP 781 H= 781:00 +CPU= 27.814 06:31:44 STEP 782 H= 782:00 +CPU= 23.554 06:32:19 STEP 783 H= 783:00 +CPU= 34.704 06:32:45 STEP 784 H= 784:00 +CPU= 25.523 06:33:07 STEP 785 H= 785:00 +CPU= 22.388 06:33:47 STEP 786 H= 786:00 +CPU= 39.633 06:34:10 STEP 787 H= 787:00 +CPU= 23.077 06:34:33 STEP 788 H= 788:00 +CPU= 22.878 06:35:13 STEP 789 H= 789:00 +CPU= 39.150 06:35:36 STEP 790 H= 790:00 +CPU= 23.015 06:35:59 STEP 791 H= 791:00 +CPU= 23.144 06:36:46 STEP 792 H= 792:00 +CPU= 46.101 Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000792 Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000792 Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000792 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184977/ICMGGhq0f+000768 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184977/ICMSHhq0f+000768 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184977/ICMUAhq0f+000768 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184977/ICMGGhq0f+000780 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184977/ICMSHhq0f+000780 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184977/ICMUAhq0f+000780 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0333_1999050100_123_968_12184977_0_r2137061363_32.zip Uploading the intermediate file: upload_file_32.zip 06:37:08 STEP 793 H= 793:00 +CPU= 22.524 06:37:31 STEP 794 H= 794:00 +CPU= 23.206 06:38:11 STEP 795 H= 795:00 +CPU= 39.303 06:38:34 STEP 796 H= 796:00 +CPU= 23.090 06:38:58 STEP 797 H= 797:00 +CPU= 24.128 06:39:38 STEP 798 H= 798:00 +CPU= 38.855 06:40:00 STEP 799 H= 799:00 +CPU= 22.437 06:40:27 STEP 800 H= 800:00 +CPU= 26.398 06:41:04 STEP 801 H= 801:00 +CPU= 36.313 06:41:27 STEP 802 H= 802:00 +CPU= 23.044 06:41:54 STEP 803 H= 803:00 +CPU= 27.133 06:42:33 STEP 804 H= 804:00 +CPU= 38.479 Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000804 Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000804 Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000804 06:42:56 STEP 805 H= 805:00 +CPU= 23.452 06:43:24 STEP 806 H= 806:00 +CPU= 27.605 06:43:59 STEP 807 H= 807:00 +CPU= 34.982 06:44:25 STEP 808 H= 808:00 +CPU= 25.197 06:44:50 STEP 809 H= 809:00 +CPU= 25.369 06:45:26 STEP 810 H= 810:00 +CPU= 35.586 06:45:51 STEP 811 H= 811:00 +CPU= 24.894 06:46:14 STEP 812 H= 812:00 +CPU= 23.142 06:46:50 STEP 813 H= 813:00 +CPU= 35.621 06:47:18 STEP 814 H= 814:00 +CPU= 27.619 06:47:41 STEP 815 H= 815:00 +CPU= 23.147 06:48:29 STEP 816 H= 816:00 +CPU= 46.830 Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000816 Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000816 Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000816 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184977/ICMGGhq0f+000792 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184977/ICMSHhq0f+000792 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184977/ICMUAhq0f+000792 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184977/ICMGGhq0f+000804 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184977/ICMSHhq0f+000804 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184977/ICMUAhq0f+000804 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0333_1999050100_123_968_12184977_0_r2137061363_33.zip Uploading the intermediate file: upload_file_33.zip 06:48:54 STEP 817 H= 817:00 +CPU= 25.483 06:49:17 STEP 818 H= 818:00 +CPU= 22.916 06:49:57 STEP 819 H= 819:00 +CPU= 38.971 06:50:20 STEP 820 H= 820:00 +CPU= 23.535 06:50:43 STEP 821 H= 821:00 +CPU= 22.693 06:51:22 STEP 822 H= 822:00 +CPU= 38.941 06:51:45 STEP 823 H= 823:00 +CPU= 22.817 06:52:08 STEP 824 H= 824:00 +CPU= 23.148 06:52:46 STEP 825 H= 825:00 +CPU= 37.152 06:53:09 STEP 826 H= 826:00 +CPU= 23.272 06:53:35 STEP 827 H= 827:00 +CPU= 25.920 06:54:16 STEP 828 H= 828:00 +CPU= 41.355 Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000828 Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000828 Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000828 06:54:39 STEP 829 H= 829:00 +CPU= 22.510 06:55:07 STEP 830 H= 830:00 +CPU= 27.795 06:55:42 STEP 831 H= 831:00 +CPU= 35.171 06:56:07 STEP 832 H= 832:00 +CPU= 24.778 06:56:33 STEP 833 H= 833:00 +CPU= 25.763 06:57:09 STEP 834 H= 834:00 +CPU= 35.861 06:57:35 STEP 835 H= 835:00 +CPU= 25.260 06:58:01 STEP 836 H= 836:00 +CPU= 25.781 06:58:36 STEP 837 H= 837:00 +CPU= 34.954 06:59:04 STEP 838 H= 838:00 +CPU= 27.856 06:59:27 STEP 839 H= 839:00 +CPU= 23.241 07:00:10 STEP 840 H= 840:00 +CPU= 42.663 Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000840 Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000840 Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000840 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184977/ICMGGhq0f+000816 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184977/ICMSHhq0f+000816 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184977/ICMUAhq0f+000816 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184977/ICMGGhq0f+000828 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184977/ICMSHhq0f+000828 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184977/ICMUAhq0f+000828 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0333_1999050100_123_968_12184977_0_r2137061363_34.zip Uploading the intermediate file: upload_file_34.zip 07:00:37 STEP 841 H= 841:00 +CPU= 26.964 07:01:00 STEP 842 H= 842:00 +CPU= 22.929 07:01:39 STEP 843 H= 843:00 +CPU= 38.119 07:02:04 STEP 844 H= 844:00 +CPU= 24.801 07:02:26 STEP 845 H= 845:00 +CPU= 22.222 07:03:06 STEP 846 H= 846:00 +CPU= 39.954 07:03:29 STEP 847 H= 847:00 +CPU= 22.612 07:03:52 STEP 848 H= 848:00 +CPU= 22.863 07:04:31 STEP 849 H= 849:00 +CPU= 39.186 07:04:54 STEP 850 H= 850:00 +CPU= 22.898 07:05:16 STEP 851 H= 851:00 +CPU= 22.183 07:06:00 STEP 852 H= 852:00 +CPU= 43.529 Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000852 Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000852 Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000852 07:06:23 STEP 853 H= 853:00 +CPU= 22.557 07:06:46 STEP 854 H= 854:00 +CPU= 22.933 07:07:25 STEP 855 H= 855:00 +CPU= 38.920 Suspend request received from the BOINC client, suspending the child process Resuming the child process 07:08:35 STEP 856 H= 856:00 +CPU= 33.878 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 07:09:43 STEP 857 H= 857:00 +CPU= 32.861 Suspend request received from the BOINC client, suspending the child process Resuming the child process 07:10:43 STEP 858 H= 858:00 +CPU= 45.348 Suspend request received from the BOINC client, suspending the child process Resuming the child process 07:11:25 STEP 859 H= 859:00 +CPU= 29.485 07:11:52 STEP 860 H= 860:00 +CPU= 25.735 07:12:34 STEP 861 H= 861:00 +CPU= 41.169 07:12:59 STEP 862 H= 862:00 +CPU= 24.893 07:13:23 STEP 863 H= 863:00 +CPU= 23.162 07:14:21 STEP 864 H= 864:00 +CPU= 53.876 Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000864 Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000864 Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000864 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184977/ICMGGhq0f+000840 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184977/ICMSHhq0f+000840 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184977/ICMUAhq0f+000840 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184977/ICMGGhq0f+000852 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184977/ICMSHhq0f+000852 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184977/ICMUAhq0f+000852 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0333_1999050100_123_968_12184977_0_r2137061363_35.zip Uploading the intermediate file: upload_file_35.zip 07:14:53 STEP 865 H= 865:00 +CPU= 29.339 07:15:26 STEP 866 H= 866:00 +CPU= 30.986 07:16:10 STEP 867 H= 867:00 +CPU= 41.022 07:16:41 STEP 868 H= 868:00 +CPU= 29.200 07:17:05 STEP 869 H= 869:00 +CPU= 24.390 07:17:53 STEP 870 H= 870:00 +CPU= 44.350 07:18:21 STEP 871 H= 871:00 +CPU= 26.730 07:18:52 STEP 872 H= 872:00 +CPU= 28.607 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 07:20:03 STEP 873 H= 873:00 +CPU= 44.725 07:20:32 STEP 874 H= 874:00 +CPU= 27.575 07:21:03 STEP 875 H= 875:00 +CPU= 28.837 07:21:51 STEP 876 H= 876:00 +CPU= 45.090 Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000876 Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000876 Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000876 Quit request received from BOINC client, ending the child process (argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu (argv1) start_date: 1999050100 (argv2) exptid: hq0f (argv3) unique_member_id: 0333 (argv4) batchid: 968 (argv5) wuid: 12184977 (argv6) fclen: 123 (argv7) app_name: oifs_43r3_ps (argv8) nthreads: 1 Working directory is: /var/lib/boinc-client/slots/1 Project directory is: /var/lib/boinc-client/projects/climateprediction.net/ app name: oifs_43r3_ps version: 1.05 Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184977 ..mkdir for temp folder for results failed Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/1/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip Unzipping the app zip file: /var/lib/boinc-client/slots/1/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip Copying the namelist files from: ../../projects/climateprediction.net/jf_0a2293733ba4099fbe133f2b8f3de008 to: /var/lib/boinc-client/slots/1/oifs_43r3_ps_0333_1999050100_123_968_12184977.zip Unzipping the namelist zip file: /var/lib/boinc-client/slots/1/oifs_43r3_ps_0333_1999050100_123_968_12184977.zip ic_ancil_file: ic_ancil_12184977 ifsdata_file: ifsdata_12184977 climate_data_file: clim_data_12184977 horiz_resolution: 159 vert_resolution: 60 grid_type: l_2 upload_interval: 24 utstep: 3600 nfrres: restart dump frequency (steps) 24 Copying IC ancils from: ../../projects/climateprediction.net/jf_300ad98e3e727469f31e1eaa7b8b5640 to: /var/lib/boinc-client/slots/1/ic_ancil_12184977.zip Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/1/ic_ancil_12184977.zip ..mkdir for ifsdata folder failed Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_422ffaf33a1d0a51301a373adc594014 to: /var/lib/boinc-client/slots/1/ifsdata/ifsdata_12184977.zip Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/1/ifsdata/ifsdata_12184977.zip ..mkdir for the climate data folder failed Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/1/159l_2/clim_data_12184977.zip Unzipping the climate data zip file: /var/lib/boinc-client/slots/1/159l_2/clim_data_12184977.zip Checking for progress XML file: /var/lib/boinc-client/slots/1/progress_file_12184977.xml Opened progress file ok : /var/lib/boinc-client/slots/1/progress_file_12184977.xml -- Model is restarting -- Adjusting last_iter, 877, to previous model restart step. Creating progress file: /var/lib/boinc-client/slots/1/progress_file_12184977.xml last_cpu_time: 34590 upload_file_number: 36 last_iter: 863 last_upload: 3110400 model_completed: 0 total_length_of_simulation: 10627200 result_base_name: oifs_43r3_ps_0333_1999050100_123_968_12184977_0_r2137061363 The child process has been launched with process id: 12358 Executing the command: /var/lib/boinc-client/slots/1/oifs_43r3_model.exe [EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1 [EC_DRHOOK:ruly-Inspiron-5558:1:1:12358:12358] [20230205:072315:1675610595:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0 [EC_DRHOOK:ruly-Inspiron-5558:1:1:12358:12358] [20230205:072315:1675610595:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1 [EC_DRHOOK:ruly-Inspiron-5558:1:1:12358:12358] [20230205:072315:1675610595:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0) [EC_DRHOOK:ruly-Inspiron-5558:1:1:12358:12358] [20230205:072315:1675610595:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1 [EC_DRHOOK:ruly-Inspiron-5558:1:1:12358:12358] [20230205:072315:1675610595:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000 [EC_DRHOOK:ruly-Inspiron-5558:1:1:12358:12358] [20230205:072315:1675610595:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647) [EC_DRHOOK:ruly-Inspiron-5558:1:1:12358:12358] [20230205:072315:1675610595:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16 [EC_DRHOOK:ruly-Inspiron-5558:1:1:12358:12358] [20230205:072315:1675610595:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0 [EC_DRHOOK:ruly-Inspiron-5558:1:1:12358:12358] [20230205:072315:1675610595:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500 [EC_DRHOOK:ruly-Inspiron-5558:1:1:12358:12358] [20230205:072315:1675610595:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1 [EC_DRHOOK:ruly-Inspiron-5558:1:1:12358:12358] [20230205:072315:1675610595:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1 [EC_DRHOOK:ruly-Inspiron-5558:1:1:12358:12358] [20230205:072315:1675610595:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1 [EC_DRHOOK:ruly-Inspiron-5558:1:1:12358:12358] [20230205:072315:1675610595:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1 [EC_DRHOOK:ruly-Inspiron-5558:1:1:12358:12358] [20230205:072315:1675610595:0.001] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef> [EC_DRHOOK:ruly-Inspiron-5558:1:1:12358:12358] [20230205:072315:1675610595:0.001] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef> [EC_DRHOOK:ruly-Inspiron-5558:1:1:12358:12358] [20230205:072315:1675610595:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:ruly-Inspiron-5558:1:1:12358:12358] [20230205:072315:1675610595:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil)) [EC_DRHOOK:ruly-Inspiron-5558:1:1:12358:12358] [20230205:072315:1675610595:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:ruly-Inspiron-5558:1:1:12358:12358] [20230205:072315:1675610595:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil)) [EC_DRHOOK:ruly-Inspiron-5558:1:1:12358:12358] [20230205:072315:1675610595:0.001] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1 [EC_DRHOOK:ruly-Inspiron-5558:1:1:12358:12358] [20230205:072315:1675610595:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:ruly-Inspiron-5558:1:1:12358:12358] [20230205:072315:1675610595:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:ruly-Inspiron-5558:1:1:12358:12358] [20230205:072315:1675610595:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil)) [EC_DRHOOK:ruly-Inspiron-5558:1:1:12358:12358] [20230205:072315:1675610595:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:ruly-Inspiron-5558:1:1:12358:12358] [20230205:072315:1675610595:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:ruly-Inspiron-5558:1:1:12358:12358] [20230205:072315:1675610595:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:ruly-Inspiron-5558:1:1:12358:12358] [20230205:072315:1675610595:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil)) [EC_DRHOOK:ruly-Inspiron-5558:1:1:12358:12358] [20230205:072315:1675610595:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil)) [EC_DRHOOK:ruly-Inspiron-5558:1:1:12358:12358] [20230205:072315:1675610595:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil)) [EC_DRHOOK:ruly-Inspiron-5558:1:1:12358:12358] [20230205:072315:1675610595:0.001] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef> Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 07:24:34 STEP 864 H= 864:00 +CPU= 48.891 07:25:03 STEP 865 H= 865:00 +CPU= 27.958 07:25:31 STEP 866 H= 866:00 +CPU= 26.895 Suspend request received from the BOINC client, suspending the child process Resuming the child process 07:26:50 STEP 867 H= 867:00 +CPU= 43.953 07:27:21 STEP 868 H= 868:00 +CPU= 28.613 07:27:50 STEP 869 H= 869:00 +CPU= 27.723 07:28:41 STEP 870 H= 870:00 +CPU= 47.506 07:29:11 STEP 871 H= 871:00 +CPU= 27.651 07:29:39 STEP 872 H= 872:00 +CPU= 28.085 07:30:15 STEP 873 H= 873:00 +CPU= 35.604 07:30:41 STEP 874 H= 874:00 +CPU= 25.239 07:31:07 STEP 875 H= 875:00 +CPU= 25.445 07:31:47 STEP 876 H= 876:00 +CPU= 39.726 Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000876 Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000876 Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000876 07:32:15 STEP 877 H= 877:00 +CPU= 28.145 07:32:39 STEP 878 H= 878:00 +CPU= 23.237 07:33:15 STEP 879 H= 879:00 +CPU= 35.684 07:33:40 STEP 880 H= 880:00 +CPU= 25.277 07:34:03 STEP 881 H= 881:00 +CPU= 22.393 Suspend request received from the BOINC client, suspending the child process Resuming the child process 07:35:04 STEP 882 H= 882:00 +CPU= 45.960 Suspend request received from the BOINC client, suspending the child process Resuming the child process 07:35:48 STEP 883 H= 883:00 +CPU= 30.916 Suspend request received from the BOINC client, suspending the child process Resuming the child process 07:36:38 STEP 884 H= 884:00 +CPU= 33.639 07:37:30 STEP 885 H= 885:00 +CPU= 47.265 Suspend request received from the BOINC client, suspending the child process Resuming the child process 07:38:18 STEP 886 H= 886:00 +CPU= 33.663 07:38:50 STEP 887 H= 887:00 +CPU= 29.939 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 07:41:18 STEP 888 H= 888:00 +CPU= 57.914 Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000888 Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000888 Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000888 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184977/ICMGGhq0f+000864 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184977/ICMSHhq0f+000864 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184977/ICMUAhq0f+000864 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184977/ICMGGhq0f+000876 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184977/ICMSHhq0f+000876 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12184977/ICMUAhq0f+000876 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0333_1999050100_123_968_12184977_0_r2137061363_36.zip Uploading the intermediate file: upload_file_36.zip Abort request received from BOINC client, ending the child process </stderr_txt> ]]> |
Latest Trickles Received | ||||||
---|---|---|---|---|---|---|
Time Sent (UTC) | Host ID | Result ID | Result Name | Timestep | CPU Time (sec) | Average (sec/TS) |
09 May 2023 16:47:10 | 1509518 | 22270260 | oifs_43r3_ps_0333_1999050100_123_968_12184977_0 | 2,505,600 | 26,896 | 0.0107 |
09 May 2023 16:33:21 | 1509518 | 22270260 | oifs_43r3_ps_0333_1999050100_123_968_12184977_0 | 1,382,400 | 11,839 | 0.0086 |
©2024 cpdn.org