Name | oifs_43r3_ps_0253_2002050100_123_971_12187897_0 |
Workunit | 12187897 |
Created | 23 Dec 2022, 2:15:17 UTC |
Sent | 18 Jan 2023, 3:22:05 UTC |
Report deadline | 19 Mar 2023, 3:22:05 UTC |
Received | 29 Jan 2023, 12:50:47 UTC |
Server state | Over |
Outcome | Success |
Client state | Done |
Exit status | 0 (0x00000000) |
Computer ID | 1536107 |
Run time | 2 days 3 hours 48 min 18 sec |
CPU time | 2 days 0 hours 39 min 4 sec |
Validate state | Valid |
Credit | 14,873.04 |
Device peak FLOPS | 1.00 GFLOPS |
Application version | OpenIFS 43r3 Perturbed Surface v1.05 x86_64-pc-linux-gnu |
Peak working set size | 4,410.14 MB |
Peak swap size | 4,978.16 MB |
Peak disk usage | 6,062.74 MB |
Stderr | <core_client_version>7.20.2</core_client_version> <![CDATA[ <stderr_txt> 2050100_123_971_12187897_0_r2102816723_111.zip Uploading the intermediate file: upload_file_111.zip 21:50:21 STEP 2689 H=2689:00 +CPU= 42.392 21:51:05 STEP 2690 H=2690:00 +CPU= 43.268 21:52:13 STEP 2691 H=2691:00 +CPU= 67.283 21:52:57 STEP 2692 H=2692:00 +CPU= 43.661 21:53:40 STEP 2693 H=2693:00 +CPU= 42.726 21:54:49 STEP 2694 H=2694:00 +CPU= 69.075 21:55:32 STEP 2695 H=2695:00 +CPU= 42.717 21:56:16 STEP 2696 H=2696:00 +CPU= 43.524 21:57:24 STEP 2697 H=2697:00 +CPU= 67.663 21:58:08 STEP 2698 H=2698:00 +CPU= 43.654 21:58:50 STEP 2699 H=2699:00 +CPU= 42.346 22:00:06 STEP 2700 H=2700:00 +CPU= 74.837 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002700 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002700 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002700 22:00:50 STEP 2701 H=2701:00 +CPU= 43.291 22:01:35 STEP 2702 H=2702:00 +CPU= 44.116 22:02:45 STEP 2703 H=2703:00 +CPU= 69.486 22:03:32 STEP 2704 H=2704:00 +CPU= 44.987 22:04:17 STEP 2705 H=2705:00 +CPU= 43.961 22:05:30 STEP 2706 H=2706:00 +CPU= 71.469 22:06:17 STEP 2707 H=2707:00 +CPU= 44.707 22:07:05 STEP 2708 H=2708:00 +CPU= 45.938 22:08:19 STEP 2709 H=2709:00 +CPU= 71.861 22:09:08 STEP 2710 H=2710:00 +CPU= 46.480 22:09:56 STEP 2711 H=2711:00 +CPU= 46.293 22:11:24 STEP 2712 H=2712:00 +CPU= 83.604 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002712 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002712 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002712 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12187897/ICMGGhq0f+002688 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12187897/ICMSHhq0f+002688 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12187897/ICMUAhq0f+002688 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12187897/ICMGGhq0f+002700 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12187897/ICMSHhq0f+002700 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12187897/ICMUAhq0f+002700 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0253_2002050100_123_971_12187897_0_r2102816723_112.zip Uploading the intermediate file: upload_file_112.zip 22:12:11 STEP 2713 H=2713:00 +CPU= 45.284 22:12:57 STEP 2714 H=2714:00 +CPU= 45.474 22:14:10 STEP 2715 H=2715:00 +CPU= 70.660 22:14:59 STEP 2716 H=2716:00 +CPU= 46.356 22:15:46 STEP 2717 H=2717:00 +CPU= 45.169 22:17:04 STEP 2718 H=2718:00 +CPU= 74.466 22:17:50 STEP 2719 H=2719:00 +CPU= 44.392 22:18:39 STEP 2720 H=2720:00 +CPU= 47.269 22:19:55 STEP 2721 H=2721:00 +CPU= 72.475 22:20:45 STEP 2722 H=2722:00 +CPU= 47.401 22:21:34 STEP 2723 H=2723:00 +CPU= 46.353 22:23:01 STEP 2724 H=2724:00 +CPU= 82.001 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002724 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002724 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002724 22:23:48 STEP 2725 H=2725:00 +CPU= 45.543 22:24:37 STEP 2726 H=2726:00 +CPU= 46.882 22:25:52 STEP 2727 H=2727:00 +CPU= 72.199 22:26:42 STEP 2728 H=2728:00 +CPU= 48.000 22:27:31 STEP 2729 H=2729:00 +CPU= 46.167 22:28:51 STEP 2730 H=2730:00 +CPU= 75.586 22:29:36 STEP 2731 H=2731:00 +CPU= 43.464 22:30:21 STEP 2732 H=2732:00 +CPU= 44.833 22:31:33 STEP 2733 H=2733:00 +CPU= 70.077 22:32:21 STEP 2734 H=2734:00 +CPU= 46.423 22:33:08 STEP 2735 H=2735:00 +CPU= 45.579 22:34:40 STEP 2736 H=2736:00 +CPU= 82.359 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002736 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002736 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002736 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12187897/ICMGGhq0f+002712 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12187897/ICMSHhq0f+002712 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12187897/ICMUAhq0f+002712 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12187897/ICMGGhq0f+002724 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12187897/ICMSHhq0f+002724 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12187897/ICMUAhq0f+002724 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0253_2002050100_123_971_12187897_0_r2102816723_113.zip Uploading the intermediate file: upload_file_113.zip 22:35:26 STEP 2737 H=2737:00 +CPU= 45.082 22:36:12 STEP 2738 H=2738:00 +CPU= 45.067 22:37:25 STEP 2739 H=2739:00 +CPU= 70.858 22:38:13 STEP 2740 H=2740:00 +CPU= 46.008 22:38:59 STEP 2741 H=2741:00 +CPU= 44.880 22:40:11 STEP 2742 H=2742:00 +CPU= 70.708 22:40:53 STEP 2743 H=2743:00 +CPU= 41.877 22:41:37 STEP 2744 H=2744:00 +CPU= 43.232 22:42:44 STEP 2745 H=2745:00 +CPU= 66.964 22:43:27 STEP 2746 H=2746:00 +CPU= 42.919 22:44:10 STEP 2747 H=2747:00 +CPU= 42.511 22:45:26 STEP 2748 H=2748:00 +CPU= 74.931 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002748 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002748 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002748 22:46:09 STEP 2749 H=2749:00 +CPU= 42.598 22:46:53 STEP 2750 H=2750:00 +CPU= 43.386 22:48:00 STEP 2751 H=2751:00 +CPU= 67.446 22:48:44 STEP 2752 H=2752:00 +CPU= 43.255 22:49:26 STEP 2753 H=2753:00 +CPU= 42.010 22:50:35 STEP 2754 H=2754:00 +CPU= 68.436 22:51:17 STEP 2755 H=2755:00 +CPU= 41.926 22:52:01 STEP 2756 H=2756:00 +CPU= 43.555 22:53:10 STEP 2757 H=2757:00 +CPU= 67.889 22:53:56 STEP 2758 H=2758:00 +CPU= 45.243 Quit request received from BOINC client, ending the child process (argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu (argv1) start_date: 2002050100 (argv2) exptid: hq0f (argv3) unique_member_id: 0253 (argv4) batchid: 971 (argv5) wuid: 12187897 (argv6) fclen: 123 (argv7) app_name: oifs_43r3_ps (argv8) nthreads: 1 Working directory is: /var/lib/boinc-client/slots/0 Project directory is: /var/lib/boinc-client/projects/climateprediction.net/ app name: oifs_43r3_ps version: 1.05 Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12187897 ..mkdir for temp folder for results failed Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/0/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip Unzipping the app zip file: /var/lib/boinc-client/slots/0/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip Copying the namelist files from: ../../projects/climateprediction.net/jf_f2181658f1650845bd008db5673844d5 to: /var/lib/boinc-client/slots/0/oifs_43r3_ps_0253_2002050100_123_971_12187897.zip Unzipping the namelist zip file: /var/lib/boinc-client/slots/0/oifs_43r3_ps_0253_2002050100_123_971_12187897.zip ic_ancil_file: ic_ancil_12187897 ifsdata_file: ifsdata_12187897 climate_data_file: clim_data_12187897 horiz_resolution: 159 vert_resolution: 60 grid_type: l_2 upload_interval: 24 utstep: 3600 nfrres: restart dump frequency (steps) 24 Copying IC ancils from: ../../projects/climateprediction.net/jf_c566bbf2cb05fe105f907602b6e07fcb to: /var/lib/boinc-client/slots/0/ic_ancil_12187897.zip Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/0/ic_ancil_12187897.zip ..mkdir for ifsdata folder failed Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_ab365c71aa764bd1df85fd88e5e809dc to: /var/lib/boinc-client/slots/0/ifsdata/ifsdata_12187897.zip Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/0/ifsdata/ifsdata_12187897.zip ..mkdir for the climate data folder failed Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/0/159l_2/clim_data_12187897.zip Unzipping the climate data zip file: /var/lib/boinc-client/slots/0/159l_2/clim_data_12187897.zip Checking for progress XML file: /var/lib/boinc-client/slots/0/progress_file_12187897.xml Opened progress file ok : /var/lib/boinc-client/slots/0/progress_file_12187897.xml -- Model is restarting -- Adjusting last_iter, 2759, to previous model restart step. Creating progress file: /var/lib/boinc-client/slots/0/progress_file_12187897.xml last_cpu_time: 158885 upload_file_number: 114 last_iter: 2735 last_upload: 9849600 model_completed: 0 total_length_of_simulation: 10627200 result_base_name: oifs_43r3_ps_0253_2002050100_123_971_12187897_0_r2102816723 The child process has been launched with process id: 169993 Executing the command: /var/lib/boinc-client/slots/0/oifs_43r3_model.exe [EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1 [EC_DRHOOK:Shaynovo:1:1:169993:169993] [20230128:225705:1674971825:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0 [EC_DRHOOK:Shaynovo:1:1:169993:169993] [20230128:225705:1674971825:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1 [EC_DRHOOK:Shaynovo:1:1:169993:169993] [20230128:225705:1674971825:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0) [EC_DRHOOK:Shaynovo:1:1:169993:169993] [20230128:225705:1674971825:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1 [EC_DRHOOK:Shaynovo:1:1:169993:169993] [20230128:225705:1674971825:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000 [EC_DRHOOK:Shaynovo:1:1:169993:169993] [20230128:225705:1674971825:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647) [EC_DRHOOK:Shaynovo:1:1:169993:169993] [20230128:225705:1674971825:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16 [EC_DRHOOK:Shaynovo:1:1:169993:169993] [20230128:225705:1674971825:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0 [EC_DRHOOK:Shaynovo:1:1:169993:169993] [20230128:225705:1674971825:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500 [EC_DRHOOK:Shaynovo:1:1:169993:169993] [20230128:225705:1674971825:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1 [EC_DRHOOK:Shaynovo:1:1:169993:169993] [20230128:225705:1674971825:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1 [EC_DRHOOK:Shaynovo:1:1:169993:169993] [20230128:225705:1674971825:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1 [EC_DRHOOK:Shaynovo:1:1:169993:169993] [20230128:225705:1674971825:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1 [EC_DRHOOK:Shaynovo:1:1:169993:169993] [20230128:225705:1674971825:0.001] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef> [EC_DRHOOK:Shaynovo:1:1:169993:169993] [20230128:225705:1674971825:0.001] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef> [EC_DRHOOK:Shaynovo:1:1:169993:169993] [20230128:225705:1674971825:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:Shaynovo:1:1:169993:169993] [20230128:225705:1674971825:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil)) [EC_DRHOOK:Shaynovo:1:1:169993:169993] [20230128:225705:1674971825:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:Shaynovo:1:1:169993:169993] [20230128:225705:1674971825:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil)) [EC_DRHOOK:Shaynovo:1:1:169993:169993] [20230128:225705:1674971825:0.001] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1 [EC_DRHOOK:Shaynovo:1:1:169993:169993] [20230128:225705:1674971825:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:Shaynovo:1:1:169993:169993] [20230128:225705:1674971825:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:Shaynovo:1:1:169993:169993] [20230128:225705:1674971825:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil)) [EC_DRHOOK:Shaynovo:1:1:169993:169993] [20230128:225705:1674971825:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:Shaynovo:1:1:169993:169993] [20230128:225705:1674971825:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:Shaynovo:1:1:169993:169993] [20230128:225705:1674971825:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:Shaynovo:1:1:169993:169993] [20230128:225705:1674971825:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil)) [EC_DRHOOK:Shaynovo:1:1:169993:169993] [20230128:225705:1674971825:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil)) [EC_DRHOOK:Shaynovo:1:1:169993:169993] [20230128:225705:1674971825:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil)) [EC_DRHOOK:Shaynovo:1:1:169993:169993] [20230128:225705:1674971825:0.001] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef> 22:58:41 STEP 2736 H=2736:00 +CPU= 77.181 22:59:36 STEP 2737 H=2737:00 +CPU= 52.356 23:00:40 STEP 2738 H=2738:00 +CPU= 56.740 23:02:16 STEP 2739 H=2739:00 +CPU= 86.691 23:03:16 STEP 2740 H=2740:00 +CPU= 55.600 23:04:16 STEP 2741 H=2741:00 +CPU= 54.292 Suspend request received from the BOINC client, suspending the child process Resuming the child process 23:06:32 STEP 2742 H=2742:00 +CPU= 96.332 Suspend request received from the BOINC client, suspending the child process Resuming the child process 23:08:04 STEP 2743 H=2743:00 +CPU= 61.398 Suspend request received from the BOINC client, suspending the child process Resuming the child process 23:09:38 STEP 2744 H=2744:00 +CPU= 63.257 23:11:48 STEP 2745 H=2745:00 +CPU= 98.875 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 23:13:33 STEP 2746 H=2746:00 +CPU= 63.216 23:14:53 STEP 2747 H=2747:00 +CPU= 61.411 Suspend request received from the BOINC client, suspending the child process Resuming the child process 23:17:28 STEP 2748 H=2748:00 +CPU=109.198 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002748 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002748 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002748 23:18:48 STEP 2749 H=2749:00 +CPU= 61.532 23:20:10 STEP 2750 H=2750:00 +CPU= 63.051 Suspend request received from the BOINC client, suspending the child process Resuming the child process 23:22:28 STEP 2751 H=2751:00 +CPU= 98.161 23:23:51 STEP 2752 H=2752:00 +CPU= 63.360 Suspend request received from the BOINC client, suspending the child process Resuming the child process 23:25:21 STEP 2753 H=2753:00 +CPU= 61.397 23:27:31 STEP 2754 H=2754:00 +CPU= 99.704 23:28:50 STEP 2755 H=2755:00 +CPU= 61.230 23:30:12 STEP 2756 H=2756:00 +CPU= 62.906 Suspend request received from the BOINC client, suspending the child process Resuming the child process 23:32:30 STEP 2757 H=2757:00 +CPU= 97.712 Suspend request received from the BOINC client, suspending the child process Resuming the child process 23:34:05 STEP 2758 H=2758:00 +CPU= 63.634 Suspend request received from the BOINC client, suspending the child process Resuming the child process 23:35:41 STEP 2759 H=2759:00 +CPU= 63.602 Suspend request received from the BOINC client, suspending the child process Resuming the child process 23:38:21 STEP 2760 H=2760:00 +CPU=114.794 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002760 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002760 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002760 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12187897/ICMGGhq0f+002736 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12187897/ICMSHhq0f+002736 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12187897/ICMUAhq0f+002736 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12187897/ICMGGhq0f+002748 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12187897/ICMSHhq0f+002748 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12187897/ICMUAhq0f+002748 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0253_2002050100_123_971_12187897_0_r2102816723_114.zip Uploading the intermediate file: upload_file_114.zip 23:39:41 STEP 2761 H=2761:00 +CPU= 61.428 23:41:02 STEP 2762 H=2762:00 +CPU= 62.841 Suspend request received from the BOINC client, suspending the child process Resuming the child process 23:43:19 STEP 2763 H=2763:00 +CPU= 97.814 Suspend request received from the BOINC client, suspending the child process Resuming the child process 23:44:53 STEP 2764 H=2764:00 +CPU= 63.114 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 23:46:37 STEP 2765 H=2765:00 +CPU= 61.774 Suspend request received from the BOINC client, suspending the child process Resuming the child process 23:49:04 STEP 2766 H=2766:00 +CPU=103.814 23:50:25 STEP 2767 H=2767:00 +CPU= 61.656 23:51:48 STEP 2768 H=2768:00 +CPU= 63.462 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 23:54:20 STEP 2769 H=2769:00 +CPU= 98.720 Suspend request received from the BOINC client, suspending the child process Resuming the child process 23:55:55 STEP 2770 H=2770:00 +CPU= 63.627 Suspend request received from the BOINC client, suspending the child process Resuming the child process 23:57:27 STEP 2771 H=2771:00 +CPU= 61.578 Suspend request received from the BOINC client, suspending the child process Resuming the child process 00:00:01 STEP 2772 H=2772:00 +CPU=109.009 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002772 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002772 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002772 00:01:17 STEP 2773 H=2773:00 +CPU= 60.333 00:02:36 STEP 2774 H=2774:00 +CPU= 62.343 00:04:43 STEP 2775 H=2775:00 +CPU= 98.060 Suspend request received from the BOINC client, suspending the child process Resuming the child process 00:06:12 STEP 2776 H=2776:00 +CPU= 61.629 Suspend request received from the BOINC client, suspending the child process Resuming the child process 00:07:44 STEP 2777 H=2777:00 +CPU= 61.644 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 00:10:19 STEP 2778 H=2778:00 +CPU=100.466 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 00:12:16 STEP 2779 H=2779:00 +CPU= 61.798 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 00:14:03 STEP 2780 H=2780:00 +CPU= 63.417 Suspend request received from the BOINC client, suspending the child process Resuming the child process 00:16:19 STEP 2781 H=2781:00 +CPU= 97.302 Suspend request received from the BOINC client, suspending the child process Resuming the child process 00:17:51 STEP 2782 H=2782:00 +CPU= 62.756 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 00:19:37 STEP 2783 H=2783:00 +CPU= 63.560 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 00:22:33 STEP 2784 H=2784:00 +CPU=115.361 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002784 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002784 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002784 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12187897/ICMGGhq0f+002760 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12187897/ICMSHhq0f+002760 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12187897/ICMUAhq0f+002760 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12187897/ICMGGhq0f+002772 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12187897/ICMSHhq0f+002772 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12187897/ICMUAhq0f+002772 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0253_2002050100_123_971_12187897_0_r2102816723_115.zip Uploading the intermediate file: upload_file_115.zip 00:23:50 STEP 2785 H=2785:00 +CPU= 60.294 Suspend request received from the BOINC client, suspending the child process Resuming the child process 00:25:23 STEP 2786 H=2786:00 +CPU= 62.999 Suspend request received from the BOINC client, suspending the child process Resuming the child process 00:27:40 STEP 2787 H=2787:00 +CPU= 97.486 Suspend request received from the BOINC client, suspending the child process Resuming the child process 00:29:13 STEP 2788 H=2788:00 +CPU= 63.244 00:30:34 STEP 2789 H=2789:00 +CPU= 62.309 00:32:44 STEP 2790 H=2790:00 +CPU=100.816 Suspend request received from the BOINC client, suspending the child process Resuming the child process 00:34:17 STEP 2791 H=2791:00 +CPU= 61.924 Suspend request received from the BOINC client, suspending the child process Resuming the child process 00:35:49 STEP 2792 H=2792:00 +CPU= 62.808 Suspend request received from the BOINC client, suspending the child process Resuming the child process 00:38:08 STEP 2793 H=2793:00 +CPU= 98.141 Suspend request received from the BOINC client, suspending the child process Resuming the child process 00:39:42 STEP 2794 H=2794:00 +CPU= 63.307 Suspend request received from the BOINC client, suspending the child process Resuming the child process 00:41:13 STEP 2795 H=2795:00 +CPU= 61.565 Suspend request received from the BOINC client, suspending the child process Resuming the child process 00:43:46 STEP 2796 H=2796:00 +CPU=108.794 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002796 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002796 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002796 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 00:45:28 STEP 2797 H=2797:00 +CPU= 61.574 Suspend request received from the BOINC client, suspending the child process Resuming the child process 00:47:00 STEP 2798 H=2798:00 +CPU= 62.809 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 00:49:47 STEP 2799 H=2799:00 +CPU= 99.383 Suspend request received from the BOINC client, suspending the child process Resuming the child process 00:51:23 STEP 2800 H=2800:00 +CPU= 63.788 00:52:44 STEP 2801 H=2801:00 +CPU= 61.596 Suspend request received from the BOINC client, suspending the child process Resuming the child process 00:55:08 STEP 2802 H=2802:00 +CPU=100.610 Suspend request received from the BOINC client, suspending the child process Resuming the child process 00:56:41 STEP 2803 H=2803:00 +CPU= 61.505 Suspend request received from the BOINC client, suspending the child process Resuming the child process 00:58:14 STEP 2804 H=2804:00 +CPU= 63.322 01:00:22 STEP 2805 H=2805:00 +CPU= 98.334 Suspend request received from the BOINC client, suspending the child process Resuming the child process 01:01:55 STEP 2806 H=2806:00 +CPU= 63.068 Suspend request received from the BOINC client, suspending the child process Resuming the child process 01:03:30 STEP 2807 H=2807:00 +CPU= 63.728 Suspend request received from the BOINC client, suspending the child process Resuming the child process 01:06:11 STEP 2808 H=2808:00 +CPU=114.773 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002808 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002808 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002808 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12187897/ICMGGhq0f+002784 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12187897/ICMSHhq0f+002784 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12187897/ICMUAhq0f+002784 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12187897/ICMGGhq0f+002796 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12187897/ICMSHhq0f+002796 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12187897/ICMUAhq0f+002796 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0253_2002050100_123_971_12187897_0_r2102816723_116.zip Uploading the intermediate file: upload_file_116.zip Suspend request received from the BOINC client, suspending the child process Resuming the child process 01:07:33 STEP 2809 H=2809:00 +CPU= 60.217 01:08:56 STEP 2810 H=2810:00 +CPU= 63.048 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 01:11:27 STEP 2811 H=2811:00 +CPU= 97.944 Suspend request received from the BOINC client, suspending the child process Resuming the child process 01:13:00 STEP 2812 H=2812:00 +CPU= 62.790 01:14:21 STEP 2813 H=2813:00 +CPU= 61.686 Suspend request received from the BOINC client, suspending the child process Resuming the child process 01:16:45 STEP 2814 H=2814:00 +CPU=100.083 Quit request received from BOINC client, ending the child process (argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu (argv1) start_date: 2002050100 (argv2) exptid: hq0f (argv3) unique_member_id: 0253 (argv4) batchid: 971 (argv5) wuid: 12187897 (argv6) fclen: 123 (argv7) app_name: oifs_43r3_ps (argv8) nthreads: 1 Working directory is: /var/lib/boinc-client/slots/0 Project directory is: /var/lib/boinc-client/projects/climateprediction.net/ app name: oifs_43r3_ps version: 1.05 Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12187897 ..mkdir for temp folder for results failed Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/0/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip Unzipping the app zip file: /var/lib/boinc-client/slots/0/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip Copying the namelist files from: ../../projects/climateprediction.net/jf_f2181658f1650845bd008db5673844d5 to: /var/lib/boinc-client/slots/0/oifs_43r3_ps_0253_2002050100_123_971_12187897.zip Unzipping the namelist zip file: /var/lib/boinc-client/slots/0/oifs_43r3_ps_0253_2002050100_123_971_12187897.zip ic_ancil_file: ic_ancil_12187897 ifsdata_file: ifsdata_12187897 climate_data_file: clim_data_12187897 horiz_resolution: 159 vert_resolution: 60 grid_type: l_2 upload_interval: 24 utstep: 3600 nfrres: restart dump frequency (steps) 24 Copying IC ancils from: ../../projects/climateprediction.net/jf_c566bbf2cb05fe105f907602b6e07fcb to: /var/lib/boinc-client/slots/0/ic_ancil_12187897.zip Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/0/ic_ancil_12187897.zip ..mkdir for ifsdata folder failed Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_ab365c71aa764bd1df85fd88e5e809dc to: /var/lib/boinc-client/slots/0/ifsdata/ifsdata_12187897.zip Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/0/ifsdata/ifsdata_12187897.zip ..mkdir for the climate data folder failed Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/0/159l_2/clim_data_12187897.zip Unzipping the climate data zip file: /var/lib/boinc-client/slots/0/159l_2/clim_data_12187897.zip Checking for progress XML file: /var/lib/boinc-client/slots/0/progress_file_12187897.xml Opened progress file ok : /var/lib/boinc-client/slots/0/progress_file_12187897.xml -- Model is restarting -- Adjusting last_iter, 2815, to previous model restart step. Creating progress file: /var/lib/boinc-client/slots/0/progress_file_12187897.xml last_cpu_time: 164825 upload_file_number: 117 last_iter: 2807 last_upload: 10108800 model_completed: 0 total_length_of_simulation: 10627200 result_base_name: oifs_43r3_ps_0253_2002050100_123_971_12187897_0_r2102816723 The child process has been launched with process id: 179930 Executing the command: /var/lib/boinc-client/slots/0/oifs_43r3_model.exe [EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1 [EC_DRHOOK:Shaynovo:1:1:179930:179930] [20230129:012512:1674980712:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0 [EC_DRHOOK:Shaynovo:1:1:179930:179930] [20230129:012512:1674980712:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1 [EC_DRHOOK:Shaynovo:1:1:179930:179930] [20230129:012512:1674980712:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0) [EC_DRHOOK:Shaynovo:1:1:179930:179930] [20230129:012512:1674980712:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1 [EC_DRHOOK:Shaynovo:1:1:179930:179930] [20230129:012512:1674980712:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000 [EC_DRHOOK:Shaynovo:1:1:179930:179930] [20230129:012512:1674980712:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647) [EC_DRHOOK:Shaynovo:1:1:179930:179930] [20230129:012512:1674980712:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16 [EC_DRHOOK:Shaynovo:1:1:179930:179930] [20230129:012512:1674980712:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0 [EC_DRHOOK:Shaynovo:1:1:179930:179930] [20230129:012512:1674980712:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500 [EC_DRHOOK:Shaynovo:1:1:179930:179930] [20230129:012512:1674980712:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1 [EC_DRHOOK:Shaynovo:1:1:179930:179930] [20230129:012512:1674980712:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1 [EC_DRHOOK:Shaynovo:1:1:179930:179930] [20230129:012512:1674980712:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1 [EC_DRHOOK:Shaynovo:1:1:179930:179930] [20230129:012512:1674980712:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1 [EC_DRHOOK:Shaynovo:1:1:179930:179930] [20230129:012512:1674980712:0.001] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef> [EC_DRHOOK:Shaynovo:1:1:179930:179930] [20230129:012512:1674980712:0.001] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef> [EC_DRHOOK:Shaynovo:1:1:179930:179930] [20230129:012512:1674980712:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:Shaynovo:1:1:179930:179930] [20230129:012512:1674980712:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil)) [EC_DRHOOK:Shaynovo:1:1:179930:179930] [20230129:012512:1674980712:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:Shaynovo:1:1:179930:179930] [20230129:012512:1674980712:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil)) [EC_DRHOOK:Shaynovo:1:1:179930:179930] [20230129:012512:1674980712:0.001] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1 [EC_DRHOOK:Shaynovo:1:1:179930:179930] [20230129:012512:1674980712:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:Shaynovo:1:1:179930:179930] [20230129:012512:1674980712:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:Shaynovo:1:1:179930:179930] [20230129:012512:1674980712:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil)) [EC_DRHOOK:Shaynovo:1:1:179930:179930] [20230129:012512:1674980712:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:Shaynovo:1:1:179930:179930] [20230129:012512:1674980712:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:Shaynovo:1:1:179930:179930] [20230129:012512:1674980712:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:Shaynovo:1:1:179930:179930] [20230129:012512:1674980712:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil)) [EC_DRHOOK:Shaynovo:1:1:179930:179930] [20230129:012512:1674980712:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil)) [EC_DRHOOK:Shaynovo:1:1:179930:179930] [20230129:012512:1674980712:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil)) [EC_DRHOOK:Shaynovo:1:1:179930:179930] [20230129:012512:1674980712:0.001] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef> Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Quit request received from BOINC client, ending the child process (argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu (argv1) start_date: 2002050100 (argv2) exptid: hq0f (argv3) unique_member_id: 0253 (argv4) batchid: 971 (argv5) wuid: 12187897 (argv6) fclen: 123 (argv7) app_name: oifs_43r3_ps (argv8) nthreads: 1 Working directory is: /var/lib/boinc-client/slots/0 Project directory is: /var/lib/boinc-client/projects/climateprediction.net/ app name: oifs_43r3_ps version: 1.05 Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12187897 ..mkdir for temp folder for results failed Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/0/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip Unzipping the app zip file: /var/lib/boinc-client/slots/0/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip Copying the namelist files from: ../../projects/climateprediction.net/jf_f2181658f1650845bd008db5673844d5 to: /var/lib/boinc-client/slots/0/oifs_43r3_ps_0253_2002050100_123_971_12187897.zip Unzipping the namelist zip file: /var/lib/boinc-client/slots/0/oifs_43r3_ps_0253_2002050100_123_971_12187897.zip ic_ancil_file: ic_ancil_12187897 ifsdata_file: ifsdata_12187897 climate_data_file: clim_data_12187897 horiz_resolution: 159 vert_resolution: 60 grid_type: l_2 upload_interval: 24 utstep: 3600 nfrres: restart dump frequency (steps) 24 Copying IC ancils from: ../../projects/climateprediction.net/jf_c566bbf2cb05fe105f907602b6e07fcb to: /var/lib/boinc-client/slots/0/ic_ancil_12187897.zip Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/0/ic_ancil_12187897.zip ..mkdir for ifsdata folder failed Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_ab365c71aa764bd1df85fd88e5e809dc to: /var/lib/boinc-client/slots/0/ifsdata/ifsdata_12187897.zip Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/0/ifsdata/ifsdata_12187897.zip ..mkdir for the climate data folder failed Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/0/159l_2/clim_data_12187897.zip Unzipping the climate data zip file: /var/lib/boinc-client/slots/0/159l_2/clim_data_12187897.zip Checking for progress XML file: /var/lib/boinc-client/slots/0/progress_file_12187897.xml Opened progress file ok : /var/lib/boinc-client/slots/0/progress_file_12187897.xml -- Model is restarting -- Adjusting last_iter, 2808, to previous model restart step. Creating progress file: /var/lib/boinc-client/slots/0/progress_file_12187897.xml last_cpu_time: 164911 upload_file_number: 117 last_iter: 2807 last_upload: 10108800 model_completed: 0 total_length_of_simulation: 10627200 result_base_name: oifs_43r3_ps_0253_2002050100_123_971_12187897_0_r2102816723 The child process has been launched with process id: 182160 Executing the command: /var/lib/boinc-client/slots/0/oifs_43r3_model.exe [EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1 [EC_DRHOOK:Shaynovo:1:1:182160:182160] [20230129:013025:1674981025:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0 [EC_DRHOOK:Shaynovo:1:1:182160:182160] [20230129:013025:1674981025:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1 [EC_DRHOOK:Shaynovo:1:1:182160:182160] [20230129:013025:1674981025:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0) [EC_DRHOOK:Shaynovo:1:1:182160:182160] [20230129:013025:1674981025:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1 [EC_DRHOOK:Shaynovo:1:1:182160:182160] [20230129:013025:1674981025:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000 [EC_DRHOOK:Shaynovo:1:1:182160:182160] [20230129:013025:1674981025:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647) [EC_DRHOOK:Shaynovo:1:1:182160:182160] [20230129:013025:1674981025:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16 [EC_DRHOOK:Shaynovo:1:1:182160:182160] [20230129:013025:1674981025:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0 [EC_DRHOOK:Shaynovo:1:1:182160:182160] [20230129:013025:1674981025:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500 [EC_DRHOOK:Shaynovo:1:1:182160:182160] [20230129:013025:1674981025:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1 [EC_DRHOOK:Shaynovo:1:1:182160:182160] [20230129:013025:1674981025:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1 [EC_DRHOOK:Shaynovo:1:1:182160:182160] [20230129:013025:1674981025:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1 [EC_DRHOOK:Shaynovo:1:1:182160:182160] [20230129:013025:1674981025:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1 [EC_DRHOOK:Shaynovo:1:1:182160:182160] [20230129:013025:1674981025:0.001] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef> [EC_DRHOOK:Shaynovo:1:1:182160:182160] [20230129:013025:1674981025:0.001] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef> [EC_DRHOOK:Shaynovo:1:1:182160:182160] [20230129:013025:1674981025:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:Shaynovo:1:1:182160:182160] [20230129:013025:1674981025:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil)) [EC_DRHOOK:Shaynovo:1:1:182160:182160] [20230129:013025:1674981025:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:Shaynovo:1:1:182160:182160] [20230129:013025:1674981025:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil)) [EC_DRHOOK:Shaynovo:1:1:182160:182160] [20230129:013025:1674981025:0.001] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1 [EC_DRHOOK:Shaynovo:1:1:182160:182160] [20230129:013025:1674981025:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:Shaynovo:1:1:182160:182160] [20230129:013025:1674981025:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:Shaynovo:1:1:182160:182160] [20230129:013025:1674981025:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil)) [EC_DRHOOK:Shaynovo:1:1:182160:182160] [20230129:013025:1674981025:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:Shaynovo:1:1:182160:182160] [20230129:013025:1674981025:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:Shaynovo:1:1:182160:182160] [20230129:013025:1674981025:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:Shaynovo:1:1:182160:182160] [20230129:013025:1674981025:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil)) [EC_DRHOOK:Shaynovo:1:1:182160:182160] [20230129:013025:1674981025:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil)) [EC_DRHOOK:Shaynovo:1:1:182160:182160] [20230129:013025:1674981025:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil)) [EC_DRHOOK:Shaynovo:1:1:182160:182160] [20230129:013025:1674981025:0.001] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef> 01:33:18 STEP 2808 H=2808:00 +CPU=112.990 01:34:38 STEP 2809 H=2809:00 +CPU= 61.211 01:35:57 STEP 2810 H=2810:00 +CPU= 62.293 01:38:01 STEP 2811 H=2811:00 +CPU= 97.589 01:39:22 STEP 2812 H=2812:00 +CPU= 62.632 01:40:41 STEP 2813 H=2813:00 +CPU= 61.034 01:42:50 STEP 2814 H=2814:00 +CPU= 99.735 01:44:09 STEP 2815 H=2815:00 +CPU= 61.045 01:45:30 STEP 2816 H=2816:00 +CPU= 62.769 01:47:36 STEP 2817 H=2817:00 +CPU= 98.020 01:48:57 STEP 2818 H=2818:00 +CPU= 62.785 01:50:16 STEP 2819 H=2819:00 +CPU= 61.026 01:52:34 STEP 2820 H=2820:00 +CPU=108.525 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002820 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002820 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002820 01:53:53 STEP 2821 H=2821:00 +CPU= 61.064 01:55:15 STEP 2822 H=2822:00 +CPU= 63.059 01:57:21 STEP 2823 H=2823:00 +CPU= 98.007 01:58:41 STEP 2824 H=2824:00 +CPU= 62.622 01:59:57 STEP 2825 H=2825:00 +CPU= 60.609 02:02:02 STEP 2826 H=2826:00 +CPU= 99.060 02:03:14 STEP 2827 H=2827:00 +CPU= 59.443 02:04:19 STEP 2828 H=2828:00 +CPU= 58.065 02:06:00 STEP 2829 H=2829:00 +CPU= 89.975 02:07:05 STEP 2830 H=2830:00 +CPU= 57.982 02:08:11 STEP 2831 H=2831:00 +CPU= 58.372 02:10:12 STEP 2832 H=2832:00 +CPU=105.573 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002832 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002832 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002832 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12187897/ICMGGhq0f+002808 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12187897/ICMSHhq0f+002808 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12187897/ICMUAhq0f+002808 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12187897/ICMGGhq0f+002820 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12187897/ICMSHhq0f+002820 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12187897/ICMUAhq0f+002820 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0253_2002050100_123_971_12187897_0_r2102816723_117.zip Uploading the intermediate file: upload_file_117.zip 02:11:17 STEP 2833 H=2833:00 +CPU= 56.999 02:12:22 STEP 2834 H=2834:00 +CPU= 57.999 02:14:03 STEP 2835 H=2835:00 +CPU= 90.602 02:15:09 STEP 2836 H=2836:00 +CPU= 58.211 02:16:12 STEP 2837 H=2837:00 +CPU= 56.556 02:17:56 STEP 2838 H=2838:00 +CPU= 92.207 02:18:59 STEP 2839 H=2839:00 +CPU= 56.545 02:20:04 STEP 2840 H=2840:00 +CPU= 58.015 02:21:46 STEP 2841 H=2841:00 +CPU= 90.435 02:22:51 STEP 2842 H=2842:00 +CPU= 58.202 02:23:55 STEP 2843 H=2843:00 +CPU= 56.560 02:25:47 STEP 2844 H=2844:00 +CPU= 99.956 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002844 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002844 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002844 02:26:51 STEP 2845 H=2845:00 +CPU= 56.759 02:27:56 STEP 2846 H=2846:00 +CPU= 57.860 02:29:36 STEP 2847 H=2847:00 +CPU= 89.833 02:30:42 STEP 2848 H=2848:00 +CPU= 58.339 02:31:45 STEP 2849 H=2849:00 +CPU= 56.023 02:33:28 STEP 2850 H=2850:00 +CPU= 91.976 02:34:32 STEP 2851 H=2851:00 +CPU= 56.719 02:35:37 STEP 2852 H=2852:00 +CPU= 57.891 02:37:18 STEP 2853 H=2853:00 +CPU= 90.152 02:38:23 STEP 2854 H=2854:00 +CPU= 57.630 02:39:28 STEP 2855 H=2855:00 +CPU= 58.311 02:41:29 STEP 2856 H=2856:00 +CPU=106.049 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002856 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002856 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002856 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12187897/ICMGGhq0f+002832 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12187897/ICMSHhq0f+002832 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12187897/ICMUAhq0f+002832 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12187897/ICMGGhq0f+002844 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12187897/ICMSHhq0f+002844 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12187897/ICMUAhq0f+002844 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0253_2002050100_123_971_12187897_0_r2102816723_118.zip Uploading the intermediate file: upload_file_118.zip 02:42:34 STEP 2857 H=2857:00 +CPU= 57.038 02:43:39 STEP 2858 H=2858:00 +CPU= 57.889 02:45:20 STEP 2859 H=2859:00 +CPU= 89.948 02:46:25 STEP 2860 H=2860:00 +CPU= 58.135 02:47:28 STEP 2861 H=2861:00 +CPU= 56.331 02:49:12 STEP 2862 H=2862:00 +CPU= 92.020 02:50:15 STEP 2863 H=2863:00 +CPU= 56.464 02:51:20 STEP 2864 H=2864:00 +CPU= 58.037 02:53:01 STEP 2865 H=2865:00 +CPU= 90.223 02:54:07 STEP 2866 H=2866:00 +CPU= 58.182 02:55:10 STEP 2867 H=2867:00 +CPU= 56.341 02:57:03 STEP 2868 H=2868:00 +CPU=100.276 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002868 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002868 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002868 02:58:06 STEP 2869 H=2869:00 +CPU= 56.287 02:59:10 STEP 2870 H=2870:00 +CPU= 57.949 03:00:52 STEP 2871 H=2871:00 +CPU= 90.508 03:01:58 STEP 2872 H=2872:00 +CPU= 58.141 03:03:01 STEP 2873 H=2873:00 +CPU= 56.576 03:04:45 STEP 2874 H=2874:00 +CPU= 92.135 03:05:48 STEP 2875 H=2875:00 +CPU= 56.065 03:06:53 STEP 2876 H=2876:00 +CPU= 58.187 03:08:34 STEP 2877 H=2877:00 +CPU= 90.073 03:09:40 STEP 2878 H=2878:00 +CPU= 58.110 03:10:45 STEP 2879 H=2879:00 +CPU= 57.956 03:12:46 STEP 2880 H=2880:00 +CPU=105.890 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002880 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002880 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002880 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12187897/ICMGGhq0f+002856 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12187897/ICMSHhq0f+002856 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12187897/ICMUAhq0f+002856 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12187897/ICMGGhq0f+002868 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12187897/ICMSHhq0f+002868 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12187897/ICMUAhq0f+002868 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0253_2002050100_123_971_12187897_0_r2102816723_119.zip Uploading the intermediate file: upload_file_119.zip 03:13:50 STEP 2881 H=2881:00 +CPU= 56.526 03:14:56 STEP 2882 H=2882:00 +CPU= 58.083 03:16:37 STEP 2883 H=2883:00 +CPU= 90.152 03:17:42 STEP 2884 H=2884:00 +CPU= 57.961 03:18:45 STEP 2885 H=2885:00 +CPU= 56.519 03:20:29 STEP 2886 H=2886:00 +CPU= 91.923 03:21:32 STEP 2887 H=2887:00 +CPU= 56.657 03:22:37 STEP 2888 H=2888:00 +CPU= 57.910 03:24:18 STEP 2889 H=2889:00 +CPU= 89.888 03:25:23 STEP 2890 H=2890:00 +CPU= 58.125 03:26:26 STEP 2891 H=2891:00 +CPU= 56.375 03:28:19 STEP 2892 H=2892:00 +CPU=100.267 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002892 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002892 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002892 03:29:23 STEP 2893 H=2893:00 +CPU= 56.560 03:30:28 STEP 2894 H=2894:00 +CPU= 58.174 03:32:10 STEP 2895 H=2895:00 +CPU= 90.337 03:33:14 STEP 2896 H=2896:00 +CPU= 57.809 03:34:18 STEP 2897 H=2897:00 +CPU= 56.226 03:36:01 STEP 2898 H=2898:00 +CPU= 91.683 03:37:04 STEP 2899 H=2899:00 +CPU= 56.529 03:38:09 STEP 2900 H=2900:00 +CPU= 57.905 03:39:49 STEP 2901 H=2901:00 +CPU= 89.803 03:40:54 STEP 2902 H=2902:00 +CPU= 57.912 03:42:00 STEP 2903 H=2903:00 +CPU= 58.412 03:44:00 STEP 2904 H=2904:00 +CPU=105.256 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002904 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002904 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002904 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12187897/ICMGGhq0f+002880 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12187897/ICMSHhq0f+002880 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12187897/ICMUAhq0f+002880 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12187897/ICMGGhq0f+002892 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12187897/ICMSHhq0f+002892 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12187897/ICMUAhq0f+002892 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0253_2002050100_123_971_12187897_0_r2102816723_120.zip Uploading the intermediate file: upload_file_120.zip 03:45:05 STEP 2905 H=2905:00 +CPU= 56.916 03:46:10 STEP 2906 H=2906:00 +CPU= 58.063 03:47:52 STEP 2907 H=2907:00 +CPU= 90.612 03:48:58 STEP 2908 H=2908:00 +CPU= 58.214 03:50:01 STEP 2909 H=2909:00 +CPU= 56.359 03:51:44 STEP 2910 H=2910:00 +CPU= 92.093 03:52:48 STEP 2911 H=2911:00 +CPU= 56.385 03:53:53 STEP 2912 H=2912:00 +CPU= 57.949 03:55:33 STEP 2913 H=2913:00 +CPU= 90.150 03:56:39 STEP 2914 H=2914:00 +CPU= 58.263 03:57:42 STEP 2915 H=2915:00 +CPU= 56.455 03:59:34 STEP 2916 H=2916:00 +CPU= 99.896 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002916 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002916 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002916 04:00:38 STEP 2917 H=2917:00 +CPU= 56.756 04:01:43 STEP 2918 H=2918:00 +CPU= 58.115 04:03:25 STEP 2919 H=2919:00 +CPU= 90.426 04:04:31 STEP 2920 H=2920:00 +CPU= 58.038 04:05:34 STEP 2921 H=2921:00 +CPU= 56.417 04:07:18 STEP 2922 H=2922:00 +CPU= 92.182 04:08:22 STEP 2923 H=2923:00 +CPU= 56.688 04:09:27 STEP 2924 H=2924:00 +CPU= 57.879 04:11:08 STEP 2925 H=2925:00 +CPU= 89.991 04:12:13 STEP 2926 H=2926:00 +CPU= 57.989 04:13:19 STEP 2927 H=2927:00 +CPU= 58.356 04:15:19 STEP 2928 H=2928:00 +CPU=105.966 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002928 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002928 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002928 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12187897/ICMGGhq0f+002904 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12187897/ICMSHhq0f+002904 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12187897/ICMUAhq0f+002904 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12187897/ICMGGhq0f+002916 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12187897/ICMSHhq0f+002916 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12187897/ICMUAhq0f+002916 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0253_2002050100_123_971_12187897_0_r2102816723_121.zip Uploading the intermediate file: upload_file_121.zip 04:16:23 STEP 2929 H=2929:00 +CPU= 56.902 04:17:28 STEP 2930 H=2930:00 +CPU= 57.859 04:19:09 STEP 2931 H=2931:00 +CPU= 90.157 04:20:15 STEP 2932 H=2932:00 +CPU= 57.987 04:21:18 STEP 2933 H=2933:00 +CPU= 56.458 04:23:02 STEP 2934 H=2934:00 +CPU= 92.191 04:24:06 STEP 2935 H=2935:00 +CPU= 56.579 04:25:10 STEP 2936 H=2936:00 +CPU= 57.974 04:26:51 STEP 2937 H=2937:00 +CPU= 90.281 04:27:56 STEP 2938 H=2938:00 +CPU= 58.129 04:29:00 STEP 2939 H=2939:00 +CPU= 56.340 04:30:53 STEP 2940 H=2940:00 +CPU=100.316 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002940 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002940 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002940 04:31:56 STEP 2941 H=2941:00 +CPU= 56.390 04:33:01 STEP 2942 H=2942:00 +CPU= 58.214 04:34:43 STEP 2943 H=2943:00 +CPU= 90.207 04:35:47 STEP 2944 H=2944:00 +CPU= 57.692 04:36:51 STEP 2945 H=2945:00 +CPU= 56.519 04:38:34 STEP 2946 H=2946:00 +CPU= 91.818 04:39:37 STEP 2947 H=2947:00 +CPU= 56.516 04:40:42 STEP 2948 H=2948:00 +CPU= 57.839 04:42:24 STEP 2949 H=2949:00 +CPU= 90.381 04:43:29 STEP 2950 H=2950:00 +CPU= 57.719 04:44:34 STEP 2951 H=2951:00 +CPU= 58.415 04:46:32 STEP 2952 H=2952:00 +CPU=102.417 The child process terminated with status: 0 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002952 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002952 Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002952 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12187897/ICMSHhq0f+002940 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12187897/ICMUAhq0f+002940 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12187897/ICMUAhq0f+002928 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12187897/ICMGGhq0f+002952 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12187897/ICMGGhq0f+002940 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12187897/ICMSHhq0f+002928 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12187897/ICMUAhq0f+002952 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12187897/ICMGGhq0f+002928 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12187897/ICMSHhq0f+002952 Zipping up the final file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0253_2002050100_123_971_12187897_0_r2102816723_122.zip Uploading the final file: upload_file_122.zip Uploading trickle at timestep: 10623600 04:49:58 (182101): called boinc_finish(0) </stderr_txt> ]]> |
Latest Trickles Received | ||||||
---|---|---|---|---|---|---|
Time Sent (UTC) | Host ID | Result ID | Result Name | Timestep | CPU Time (sec) | Average (sec/TS) |
09 May 2023 16:47:44 | 1536107 | 22273353 | oifs_43r3_ps_0253_2002050100_123_971_12187897_0 | 10,623,600 | 171,769 | 0.0162 |
09 May 2023 16:44:55 | 1536107 | 22273353 | oifs_43r3_ps_0253_2002050100_123_971_12187897_0 | 10,623,600 | 175,144 | 0.0165 |
09 May 2023 16:44:42 | 1536107 | 22273353 | oifs_43r3_ps_0253_2002050100_123_971_12187897_0 | 9,158,400 | 147,375 | 0.0161 |
09 May 2023 16:44:26 | 1536107 | 22273353 | oifs_43r3_ps_0253_2002050100_123_971_12187897_0 | 7,516,800 | 118,542 | 0.0158 |
09 May 2023 16:43:42 | 1536107 | 22273353 | oifs_43r3_ps_0253_2002050100_123_971_12187897_0 | 6,048,000 | 96,447 | 0.0159 |
09 May 2023 16:43:33 | 1536107 | 22273353 | oifs_43r3_ps_0253_2002050100_123_971_12187897_0 | 5,184,000 | 83,797 | 0.0162 |
09 May 2023 16:43:24 | 1536107 | 22273353 | oifs_43r3_ps_0253_2002050100_123_971_12187897_0 | 4,320,000 | 70,709 | 0.0164 |
09 May 2023 16:43:13 | 1536107 | 22273353 | oifs_43r3_ps_0253_2002050100_123_971_12187897_0 | 3,456,000 | 57,642 | 0.0167 |
09 May 2023 16:43:03 | 1536107 | 22273353 | oifs_43r3_ps_0253_2002050100_123_971_12187897_0 | 2,592,000 | 44,276 | 0.0171 |
09 May 2023 16:42:48 | 1536107 | 22273353 | oifs_43r3_ps_0253_2002050100_123_971_12187897_0 | 1,728,000 | 30,600 | 0.0177 |
09 May 2023 16:42:37 | 1536107 | 22273353 | oifs_43r3_ps_0253_2002050100_123_971_12187897_0 | 864,000 | 17,204 | 0.0199 |
©2024 cpdn.org