Name | oifs_43r3_ps_0041_2021050100_123_945_12163130_1 |
Workunit | 12163130 |
Created | 29 Nov 2022, 3:16:21 UTC |
Sent | 29 Nov 2022, 3:34:02 UTC |
Report deadline | 29 Dec 2022, 3:34:02 UTC |
Received | 29 Nov 2022, 6:36:17 UTC |
Server state | Over |
Outcome | Computation error |
Client state | Compute error |
Exit status | 1 (0x00000001) Unknown error code |
Computer ID | 1536567 |
Run time | 2 hours 4 min 50 sec |
CPU time | 2 hours 3 min 50 sec |
Validate state | Invalid |
Credit | 725.76 |
Device peak FLOPS | 4.01 GFLOPS |
Application version | OpenIFS 43r3 Perturbed Surface v1.01 x86_64-pc-linux-gnu |
Peak working set size | 4,668.36 MB |
Peak swap size | 4,974.32 MB |
Peak disk usage | 1,206.71 MB |
Stderr | <core_client_version>7.14.2</core_client_version> <![CDATA[ <message> process exited with code 1 (0x1, -255)</message> <stderr_txt> (argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.01_x86_64-pc-linux-gnu (argv1) start_date: 2021050100 (argv2) exptid: hpi1 (argv3) unique_member_id: 0041 (argv4) batchid: 945 (argv5) wuid: 12163130 (argv6) fclen: 123 (argv7) app_name: oifs_43r3_ps (argv8) nthreads: 1 Working directory is: /var/lib/boinc-client/slots/13 Project directory is: /var/lib/boinc-client/projects/climateprediction.net/ app name: oifs_43r3_ps version: 1.01 Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163130 Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.01_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/13/oifs_43r3_ps_app_1.01_x86_64-pc-linux-gnu.zip Unzipping the app zip file: /var/lib/boinc-client/slots/13/oifs_43r3_ps_app_1.01_x86_64-pc-linux-gnu.zip Copying the namelist files from: ../../projects/climateprediction.net/jf_dbe56cfc7b51d82ee2366820dac9db0b to: /var/lib/boinc-client/slots/13/oifs_43r3_ps_0041_2021050100_123_945_12163130.zip Unzipping the namelist zip file: /var/lib/boinc-client/slots/13/oifs_43r3_ps_0041_2021050100_123_945_12163130.zip ic_ancil_file: ic_ancil_12163130 ifsdata_file: ifsdata_12163130 climate_data_file: clim_data_12163130 horiz_resolution: 159 vert_resolution: 60 grid_type: l_2 upload_interval: 24 utstep: 3600 Copying IC ancils from: ../../projects/climateprediction.net/jf_4b040247d1980565d5c5e0e42774a678 to: /var/lib/boinc-client/slots/13/ic_ancil_12163130.zip Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/13/ic_ancil_12163130.zip Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_36d16bd4b69f2debd93e51d9ed0b70ff to: /var/lib/boinc-client/slots/13/ifsdata/ifsdata_12163130.zip Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/13/ifsdata/ifsdata_12163130.zip Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/13/159l_2/clim_data_12163130.zip Unzipping the climate data zip file: /var/lib/boinc-client/slots/13/159l_2/clim_data_12163130.zip total_length_of_simulation: 10627200 result_base_name: oifs_43r3_ps_0041_2021050100_123_945_12163130_1_r110890581 The child process has been launched with process id: 10090 Executing the command: /var/lib/boinc-client/slots/13/./master.exe [EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1 [EC_DRHOOK:server3.server3.com:1:1:10090:10090] [20221129:033456:1669692896:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0 [EC_DRHOOK:server3.server3.com:1:1:10090:10090] [20221129:033456:1669692896:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1 [EC_DRHOOK:server3.server3.com:1:1:10090:10090] [20221129:033456:1669692896:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0) [EC_DRHOOK:server3.server3.com:1:1:10090:10090] [20221129:033456:1669692896:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1 [EC_DRHOOK:server3.server3.com:1:1:10090:10090] [20221129:033456:1669692896:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000 [EC_DRHOOK:server3.server3.com:1:1:10090:10090] [20221129:033456:1669692896:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647) [EC_DRHOOK:server3.server3.com:1:1:10090:10090] [20221129:033456:1669692896:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16 [EC_DRHOOK:server3.server3.com:1:1:10090:10090] [20221129:033456:1669692896:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0 [EC_DRHOOK:server3.server3.com:1:1:10090:10090] [20221129:033456:1669692896:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500 [EC_DRHOOK:server3.server3.com:1:1:10090:10090] [20221129:033456:1669692896:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1 [EC_DRHOOK:server3.server3.com:1:1:10090:10090] [20221129:033456:1669692896:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1 [EC_DRHOOK:server3.server3.com:1:1:10090:10090] [20221129:033456:1669692896:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1 [EC_DRHOOK:server3.server3.com:1:1:10090:10090] [20221129:033456:1669692896:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1 [EC_DRHOOK:server3.server3.com:1:1:10090:10090] [20221129:033456:1669692896:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef> [EC_DRHOOK:server3.server3.com:1:1:10090:10090] [20221129:033456:1669692896:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef> [EC_DRHOOK:server3.server3.com:1:1:10090:10090] [20221129:033456:1669692896:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:server3.server3.com:1:1:10090:10090] [20221129:033456:1669692896:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil)) [EC_DRHOOK:server3.server3.com:1:1:10090:10090] [20221129:033456:1669692896:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:server3.server3.com:1:1:10090:10090] [20221129:033456:1669692896:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil)) [EC_DRHOOK:server3.server3.com:1:1:10090:10090] [20221129:033456:1669692896:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1 [EC_DRHOOK:server3.server3.com:1:1:10090:10090] [20221129:033456:1669692896:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:server3.server3.com:1:1:10090:10090] [20221129:033456:1669692896:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:server3.server3.com:1:1:10090:10090] [20221129:033456:1669692896:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil)) [EC_DRHOOK:server3.server3.com:1:1:10090:10090] [20221129:033456:1669692896:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:server3.server3.com:1:1:10090:10090] [20221129:033456:1669692896:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:server3.server3.com:1:1:10090:10090] [20221129:033456:1669692896:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:server3.server3.com:1:1:10090:10090] [20221129:033456:1669692896:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil)) [EC_DRHOOK:server3.server3.com:1:1:10090:10090] [20221129:033456:1669692896:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil)) [EC_DRHOOK:server3.server3.com:1:1:10090:10090] [20221129:033456:1669692896:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil)) [EC_DRHOOK:server3.server3.com:1:1:10090:10090] [20221129:033456:1669692896:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef> 03:35:57 STEP 0 H= 0:00 +CPU= 57.997 Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGhpi1+000000 Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHhpi1+000000 Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAhpi1+000000 03:36:38 STEP 1 H= 1:00 +CPU= 40.420 03:37:14 STEP 2 H= 2:00 +CPU= 35.856 03:38:09 STEP 3 H= 3:00 +CPU= 54.411 03:38:44 STEP 4 H= 4:00 +CPU= 35.284 03:39:19 STEP 5 H= 5:00 +CPU= 34.634 03:40:13 STEP 6 H= 6:00 +CPU= 53.885 03:40:47 STEP 7 H= 7:00 +CPU= 34.077 03:41:22 STEP 8 H= 8:00 +CPU= 35.187 03:42:16 STEP 9 H= 9:00 +CPU= 53.658 03:42:53 STEP 10 H= 10:00 +CPU= 37.051 03:43:26 STEP 11 H= 11:00 +CPU= 33.024 03:44:25 STEP 12 H= 12:00 +CPU= 58.161 Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGhpi1+000012 Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHhpi1+000012 Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAhpi1+000012 03:44:59 STEP 13 H= 13:00 +CPU= 34.426 03:45:34 STEP 14 H= 14:00 +CPU= 34.704 03:46:25 STEP 15 H= 15:00 +CPU= 51.381 03:47:00 STEP 16 H= 16:00 +CPU= 34.301 03:47:34 STEP 17 H= 17:00 +CPU= 34.163 03:48:28 STEP 18 H= 18:00 +CPU= 54.675 03:49:03 STEP 19 H= 19:00 +CPU= 34.213 03:49:38 STEP 20 H= 20:00 +CPU= 35.404 03:50:31 STEP 21 H= 21:00 +CPU= 52.537 03:51:05 STEP 22 H= 22:00 +CPU= 34.482 03:51:40 STEP 23 H= 23:00 +CPU= 35.175 03:52:41 STEP 24 H= 24:00 +CPU= 60.110 Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGhpi1+000024 Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHhpi1+000024 Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAhpi1+000024 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163130/ICMGGhpi1+000000 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163130/ICMSHhpi1+000000 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163130/ICMUAhpi1+000000 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163130/ICMGGhpi1+000012 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163130/ICMSHhpi1+000012 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163130/ICMUAhpi1+000012 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0041_2021050100_123_945_12163130_1_r110890581_0.zip Uploading the intermediate file: upload_file_0.zip Uploading trickle at timestep: 86400 03:53:14 STEP 25 H= 25:00 +CPU= 32.976 03:53:50 STEP 26 H= 26:00 +CPU= 34.584 03:54:45 STEP 27 H= 27:00 +CPU= 54.057 03:55:21 STEP 28 H= 28:00 +CPU= 35.804 03:55:55 STEP 29 H= 29:00 +CPU= 34.056 03:56:50 STEP 30 H= 30:00 +CPU= 54.648 03:57:24 STEP 31 H= 31:00 +CPU= 34.492 03:58:00 STEP 32 H= 32:00 +CPU= 35.392 03:58:55 STEP 33 H= 33:00 +CPU= 54.863 03:59:31 STEP 34 H= 34:00 +CPU= 36.044 04:00:05 STEP 35 H= 35:00 +CPU= 34.322 04:01:04 STEP 36 H= 36:00 +CPU= 58.215 Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGhpi1+000036 Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHhpi1+000036 Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAhpi1+000036 04:01:39 STEP 37 H= 37:00 +CPU= 34.939 04:02:14 STEP 38 H= 38:00 +CPU= 35.711 04:03:08 STEP 39 H= 39:00 +CPU= 53.292 04:03:44 STEP 40 H= 40:00 +CPU= 35.619 04:04:19 STEP 41 H= 41:00 +CPU= 34.041 04:05:13 STEP 42 H= 42:00 +CPU= 52.942 04:05:49 STEP 43 H= 43:00 +CPU= 33.212 04:06:23 STEP 44 H= 44:00 +CPU= 34.248 04:07:16 STEP 45 H= 45:00 +CPU= 52.414 04:07:51 STEP 46 H= 46:00 +CPU= 35.187 04:08:27 STEP 47 H= 47:00 +CPU= 35.258 04:09:30 STEP 48 H= 48:00 +CPU= 61.886 Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGhpi1+000048 Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHhpi1+000048 Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAhpi1+000048 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163130/ICMGGhpi1+000024 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163130/ICMSHhpi1+000024 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163130/ICMUAhpi1+000024 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163130/ICMGGhpi1+000036 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163130/ICMSHhpi1+000036 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163130/ICMUAhpi1+000036 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0041_2021050100_123_945_12163130_1_r110890581_1.zip Uploading the intermediate file: upload_file_1.zip Uploading trickle at timestep: 172800 04:10:04 STEP 49 H= 49:00 +CPU= 34.594 04:10:40 STEP 50 H= 50:00 +CPU= 35.701 04:11:34 STEP 51 H= 51:00 +CPU= 52.931 04:12:09 STEP 52 H= 52:00 +CPU= 35.160 04:12:43 STEP 53 H= 53:00 +CPU= 33.906 04:13:38 STEP 54 H= 54:00 +CPU= 54.238 04:14:12 STEP 55 H= 55:00 +CPU= 33.953 04:14:48 STEP 56 H= 56:00 +CPU= 35.422 04:15:41 STEP 57 H= 57:00 +CPU= 53.235 04:16:17 STEP 58 H= 58:00 +CPU= 35.787 04:16:51 STEP 59 H= 59:00 +CPU= 33.638 04:17:50 STEP 60 H= 60:00 +CPU= 59.424 Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGhpi1+000060 Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHhpi1+000060 Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAhpi1+000060 04:18:25 STEP 61 H= 61:00 +CPU= 34.409 04:19:00 STEP 62 H= 62:00 +CPU= 34.860 04:19:53 STEP 63 H= 63:00 +CPU= 53.171 04:20:28 STEP 64 H= 64:00 +CPU= 34.802 04:21:01 STEP 65 H= 65:00 +CPU= 33.498 04:21:55 STEP 66 H= 66:00 +CPU= 52.883 04:22:28 STEP 67 H= 67:00 +CPU= 33.082 04:23:02 STEP 68 H= 68:00 +CPU= 34.166 04:23:53 STEP 69 H= 69:00 +CPU= 50.954 04:24:29 STEP 70 H= 70:00 +CPU= 35.713 04:25:05 STEP 71 H= 71:00 +CPU= 35.436 04:26:07 STEP 72 H= 72:00 +CPU= 61.767 Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGhpi1+000072 Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHhpi1+000072 Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAhpi1+000072 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163130/ICMGGhpi1+000048 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163130/ICMSHhpi1+000048 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163130/ICMUAhpi1+000048 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163130/ICMGGhpi1+000060 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163130/ICMSHhpi1+000060 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163130/ICMUAhpi1+000060 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0041_2021050100_123_945_12163130_1_r110890581_2.zip Uploading the intermediate file: upload_file_2.zip Uploading trickle at timestep: 259200 04:26:41 STEP 73 H= 73:00 +CPU= 33.029 04:27:16 STEP 74 H= 74:00 +CPU= 35.095 04:28:07 STEP 75 H= 75:00 +CPU= 51.326 04:28:42 STEP 76 H= 76:00 +CPU= 34.588 04:29:16 STEP 77 H= 77:00 +CPU= 33.829 04:30:11 STEP 78 H= 78:00 +CPU= 54.730 04:30:45 STEP 79 H= 79:00 +CPU= 33.674 04:31:20 STEP 80 H= 80:00 +CPU= 35.019 04:32:13 STEP 81 H= 81:00 +CPU= 52.633 04:32:50 STEP 82 H= 82:00 +CPU= 36.702 04:33:24 STEP 83 H= 83:00 +CPU= 34.456 04:34:21 STEP 84 H= 84:00 +CPU= 56.805 Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGhpi1+000084 Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHhpi1+000084 Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAhpi1+000084 04:34:54 STEP 85 H= 85:00 +CPU= 32.735 04:35:29 STEP 86 H= 86:00 +CPU= 34.603 04:36:20 STEP 87 H= 87:00 +CPU= 51.352 04:36:55 STEP 88 H= 88:00 +CPU= 34.881 04:37:28 STEP 89 H= 89:00 +CPU= 33.429 04:38:23 STEP 90 H= 90:00 +CPU= 53.288 04:38:58 STEP 91 H= 91:00 +CPU= 35.859 04:39:36 STEP 92 H= 92:00 +CPU= 37.345 04:40:31 STEP 93 H= 93:00 +CPU= 55.151 04:41:08 STEP 94 H= 94:00 +CPU= 36.822 04:41:44 STEP 95 H= 95:00 +CPU= 36.311 04:42:45 STEP 96 H= 96:00 +CPU= 59.723 Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGhpi1+000096 Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHhpi1+000096 Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAhpi1+000096 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163130/ICMGGhpi1+000072 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163130/ICMSHhpi1+000072 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163130/ICMUAhpi1+000072 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163130/ICMGGhpi1+000084 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163130/ICMSHhpi1+000084 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163130/ICMUAhpi1+000084 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0041_2021050100_123_945_12163130_1_r110890581_3.zip Uploading the intermediate file: upload_file_3.zip Uploading trickle at timestep: 345600 04:43:19 STEP 97 H= 97:00 +CPU= 34.437 04:43:56 STEP 98 H= 98:00 +CPU= 36.364 04:44:50 STEP 99 H= 99:00 +CPU= 53.536 04:45:25 STEP 100 H= 100:00 +CPU= 34.534 04:45:58 STEP 101 H= 101:00 +CPU= 33.590 04:46:52 STEP 102 H= 102:00 +CPU= 53.550 04:47:27 STEP 103 H= 103:00 +CPU= 34.436 04:48:02 STEP 104 H= 104:00 +CPU= 34.450 04:48:54 STEP 105 H= 105:00 +CPU= 51.855 04:49:30 STEP 106 H= 106:00 +CPU= 36.447 04:50:05 STEP 107 H= 107:00 +CPU= 35.143 04:51:03 STEP 108 H= 108:00 +CPU= 57.360 Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGhpi1+000108 Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHhpi1+000108 Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAhpi1+000108 04:51:36 STEP 109 H= 109:00 +CPU= 32.979 04:52:11 STEP 110 H= 110:00 +CPU= 34.882 04:53:02 STEP 111 H= 111:00 +CPU= 51.001 04:53:37 STEP 112 H= 112:00 +CPU= 34.872 04:54:11 STEP 113 H= 113:00 +CPU= 33.842 04:55:07 STEP 114 H= 114:00 +CPU= 55.240 04:55:42 STEP 115 H= 115:00 +CPU= 35.024 04:56:19 STEP 116 H= 116:00 +CPU= 36.634 04:57:13 STEP 117 H= 117:00 +CPU= 53.614 04:57:49 STEP 118 H= 118:00 +CPU= 36.379 04:58:25 STEP 119 H= 119:00 +CPU= 36.111 04:59:28 STEP 120 H= 120:00 +CPU= 62.078 Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGhpi1+000120 Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHhpi1+000120 Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAhpi1+000120 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163130/ICMGGhpi1+000096 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163130/ICMSHhpi1+000096 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163130/ICMUAhpi1+000096 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163130/ICMGGhpi1+000108 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163130/ICMSHhpi1+000108 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163130/ICMUAhpi1+000108 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0041_2021050100_123_945_12163130_1_r110890581_4.zip Uploading the intermediate file: upload_file_4.zip Uploading trickle at timestep: 432000 05:00:02 STEP 121 H= 121:00 +CPU= 34.130 05:00:37 STEP 122 H= 122:00 +CPU= 34.867 05:01:30 STEP 123 H= 123:00 +CPU= 52.232 05:02:06 STEP 124 H= 124:00 +CPU= 35.186 05:02:40 STEP 125 H= 125:00 +CPU= 34.033 05:03:34 STEP 126 H= 126:00 +CPU= 54.142 05:04:08 STEP 127 H= 127:00 +CPU= 33.380 05:04:43 STEP 128 H= 128:00 +CPU= 35.208 05:05:35 STEP 129 H= 129:00 +CPU= 51.678 05:06:10 STEP 130 H= 130:00 +CPU= 34.774 05:06:44 STEP 131 H= 131:00 +CPU= 34.534 05:07:42 STEP 132 H= 132:00 +CPU= 57.383 Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGhpi1+000132 Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHhpi1+000132 Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAhpi1+000132 05:08:16 STEP 133 H= 133:00 +CPU= 34.181 05:08:51 STEP 134 H= 134:00 +CPU= 34.719 05:09:44 STEP 135 H= 135:00 +CPU= 52.835 05:10:18 STEP 136 H= 136:00 +CPU= 34.152 05:10:52 STEP 137 H= 137:00 +CPU= 34.289 05:11:46 STEP 138 H= 138:00 +CPU= 53.357 05:12:19 STEP 139 H= 139:00 +CPU= 33.381 05:12:56 STEP 140 H= 140:00 +CPU= 36.669 05:13:47 STEP 141 H= 141:00 +CPU= 51.027 05:14:23 STEP 142 H= 142:00 +CPU= 35.245 05:14:58 STEP 143 H= 143:00 +CPU= 34.962 05:16:00 STEP 144 H= 144:00 +CPU= 61.377 Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGhpi1+000144 Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHhpi1+000144 Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAhpi1+000144 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163130/ICMGGhpi1+000120 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163130/ICMSHhpi1+000120 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163130/ICMUAhpi1+000120 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163130/ICMGGhpi1+000132 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163130/ICMSHhpi1+000132 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163130/ICMUAhpi1+000132 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0041_2021050100_123_945_12163130_1_r110890581_5.zip Uploading the intermediate file: upload_file_5.zip Uploading trickle at timestep: 518400 05:16:35 STEP 145 H= 145:00 +CPU= 34.249 05:17:11 STEP 146 H= 146:00 +CPU= 36.037 05:18:04 STEP 147 H= 147:00 +CPU= 52.545 05:18:38 STEP 148 H= 148:00 +CPU= 34.195 05:19:12 STEP 149 H= 149:00 +CPU= 33.012 05:20:06 STEP 150 H= 150:00 +CPU= 53.597 05:20:39 STEP 151 H= 151:00 +CPU= 33.404 05:21:14 STEP 152 H= 152:00 +CPU= 34.457 05:22:06 STEP 153 H= 153:00 +CPU= 51.903 05:22:40 STEP 154 H= 154:00 +CPU= 34.725 05:23:15 STEP 155 H= 155:00 +CPU= 34.150 05:24:16 STEP 156 H= 156:00 +CPU= 59.883 Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGhpi1+000156 Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHhpi1+000156 Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAhpi1+000156 05:24:51 STEP 157 H= 157:00 +CPU= 35.005 05:25:26 STEP 158 H= 158:00 +CPU= 34.454 05:26:18 STEP 159 H= 159:00 +CPU= 52.277 05:26:53 STEP 160 H= 160:00 +CPU= 35.018 05:27:28 STEP 161 H= 161:00 +CPU= 33.901 05:28:24 STEP 162 H= 162:00 +CPU= 54.994 05:28:59 STEP 163 H= 163:00 +CPU= 34.027 05:29:34 STEP 164 H= 164:00 +CPU= 34.925 05:30:26 STEP 165 H= 165:00 +CPU= 52.271 05:31:01 STEP 166 H= 166:00 +CPU= 35.144 05:31:37 STEP 167 H= 167:00 +CPU= 34.900 05:32:38 STEP 168 H= 168:00 +CPU= 59.879 Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGhpi1+000168 Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHhpi1+000168 Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAhpi1+000168 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163130/ICMGGhpi1+000144 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163130/ICMSHhpi1+000144 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163130/ICMUAhpi1+000144 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163130/ICMGGhpi1+000156 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163130/ICMSHhpi1+000156 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163130/ICMUAhpi1+000156 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0041_2021050100_123_945_12163130_1_r110890581_6.zip Uploading the intermediate file: upload_file_6.zip Uploading trickle at timestep: 604800 05:33:12 STEP 169 H= 169:00 +CPU= 32.766 05:33:47 STEP 170 H= 170:00 +CPU= 34.869 05:34:39 STEP 171 H= 171:00 +CPU= 51.714 05:35:14 STEP 172 H= 172:00 +CPU= 35.145 05:35:49 STEP 173 H= 173:00 +CPU= 34.229 05:36:45 STEP 174 H= 174:00 +CPU= 54.946 05:37:32 STEP 175 H= 175:00 +CPU= 35.954 05:38:07 STEP 176 H= 176:00 +CPU= 35.066 05:39:01 STEP 177 H= 177:00 +CPU= 53.451 05:39:37 STEP 178 H= 178:00 +CPU= 35.867 forrtl: severe (41): insufficient virtual memory Image PC Routine Line Source master.exe 0000000001D2B9CB Unknown Unknown Unknown master.exe 0000000000BF9848 gp_model_ 508 gp_model.F90 master.exe 00000000012E9482 gp_model_heap_ 74 gp_model_heap.F90 master.exe 0000000000BBC1CE scan2m_ 535 scan2m.F90 master.exe 000000000057A664 stepo_ 327 stepo.F90 master.exe 000000000055DEEC cnt4_ 1133 cnt4.F90 master.exe 00000000005471C9 cnt3_ 267 cnt3.F90 master.exe 0000000000546412 cnt2_ 88 cnt2.F90 master.exe 0000000000545E28 cnt1_ 92 cnt1.F90 master.exe 000000000040708D cnt0_ 146 cnt0.F90 master.exe 000000000040220F MAIN__ 96 master.F90 master.exe 00000000004021A2 Unknown Unknown Unknown master.exe 0000000001DCA390 Unknown Unknown Unknown master.exe 000000000040206E Unknown Unknown Unknown The child process terminated with status: 0 ..Failed, model did not complete successfully </stderr_txt> ]]> |
Latest Trickles Received | ||||||
---|---|---|---|---|---|---|
Time Sent (UTC) | Host ID | Result ID | Result Name | Timestep | CPU Time (sec) | Average (sec/TS) |
09 May 2023 14:57:58 | 1536567 | 22246047 | oifs_43r3_ps_0041_2021050100_123_945_12163130_1 | 604,800 | 7,030 | 0.0116 |
09 May 2023 14:57:58 | 1536567 | 22246047 | oifs_43r3_ps_0041_2021050100_123_945_12163130_1 | 518,400 | 6,032 | 0.0116 |
09 May 2023 14:57:58 | 1536567 | 22246047 | oifs_43r3_ps_0041_2021050100_123_945_12163130_1 | 432,000 | 5,049 | 0.0117 |
09 May 2023 14:57:58 | 1536567 | 22246047 | oifs_43r3_ps_0041_2021050100_123_945_12163130_1 | 345,600 | 4,045 | 0.0117 |
09 May 2023 14:57:41 | 1536567 | 22246047 | oifs_43r3_ps_0041_2021050100_123_945_12163130_1 | 259,200 | 3,053 | 0.0118 |
09 May 2023 14:57:41 | 1536567 | 22246047 | oifs_43r3_ps_0041_2021050100_123_945_12163130_1 | 172,800 | 2,068 | 0.0120 |
09 May 2023 14:57:41 | 1536567 | 22246047 | oifs_43r3_ps_0041_2021050100_123_945_12163130_1 | 86,400 | 1,071 | 0.0124 |
©2024 climateprediction.net