Name | oifs_43r3_ps_0470_2011050100_123_980_12197114_0 |
Workunit | 12197114 |
Created | 24 Dec 2022, 2:26:31 UTC |
Sent | 13 Jan 2023, 17:00:43 UTC |
Report deadline | 12 Feb 2023, 17:00:43 UTC |
Received | 15 Jan 2023, 16:57:41 UTC |
Server state | Over |
Outcome | Computation error |
Client state | Compute error |
Exit status | 196 (0x000000C4) EXIT_DISK_LIMIT_EXCEEDED |
Computer ID | 1519251 |
Run time | 4 hours 53 min 3 sec |
CPU time | 4 hours 3 min 35 sec |
Validate state | Invalid |
Credit | 0.00 |
Device peak FLOPS | 4.94 GFLOPS |
Application version | OpenIFS 43r3 Perturbed Surface v1.05 x86_64-pc-linux-gnu |
Peak working set size | 4,415.21 MB |
Peak swap size | 4,978.08 MB |
Peak disk usage | 7,697.68 MB |
Stderr | <core_client_version>7.16.6</core_client_version> <![CDATA[ <message> Disk usage limit exceeded</message> <stderr_txt> [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1 [EC_DRHOOK:p500:1:1:2220640:2220640] [20230115:094139:1673743299:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1 [EC_DRHOOK:p500:1:1:2220640:2220640] [20230115:094139:1673743299:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1 [EC_DRHOOK:p500:1:1:2220640:2220640] [20230115:094139:1673743299:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef> [EC_DRHOOK:p500:1:1:2220640:2220640] [20230115:094139:1673743299:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef> [EC_DRHOOK:p500:1:1:2220640:2220640] [20230115:094139:1673743299:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:p500:1:1:2220640:2220640] [20230115:094139:1673743299:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil)) [EC_DRHOOK:p500:1:1:2220640:2220640] [20230115:094139:1673743299:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:p500:1:1:2220640:2220640] [20230115:094139:1673743299:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil)) [EC_DRHOOK:p500:1:1:2220640:2220640] [20230115:094139:1673743299:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1 [EC_DRHOOK:p500:1:1:2220640:2220640] [20230115:094139:1673743299:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:p500:1:1:2220640:2220640] [20230115:094139:1673743299:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:p500:1:1:2220640:2220640] [20230115:094139:1673743299:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil)) [EC_DRHOOK:p500:1:1:2220640:2220640] [20230115:094139:1673743299:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:p500:1:1:2220640:2220640] [20230115:094139:1673743299:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:p500:1:1:2220640:2220640] [20230115:094139:1673743299:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:p500:1:1:2220640:2220640] [20230115:094139:1673743299:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil)) [EC_DRHOOK:p500:1:1:2220640:2220640] [20230115:094139:1673743299:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil)) [EC_DRHOOK:p500:1:1:2220640:2220640] [20230115:094139:1673743299:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil)) [EC_DRHOOK:p500:1:1:2220640:2220640] [20230115:094139:1673743299:0.001] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef> Suspend request received from the BOINC client, suspending the child process Resuming the child process 09:44:11 STEP 312 H= 312:00 +CPU= 36.873 09:44:32 STEP 313 H= 313:00 +CPU= 20.971 Suspend request received from the BOINC client, suspending the child process Resuming the child process 09:47:26 STEP 314 H= 314:00 +CPU= 21.451 Suspend request received from the BOINC client, suspending the child process Resuming the child process 09:49:40 STEP 315 H= 315:00 +CPU= 32.608 Suspend request received from the BOINC client, suspending the child process Resuming the child process 09:52:12 STEP 316 H= 316:00 +CPU= 21.310 09:52:34 STEP 317 H= 317:00 +CPU= 21.049 Suspend request received from the BOINC client, suspending the child process Resuming the child process 09:55:28 STEP 318 H= 318:00 +CPU= 33.158 09:55:49 STEP 319 H= 319:00 +CPU= 21.180 Suspend request received from the BOINC client, suspending the child process Resuming the child process 09:58:12 STEP 320 H= 320:00 +CPU= 21.305 Suspend request received from the BOINC client, suspending the child process Resuming the child process 10:00:56 STEP 321 H= 321:00 +CPU= 32.914 Suspend request received from the BOINC client, suspending the child process Resuming the child process 10:02:49 STEP 322 H= 322:00 +CPU= 21.922 10:03:10 STEP 323 H= 323:00 +CPU= 20.539 Suspend request received from the BOINC client, suspending the child process Resuming the child process 10:05:47 STEP 324 H= 324:00 +CPU= 36.059 Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000324 Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000324 Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000324 10:06:07 STEP 325 H= 325:00 +CPU= 20.280 Suspend request received from the BOINC client, suspending the child process Resuming the child process 10:08:20 STEP 326 H= 326:00 +CPU= 21.004 Suspend request received from the BOINC client, suspending the child process Resuming the child process 10:10:54 STEP 327 H= 327:00 +CPU= 32.848 Suspend request received from the BOINC client, suspending the child process Resuming the child process 10:12:46 STEP 328 H= 328:00 +CPU= 21.152 10:13:07 STEP 329 H= 329:00 +CPU= 20.601 Suspend request received from the BOINC client, suspending the child process Resuming the child process 10:15:21 STEP 330 H= 330:00 +CPU= 33.398 10:15:41 STEP 331 H= 331:00 +CPU= 19.956 Suspend request received from the BOINC client, suspending the child process Resuming the child process 10:18:35 STEP 332 H= 332:00 +CPU= 21.381 Suspend request received from the BOINC client, suspending the child process Resuming the child process 10:20:59 STEP 333 H= 333:00 +CPU= 33.168 Suspend request received from the BOINC client, suspending the child process Resuming the child process 10:23:01 STEP 334 H= 334:00 +CPU= 20.767 Suspend request received from the BOINC client, suspending the child process Resuming the child process 10:25:24 STEP 335 H= 335:00 +CPU= 21.141 10:26:02 STEP 336 H= 336:00 +CPU= 38.383 Suspend request received from the BOINC client, suspending the child process Resuming the child process Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000336 Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000336 Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000336 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197114/ICMGGhq0f+000312 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197114/ICMSHhq0f+000312 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197114/ICMUAhq0f+000312 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197114/ICMGGhq0f+000324 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197114/ICMSHhq0f+000324 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197114/ICMUAhq0f+000324 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0470_2011050100_123_980_12197114_0_r1819580094_13.zip Uploading the intermediate file: upload_file_13.zip 10:28:34 STEP 337 H= 337:00 +CPU= 20.945 10:28:56 STEP 338 H= 338:00 +CPU= 21.293 Suspend request received from the BOINC client, suspending the child process Resuming the child process 10:31:29 STEP 339 H= 339:00 +CPU= 33.029 Suspend request received from the BOINC client, suspending the child process Resuming the child process 10:33:41 STEP 340 H= 340:00 +CPU= 21.171 10:34:03 STEP 341 H= 341:00 +CPU= 21.263 Suspend request received from the BOINC client, suspending the child process Resuming the child process 10:36:48 STEP 342 H= 342:00 +CPU= 33.474 Suspend request received from the BOINC client, suspending the child process Resuming the child process 10:39:00 STEP 343 H= 343:00 +CPU= 20.686 Suspend request received from the BOINC client, suspending the child process Resuming the child process 10:40:33 STEP 344 H= 344:00 +CPU= 21.349 10:41:06 STEP 345 H= 345:00 +CPU= 32.796 Suspend request received from the BOINC client, suspending the child process Resuming the child process 10:43:28 STEP 346 H= 346:00 +CPU= 21.124 10:43:49 STEP 347 H= 347:00 +CPU= 20.166 Suspend request received from the BOINC client, suspending the child process Resuming the child process 10:46:26 STEP 348 H= 348:00 +CPU= 36.478 Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000348 Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000348 Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000348 Suspend request received from the BOINC client, suspending the child process Resuming the child process 10:49:28 STEP 349 H= 349:00 +CPU= 20.102 10:49:49 STEP 350 H= 350:00 +CPU= 21.281 10:50:21 STEP 351 H= 351:00 +CPU= 32.286 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 10:54:55 STEP 352 H= 352:00 +CPU= 21.175 10:55:15 STEP 353 H= 353:00 +CPU= 20.364 Suspend request received from the BOINC client, suspending the child process Resuming the child process 10:58:40 STEP 354 H= 354:00 +CPU= 33.869 10:59:01 STEP 355 H= 355:00 +CPU= 20.740 10:59:22 STEP 356 H= 356:00 +CPU= 20.399 Suspend request received from the BOINC client, suspending the child process Resuming the child process 11:02:45 STEP 357 H= 357:00 +CPU= 32.329 Suspend request received from the BOINC client, suspending the child process Resuming the child process 11:05:08 STEP 358 H= 358:00 +CPU= 21.255 11:05:29 STEP 359 H= 359:00 +CPU= 21.386 Suspend request received from the BOINC client, suspending the child process Resuming the child process 11:08:40 STEP 360 H= 360:00 +CPU= 38.869 Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000360 Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000360 Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000360 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197114/ICMGGhq0f+000336 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197114/ICMSHhq0f+000336 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197114/ICMUAhq0f+000336 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197114/ICMGGhq0f+000348 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197114/ICMSHhq0f+000348 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197114/ICMUAhq0f+000348 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0470_2011050100_123_980_12197114_0_r1819580094_14.zip Uploading the intermediate file: upload_file_14.zip 11:09:00 STEP 361 H= 361:00 +CPU= 20.273 Suspend request received from the BOINC client, suspending the child process Resuming the child process 11:12:03 STEP 362 H= 362:00 +CPU= 20.438 Suspend request received from the BOINC client, suspending the child process Resuming the child process 11:15:07 STEP 363 H= 363:00 +CPU= 33.150 11:15:28 STEP 364 H= 364:00 +CPU= 21.113 Suspend request received from the BOINC client, suspending the child process Resuming the child process 11:17:51 STEP 365 H= 365:00 +CPU= 20.288 Suspend request received from the BOINC client, suspending the child process Resuming the child process 11:21:27 STEP 366 H= 366:00 +CPU= 33.516 11:21:47 STEP 367 H= 367:00 +CPU= 20.364 Suspend request received from the BOINC client, suspending the child process Resuming the child process 11:25:00 STEP 368 H= 368:00 +CPU= 21.167 Suspend request received from the BOINC client, suspending the child process Resuming the child process 11:28:05 STEP 369 H= 369:00 +CPU= 32.959 11:28:26 STEP 370 H= 370:00 +CPU= 21.625 Suspend request received from the BOINC client, suspending the child process Resuming the child process 11:30:58 STEP 371 H= 371:00 +CPU= 20.468 11:31:34 STEP 372 H= 372:00 +CPU= 35.416 Suspend request received from the BOINC client, suspending the child process Resuming the child process Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000372 Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000372 Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000372 11:35:15 STEP 373 H= 373:00 +CPU= 20.701 Suspend request received from the BOINC client, suspending the child process Resuming the child process 11:37:57 STEP 374 H= 374:00 +CPU= 20.456 Suspend request received from the BOINC client, suspending the child process Resuming the child process 11:40:13 STEP 375 H= 375:00 +CPU= 33.288 Suspend request received from the BOINC client, suspending the child process Resuming the child process 11:42:36 STEP 376 H= 376:00 +CPU= 20.790 11:42:56 STEP 377 H= 377:00 +CPU= 20.661 Suspend request received from the BOINC client, suspending the child process Resuming the child process Quit request received from BOINC client, ending the child process ..Failed, model did not complete successfully (argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu (argv1) start_date: 2011050100 (argv2) exptid: hq0f (argv3) unique_member_id: 0470 (argv4) batchid: 980 (argv5) wuid: 12197114 (argv6) fclen: 123 (argv7) app_name: oifs_43r3_ps (argv8) nthreads: 1 Working directory is: /var/lib/boinc-client/slots/4 Project directory is: /var/lib/boinc-client/projects/climateprediction.net/ app name: oifs_43r3_ps version: 1.05 Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197114 ..mkdir for temp folder for results failed Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/4/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip Unzipping the app zip file: /var/lib/boinc-client/slots/4/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip Copying the namelist files from: ../../projects/climateprediction.net/jf_800066ac9e16545a34d5ff2406f1f5a8 to: /var/lib/boinc-client/slots/4/oifs_43r3_ps_0470_2011050100_123_980_12197114.zip Unzipping the namelist zip file: /var/lib/boinc-client/slots/4/oifs_43r3_ps_0470_2011050100_123_980_12197114.zip ic_ancil_file: ic_ancil_12197114 ifsdata_file: ifsdata_12197114 climate_data_file: clim_data_12197114 horiz_resolution: 159 vert_resolution: 60 grid_type: l_2 upload_interval: 24 utstep: 3600 nfrres: restart dump frequency (steps) 24 Copying IC ancils from: ../../projects/climateprediction.net/jf_8c8abd1e077c18228d5f79e83b032356 to: /var/lib/boinc-client/slots/4/ic_ancil_12197114.zip Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/4/ic_ancil_12197114.zip ..mkdir for ifsdata folder failed Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_1249124d769e249c0b1746ca0368fc1a to: /var/lib/boinc-client/slots/4/ifsdata/ifsdata_12197114.zip Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/4/ifsdata/ifsdata_12197114.zip ..mkdir for the climate data folder failed Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/4/159l_2/clim_data_12197114.zip Unzipping the climate data zip file: /var/lib/boinc-client/slots/4/159l_2/clim_data_12197114.zip Checking for progress XML file: /var/lib/boinc-client/slots/4/progress_file_12197114.xml Opened progress file ok : /var/lib/boinc-client/slots/4/progress_file_12197114.xml -- Model is restarting -- Adjusting last_iter, 378, to previous model restart step. Creating progress file: /var/lib/boinc-client/slots/4/progress_file_12197114.xml last_cpu_time: 11390 upload_file_number: 15 last_iter: 359 last_upload: 1296000 model_completed: 0 total_length_of_simulation: 10627200 result_base_name: oifs_43r3_ps_0470_2011050100_123_980_12197114_0_r1819580094 The child process has been launched with process id: 2239593 Executing the command: /var/lib/boinc-client/slots/4/oifs_43r3_model.exe [EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1 [EC_DRHOOK:p500:1:1:2239593:2239593] [20230115:114923:1673750963:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0 [EC_DRHOOK:p500:1:1:2239593:2239593] [20230115:114923:1673750963:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1 [EC_DRHOOK:p500:1:1:2239593:2239593] [20230115:114923:1673750963:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0) [EC_DRHOOK:p500:1:1:2239593:2239593] [20230115:114923:1673750963:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1 [EC_DRHOOK:p500:1:1:2239593:2239593] [20230115:114923:1673750963:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000 [EC_DRHOOK:p500:1:1:2239593:2239593] [20230115:114923:1673750963:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647) [EC_DRHOOK:p500:1:1:2239593:2239593] [20230115:114923:1673750963:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16 [EC_DRHOOK:p500:1:1:2239593:2239593] [20230115:114923:1673750963:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0 [EC_DRHOOK:p500:1:1:2239593:2239593] [20230115:114923:1673750963:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500 [EC_DRHOOK:p500:1:1:2239593:2239593] [20230115:114923:1673750963:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1 [EC_DRHOOK:p500:1:1:2239593:2239593] [20230115:114923:1673750963:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1 [EC_DRHOOK:p500:1:1:2239593:2239593] [20230115:114923:1673750963:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1 [EC_DRHOOK:p500:1:1:2239593:2239593] [20230115:114923:1673750963:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1 [EC_DRHOOK:p500:1:1:2239593:2239593] [20230115:114923:1673750963:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef> [EC_DRHOOK:p500:1:1:2239593:2239593] [20230115:114923:1673750963:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef> [EC_DRHOOK:p500:1:1:2239593:2239593] [20230115:114923:1673750963:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:p500:1:1:2239593:2239593] [20230115:114923:1673750963:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil)) [EC_DRHOOK:p500:1:1:2239593:2239593] [20230115:114923:1673750963:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:p500:1:1:2239593:2239593] [20230115:114923:1673750963:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil)) [EC_DRHOOK:p500:1:1:2239593:2239593] [20230115:114923:1673750963:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1 [EC_DRHOOK:p500:1:1:2239593:2239593] [20230115:114923:1673750963:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:p500:1:1:2239593:2239593] [20230115:114923:1673750963:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:p500:1:1:2239593:2239593] [20230115:114923:1673750963:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil)) [EC_DRHOOK:p500:1:1:2239593:2239593] [20230115:114923:1673750963:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:p500:1:1:2239593:2239593] [20230115:114923:1673750963:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:p500:1:1:2239593:2239593] [20230115:114923:1673750963:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:p500:1:1:2239593:2239593] [20230115:114923:1673750963:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil)) [EC_DRHOOK:p500:1:1:2239593:2239593] [20230115:114923:1673750963:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil)) [EC_DRHOOK:p500:1:1:2239593:2239593] [20230115:114923:1673750963:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil)) [EC_DRHOOK:p500:1:1:2239593:2239593] [20230115:114923:1673750963:0.001] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef> Suspend request received from the BOINC client, suspending the child process Resuming the child process 11:52:48 STEP 360 H= 360:00 +CPU= 36.792 Suspend request received from the BOINC client, suspending the child process Resuming the child process 11:55:10 STEP 361 H= 361:00 +CPU= 20.989 11:55:31 STEP 362 H= 362:00 +CPU= 21.252 Suspend request received from the BOINC client, suspending the child process Resuming the child process 11:57:55 STEP 363 H= 363:00 +CPU= 32.658 11:58:16 STEP 364 H= 364:00 +CPU= 20.677 Suspend request received from the BOINC client, suspending the child process Resuming the child process 12:01:08 STEP 365 H= 365:00 +CPU= 20.562 Suspend request received from the BOINC client, suspending the child process Resuming the child process 12:05:22 STEP 366 H= 366:00 +CPU= 33.958 12:05:44 STEP 367 H= 367:00 +CPU= 21.404 Suspend request received from the BOINC client, suspending the child process Resuming the child process 12:09:16 STEP 368 H= 368:00 +CPU= 21.402 Suspend request received from the BOINC client, suspending the child process Resuming the child process 12:12:59 STEP 369 H= 369:00 +CPU= 32.467 12:13:21 STEP 370 H= 370:00 +CPU= 21.270 Suspend request received from the BOINC client, suspending the child process Resuming the child process 12:16:34 STEP 371 H= 371:00 +CPU= 21.783 Suspend request received from the BOINC client, suspending the child process Resuming the child process 12:19:53 STEP 372 H= 372:00 +CPU= 36.062 Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000372 Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000372 Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000372 Suspend request received from the BOINC client, suspending the child process Resuming the child process 12:23:15 STEP 373 H= 373:00 +CPU= 20.713 12:23:37 STEP 374 H= 374:00 +CPU= 21.024 Suspend request received from the BOINC client, suspending the child process Resuming the child process 12:27:21 STEP 375 H= 375:00 +CPU= 32.444 Suspend request received from the BOINC client, suspending the child process Resuming the child process 12:30:24 STEP 376 H= 376:00 +CPU= 21.085 12:30:44 STEP 377 H= 377:00 +CPU= 20.450 Suspend request received from the BOINC client, suspending the child process Resuming the child process 12:34:20 STEP 378 H= 378:00 +CPU= 34.339 Suspend request received from the BOINC client, suspending the child process Resuming the child process 12:36:21 STEP 379 H= 379:00 +CPU= 20.150 12:36:42 STEP 380 H= 380:00 +CPU= 20.812 Suspend request received from the BOINC client, suspending the child process Resuming the child process 22:22:35 STEP 381 H= 381:00 +CPU= 32.076 22:22:56 STEP 382 H= 382:00 +CPU= 21.188 Suspend request received from the BOINC client, suspending the child process Resuming the child process 22:26:37 STEP 383 H= 383:00 +CPU= 20.508 Suspend request received from the BOINC client, suspending the child process Resuming the child process 22:29:27 STEP 384 H= 384:00 +CPU= 37.938 Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000384 Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000384 Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000384 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197114/ICMGGhq0f+000360 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197114/ICMSHhq0f+000360 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197114/ICMUAhq0f+000360 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197114/ICMGGhq0f+000372 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197114/ICMSHhq0f+000372 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197114/ICMUAhq0f+000372 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0470_2011050100_123_980_12197114_0_r1819580094_15.zip Uploading the intermediate file: upload_file_15.zip 22:29:45 STEP 385 H= 385:00 +CPU= 18.050 Suspend request received from the BOINC client, suspending the child process Resuming the child process 22:32:04 STEP 386 H= 386:00 +CPU= 19.368 Quit request received from BOINC client, ending the child process (argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu (argv1) start_date: 2011050100 (argv2) exptid: hq0f (argv3) unique_member_id: 0470 (argv4) batchid: 980 (argv5) wuid: 12197114 (argv6) fclen: 123 (argv7) app_name: oifs_43r3_ps (argv8) nthreads: 1 Working directory is: /var/lib/boinc-client/slots/4 Project directory is: /var/lib/boinc-client/projects/climateprediction.net/ app name: oifs_43r3_ps version: 1.05 Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197114 ..mkdir for temp folder for results failed Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/4/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip Unzipping the app zip file: /var/lib/boinc-client/slots/4/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip Copying the namelist files from: ../../projects/climateprediction.net/jf_800066ac9e16545a34d5ff2406f1f5a8 to: /var/lib/boinc-client/slots/4/oifs_43r3_ps_0470_2011050100_123_980_12197114.zip Unzipping the namelist zip file: /var/lib/boinc-client/slots/4/oifs_43r3_ps_0470_2011050100_123_980_12197114.zip ic_ancil_file: ic_ancil_12197114 ifsdata_file: ifsdata_12197114 climate_data_file: clim_data_12197114 horiz_resolution: 159 vert_resolution: 60 grid_type: l_2 upload_interval: 24 utstep: 3600 nfrres: restart dump frequency (steps) 24 Copying IC ancils from: ../../projects/climateprediction.net/jf_8c8abd1e077c18228d5f79e83b032356 to: /var/lib/boinc-client/slots/4/ic_ancil_12197114.zip Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/4/ic_ancil_12197114.zip ..mkdir for ifsdata folder failed Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_1249124d769e249c0b1746ca0368fc1a to: /var/lib/boinc-client/slots/4/ifsdata/ifsdata_12197114.zip Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/4/ifsdata/ifsdata_12197114.zip ..mkdir for the climate data folder failed Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/4/159l_2/clim_data_12197114.zip Unzipping the climate data zip file: /var/lib/boinc-client/slots/4/159l_2/clim_data_12197114.zip Checking for progress XML file: /var/lib/boinc-client/slots/4/progress_file_12197114.xml Opened progress file ok : /var/lib/boinc-client/slots/4/progress_file_12197114.xml -- Model is restarting -- Adjusting last_iter, 386, to previous model restart step. Creating progress file: /var/lib/boinc-client/slots/4/progress_file_12197114.xml last_cpu_time: 12073 upload_file_number: 16 last_iter: 383 last_upload: 1382400 model_completed: 0 total_length_of_simulation: 10627200 result_base_name: oifs_43r3_ps_0470_2011050100_123_980_12197114_0_r1819580094 The child process has been launched with process id: 2360989 Executing the command: /var/lib/boinc-client/slots/4/oifs_43r3_model.exe [EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1 [EC_DRHOOK:p500:1:1:2360989:2360989] [20230115:223438:1673789678:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0 [EC_DRHOOK:p500:1:1:2360989:2360989] [20230115:223438:1673789678:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1 [EC_DRHOOK:p500:1:1:2360989:2360989] [20230115:223438:1673789678:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0) [EC_DRHOOK:p500:1:1:2360989:2360989] [20230115:223438:1673789678:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1 [EC_DRHOOK:p500:1:1:2360989:2360989] [20230115:223438:1673789678:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000 [EC_DRHOOK:p500:1:1:2360989:2360989] [20230115:223438:1673789678:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647) [EC_DRHOOK:p500:1:1:2360989:2360989] [20230115:223438:1673789678:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16 [EC_DRHOOK:p500:1:1:2360989:2360989] [20230115:223438:1673789678:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0 [EC_DRHOOK:p500:1:1:2360989:2360989] [20230115:223438:1673789678:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500 [EC_DRHOOK:p500:1:1:2360989:2360989] [20230115:223438:1673789678:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1 [EC_DRHOOK:p500:1:1:2360989:2360989] [20230115:223438:1673789678:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1 [EC_DRHOOK:p500:1:1:2360989:2360989] [20230115:223438:1673789678:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1 [EC_DRHOOK:p500:1:1:2360989:2360989] [20230115:223438:1673789678:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1 [EC_DRHOOK:p500:1:1:2360989:2360989] [20230115:223438:1673789678:0.001] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef> [EC_DRHOOK:p500:1:1:2360989:2360989] [20230115:223438:1673789678:0.001] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef> [EC_DRHOOK:p500:1:1:2360989:2360989] [20230115:223438:1673789678:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:p500:1:1:2360989:2360989] [20230115:223438:1673789678:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil)) [EC_DRHOOK:p500:1:1:2360989:2360989] [20230115:223438:1673789678:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:p500:1:1:2360989:2360989] [20230115:223438:1673789678:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil)) [EC_DRHOOK:p500:1:1:2360989:2360989] [20230115:223438:1673789678:0.001] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1 [EC_DRHOOK:p500:1:1:2360989:2360989] [20230115:223438:1673789678:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:p500:1:1:2360989:2360989] [20230115:223438:1673789678:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:p500:1:1:2360989:2360989] [20230115:223438:1673789678:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil)) [EC_DRHOOK:p500:1:1:2360989:2360989] [20230115:223438:1673789678:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:p500:1:1:2360989:2360989] [20230115:223438:1673789678:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:p500:1:1:2360989:2360989] [20230115:223438:1673789678:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:p500:1:1:2360989:2360989] [20230115:223438:1673789678:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil)) [EC_DRHOOK:p500:1:1:2360989:2360989] [20230115:223438:1673789678:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil)) [EC_DRHOOK:p500:1:1:2360989:2360989] [20230115:223438:1673789678:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil)) [EC_DRHOOK:p500:1:1:2360989:2360989] [20230115:223438:1673789678:0.001] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef> Suspend request received from the BOINC client, suspending the child process Resuming the child process 22:37:18 STEP 384 H= 384:00 +CPU= 36.572 Suspend request received from the BOINC client, suspending the child process Resuming the child process 22:39:40 STEP 385 H= 385:00 +CPU= 20.983 Suspend request received from the BOINC client, suspending the child process Resuming the child process 22:41:33 STEP 386 H= 386:00 +CPU= 21.405 22:42:03 STEP 387 H= 387:00 +CPU= 30.778 Suspend request received from the BOINC client, suspending the child process Resuming the child process 22:44:46 STEP 388 H= 388:00 +CPU= 20.696 Suspend request received from the BOINC client, suspending the child process Resuming the child process 22:46:48 STEP 389 H= 389:00 +CPU= 20.253 Suspend request received from the BOINC client, suspending the child process Resuming the child process 22:49:02 STEP 390 H= 390:00 +CPU= 33.280 Suspend request received from the BOINC client, suspending the child process Resuming the child process 22:51:01 STEP 391 H= 391:00 +CPU= 19.263 22:51:22 STEP 392 H= 392:00 +CPU= 20.960 Suspend request received from the BOINC client, suspending the child process Resuming the child process 22:53:56 STEP 393 H= 393:00 +CPU= 31.680 Suspend request received from the BOINC client, suspending the child process Resuming the child process 22:56:08 STEP 394 H= 394:00 +CPU= 20.401 Suspend request received from the BOINC client, suspending the child process Resuming the child process 22:58:30 STEP 395 H= 395:00 +CPU= 20.834 Suspend request received from the BOINC client, suspending the child process Resuming the child process 23:00:47 STEP 396 H= 396:00 +CPU= 36.180 Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000396 Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000396 Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000396 Suspend request received from the BOINC client, suspending the child process Resuming the child process 23:02:48 STEP 397 H= 397:00 +CPU= 20.159 Suspend request received from the BOINC client, suspending the child process Resuming the child process 23:05:00 STEP 398 H= 398:00 +CPU= 20.808 Suspend request received from the BOINC client, suspending the child process Resuming the child process 23:07:05 STEP 399 H= 399:00 +CPU= 32.446 23:07:26 STEP 400 H= 400:00 +CPU= 20.877 Suspend request received from the BOINC client, suspending the child process Resuming the child process 23:09:36 STEP 401 H= 401:00 +CPU= 20.233 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 23:13:23 STEP 402 H= 402:00 +CPU= 32.380 23:13:43 STEP 403 H= 403:00 +CPU= 20.255 Suspend request received from the BOINC client, suspending the child process Resuming the child process 23:15:36 STEP 404 H= 404:00 +CPU= 21.161 Suspend request received from the BOINC client, suspending the child process Resuming the child process 23:18:20 STEP 405 H= 405:00 +CPU= 32.099 Suspend request received from the BOINC client, suspending the child process Resuming the child process 23:20:22 STEP 406 H= 406:00 +CPU= 21.009 23:20:43 STEP 407 H= 407:00 +CPU= 21.498 Suspend request received from the BOINC client, suspending the child process Resuming the child process 23:23:33 STEP 408 H= 408:00 +CPU= 37.493 Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000408 Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000408 Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000408 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197114/ICMGGhq0f+000384 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197114/ICMSHhq0f+000384 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197114/ICMUAhq0f+000384 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197114/ICMGGhq0f+000396 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197114/ICMSHhq0f+000396 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197114/ICMUAhq0f+000396 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0470_2011050100_123_980_12197114_0_r1819580094_16.zip Uploading the intermediate file: upload_file_16.zip 23:23:51 STEP 409 H= 409:00 +CPU= 18.239 Suspend request received from the BOINC client, suspending the child process Resuming the child process 23:25:54 STEP 410 H= 410:00 +CPU= 19.709 Suspend request received from the BOINC client, suspending the child process Resuming the child process 23:28:39 STEP 411 H= 411:00 +CPU= 32.499 Suspend request received from the BOINC client, suspending the child process Resuming the child process 23:31:11 STEP 412 H= 412:00 +CPU= 20.757 23:31:31 STEP 413 H= 413:00 +CPU= 20.185 Suspend request received from the BOINC client, suspending the child process Resuming the child process 23:33:45 STEP 414 H= 414:00 +CPU= 33.013 Suspend request received from the BOINC client, suspending the child process Resuming the child process 23:36:06 STEP 415 H= 415:00 +CPU= 19.650 23:36:28 STEP 416 H= 416:00 +CPU= 21.348 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 23:40:43 STEP 417 H= 417:00 +CPU= 33.175 23:41:05 STEP 418 H= 418:00 +CPU= 21.701 Suspend request received from the BOINC client, suspending the child process Resuming the child process 23:43:17 STEP 419 H= 419:00 +CPU= 21.104 Suspend request received from the BOINC client, suspending the child process Resuming the child process 23:45:24 STEP 420 H= 420:00 +CPU= 35.447 Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000420 Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000420 Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000420 Suspend request received from the BOINC client, suspending the child process Resuming the child process 23:48:06 STEP 421 H= 421:00 +CPU= 19.835 23:48:26 STEP 422 H= 422:00 +CPU= 20.771 Suspend request received from the BOINC client, suspending the child process Resuming the child process 23:50:20 STEP 423 H= 423:00 +CPU= 31.676 Suspend request received from the BOINC client, suspending the child process Resuming the child process 23:52:24 STEP 424 H= 424:00 +CPU= 21.304 Suspend request received from the BOINC client, suspending the child process Resuming the child process 23:54:15 STEP 425 H= 425:00 +CPU= 19.622 Suspend request received from the BOINC client, suspending the child process Resuming the child process 23:56:39 STEP 426 H= 426:00 +CPU= 32.933 23:56:59 STEP 427 H= 427:00 +CPU= 19.747 Suspend request received from the BOINC client, suspending the child process Resuming the child process 23:59:10 STEP 428 H= 428:00 +CPU= 19.779 Suspend request received from the BOINC client, suspending the child process Resuming the child process 00:01:43 STEP 429 H= 429:00 +CPU= 31.825 00:02:03 STEP 430 H= 430:00 +CPU= 20.148 Suspend request received from the BOINC client, suspending the child process Resuming the child process 00:04:55 STEP 431 H= 431:00 +CPU= 19.804 00:05:32 STEP 432 H= 432:00 +CPU= 36.545 Suspend request received from the BOINC client, suspending the child process Resuming the child process Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000432 Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000432 Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000432 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197114/ICMGGhq0f+000408 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197114/ICMSHhq0f+000408 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197114/ICMUAhq0f+000408 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197114/ICMGGhq0f+000420 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197114/ICMSHhq0f+000420 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197114/ICMUAhq0f+000420 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0470_2011050100_123_980_12197114_0_r1819580094_17.zip Uploading the intermediate file: upload_file_17.zip 00:08:04 STEP 433 H= 433:00 +CPU= 19.847 Suspend request received from the BOINC client, suspending the child process Resuming the child process 00:10:46 STEP 434 H= 434:00 +CPU= 20.018 00:11:18 STEP 435 H= 435:00 +CPU= 31.987 Suspend request received from the BOINC client, suspending the child process Resuming the child process 00:13:51 STEP 436 H= 436:00 +CPU= 20.355 00:14:11 STEP 437 H= 437:00 +CPU= 20.018 Suspend request received from the BOINC client, suspending the child process Resuming the child process 00:16:44 STEP 438 H= 438:00 +CPU= 32.325 00:17:04 STEP 439 H= 439:00 +CPU= 20.103 Suspend request received from the BOINC client, suspending the child process Resuming the child process 00:19:36 STEP 440 H= 440:00 +CPU= 20.272 Suspend request received from the BOINC client, suspending the child process Resuming the child process 00:21:50 STEP 441 H= 441:00 +CPU= 31.962 00:22:10 STEP 442 H= 442:00 +CPU= 20.494 Suspend request received from the BOINC client, suspending the child process Resuming the child process 00:24:12 STEP 443 H= 443:00 +CPU= 19.838 Suspend request received from the BOINC client, suspending the child process Resuming the child process 00:26:58 STEP 444 H= 444:00 +CPU= 34.418 Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000444 Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000444 Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000444 00:27:17 STEP 445 H= 445:00 +CPU= 19.390 Suspend request received from the BOINC client, suspending the child process Resuming the child process 00:29:08 STEP 446 H= 446:00 +CPU= 20.795 Suspend request received from the BOINC client, suspending the child process Resuming the child process 00:31:20 STEP 447 H= 447:00 +CPU= 31.243 00:31:40 STEP 448 H= 448:00 +CPU= 19.869 Suspend request received from the BOINC client, suspending the child process Resuming the child process 00:33:50 STEP 449 H= 449:00 +CPU= 19.252 Suspend request received from the BOINC client, suspending the child process Resuming the child process 00:35:54 STEP 450 H= 450:00 +CPU= 32.394 00:36:13 STEP 451 H= 451:00 +CPU= 19.452 00:36:34 STEP 452 H= 452:00 +CPU= 21.180 Suspend request received from the BOINC client, suspending the child process Resuming the child process 00:39:37 STEP 453 H= 453:00 +CPU= 31.863 Suspend request received from the BOINC client, suspending the child process Resuming the child process 00:41:59 STEP 454 H= 454:00 +CPU= 20.490 Suspend request received from the BOINC client, suspending the child process Resuming the child process 00:43:51 STEP 455 H= 455:00 +CPU= 20.162 Suspend request received from the BOINC client, suspending the child process Resuming the child process 00:46:20 STEP 456 H= 456:00 +CPU= 37.204 Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000456 Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000456 Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000456 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197114/ICMGGhq0f+000432 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197114/ICMSHhq0f+000432 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197114/ICMUAhq0f+000432 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197114/ICMGGhq0f+000444 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197114/ICMSHhq0f+000444 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197114/ICMUAhq0f+000444 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0470_2011050100_123_980_12197114_0_r1819580094_18.zip Uploading the intermediate file: upload_file_18.zip 00:46:39 STEP 457 H= 457:00 +CPU= 19.540 Suspend request received from the BOINC client, suspending the child process Resuming the child process 00:48:29 STEP 458 H= 458:00 +CPU= 19.073 Suspend request received from the BOINC client, suspending the child process Resuming the child process 00:50:52 STEP 459 H= 459:00 +CPU= 31.200 Suspend request received from the BOINC client, suspending the child process Resuming the child process 00:52:24 STEP 460 H= 460:00 +CPU= 19.900 00:52:44 STEP 461 H= 461:00 +CPU= 20.097 Quit request received from BOINC client, ending the child process (argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu (argv1) start_date: 2011050100 (argv2) exptid: hq0f (argv3) unique_member_id: 0470 (argv4) batchid: 980 (argv5) wuid: 12197114 (argv6) fclen: 123 (argv7) app_name: oifs_43r3_ps (argv8) nthreads: 1 Working directory is: /var/lib/boinc-client/slots/4 Project directory is: /var/lib/boinc-client/projects/climateprediction.net/ app name: oifs_43r3_ps version: 1.05 Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12197114 ..mkdir for temp folder for results failed Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/4/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip Unzipping the app zip file: /var/lib/boinc-client/slots/4/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip Copying the namelist files from: ../../projects/climateprediction.net/jf_800066ac9e16545a34d5ff2406f1f5a8 to: /var/lib/boinc-client/slots/4/oifs_43r3_ps_0470_2011050100_123_980_12197114.zip Unzipping the namelist zip file: /var/lib/boinc-client/slots/4/oifs_43r3_ps_0470_2011050100_123_980_12197114.zip ic_ancil_file: ic_ancil_12197114 ifsdata_file: ifsdata_12197114 climate_data_file: clim_data_12197114 horiz_resolution: 159 vert_resolution: 60 grid_type: l_2 upload_interval: 24 utstep: 3600 nfrres: restart dump frequency (steps) 24 Copying IC ancils from: ../../projects/climateprediction.net/jf_8c8abd1e077c18228d5f79e83b032356 to: /var/lib/boinc-client/slots/4/ic_ancil_12197114.zip Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/4/ic_ancil_12197114.zip ..mkdir for ifsdata folder failed Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_1249124d769e249c0b1746ca0368fc1a to: /var/lib/boinc-client/slots/4/ifsdata/ifsdata_12197114.zip Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/4/ifsdata/ifsdata_12197114.zip ..mkdir for the climate data folder failed Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/4/159l_2/clim_data_12197114.zip Unzipping the climate data zip file: /var/lib/boinc-client/slots/4/159l_2/clim_data_12197114.zip Checking for progress XML file: /var/lib/boinc-client/slots/4/progress_file_12197114.xml Opened progress file ok : /var/lib/boinc-client/slots/4/progress_file_12197114.xml -- Model is restarting -- Adjusting last_iter, 461, to previous model restart step. Creating progress file: /var/lib/boinc-client/slots/4/progress_file_12197114.xml last_cpu_time: 13992 upload_file_number: 19 last_iter: 455 last_upload: 1641600 model_completed: 0 total_length_of_simulation: 10627200 result_base_name: oifs_43r3_ps_0470_2011050100_123_980_12197114_0_r1819580094 The child process has been launched with process id: 2380676 Executing the command: /var/lib/boinc-client/slots/4/oifs_43r3_model.exe [EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1 [EC_DRHOOK:p500:1:1:2380676:2380676] [20230116:005512:1673798112:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0 [EC_DRHOOK:p500:1:1:2380676:2380676] [20230116:005512:1673798112:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1 [EC_DRHOOK:p500:1:1:2380676:2380676] [20230116:005512:1673798112:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0) [EC_DRHOOK:p500:1:1:2380676:2380676] [20230116:005512:1673798112:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1 [EC_DRHOOK:p500:1:1:2380676:2380676] [20230116:005512:1673798112:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000 [EC_DRHOOK:p500:1:1:2380676:2380676] [20230116:005512:1673798112:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647) [EC_DRHOOK:p500:1:1:2380676:2380676] [20230116:005512:1673798112:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16 [EC_DRHOOK:p500:1:1:2380676:2380676] [20230116:005512:1673798112:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0 [EC_DRHOOK:p500:1:1:2380676:2380676] [20230116:005512:1673798112:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500 [EC_DRHOOK:p500:1:1:2380676:2380676] [20230116:005512:1673798112:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1 [EC_DRHOOK:p500:1:1:2380676:2380676] [20230116:005512:1673798112:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1 [EC_DRHOOK:p500:1:1:2380676:2380676] [20230116:005512:1673798112:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1 [EC_DRHOOK:p500:1:1:2380676:2380676] [20230116:005512:1673798112:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1 [EC_DRHOOK:p500:1:1:2380676:2380676] [20230116:005512:1673798112:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef> [EC_DRHOOK:p500:1:1:2380676:2380676] [20230116:005512:1673798112:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef> [EC_DRHOOK:p500:1:1:2380676:2380676] [20230116:005512:1673798112:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:p500:1:1:2380676:2380676] [20230116:005512:1673798112:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil)) [EC_DRHOOK:p500:1:1:2380676:2380676] [20230116:005512:1673798112:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:p500:1:1:2380676:2380676] [20230116:005512:1673798112:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil)) [EC_DRHOOK:p500:1:1:2380676:2380676] [20230116:005512:1673798112:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1 [EC_DRHOOK:p500:1:1:2380676:2380676] [20230116:005512:1673798112:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:p500:1:1:2380676:2380676] [20230116:005512:1673798112:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:p500:1:1:2380676:2380676] [20230116:005512:1673798112:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil)) [EC_DRHOOK:p500:1:1:2380676:2380676] [20230116:005512:1673798112:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:p500:1:1:2380676:2380676] [20230116:005512:1673798112:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:p500:1:1:2380676:2380676] [20230116:005512:1673798112:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:p500:1:1:2380676:2380676] [20230116:005512:1673798112:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil)) [EC_DRHOOK:p500:1:1:2380676:2380676] [20230116:005512:1673798112:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil)) [EC_DRHOOK:p500:1:1:2380676:2380676] [20230116:005512:1673798112:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil)) [EC_DRHOOK:p500:1:1:2380676:2380676] [20230116:005512:1673798112:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef> 00:55:50 STEP 456 H= 456:00 +CPU= 35.199 Suspend request received from the BOINC client, suspending the child process Resuming the child process 00:58:21 STEP 457 H= 457:00 +CPU= 19.663 Suspend request received from the BOINC client, suspending the child process Resuming the child process 01:00:22 STEP 458 H= 458:00 +CPU= 20.425 01:00:54 STEP 459 H= 459:00 +CPU= 31.446 Suspend request received from the BOINC client, suspending the child process Resuming the child process 01:03:36 STEP 460 H= 460:00 +CPU= 20.160 01:03:55 STEP 461 H= 461:00 +CPU= 19.299 Suspend request received from the BOINC client, suspending the child process Resuming the child process 01:06:29 STEP 462 H= 462:00 +CPU= 32.352 01:06:48 STEP 463 H= 463:00 +CPU= 19.361 01:07:09 STEP 464 H= 464:00 +CPU= 20.438 Suspend request received from the BOINC client, suspending the child process Resuming the child process 01:10:21 STEP 465 H= 465:00 +CPU= 32.242 01:10:41 STEP 466 H= 466:00 +CPU= 19.604 Suspend request received from the BOINC client, suspending the child process Resuming the child process 01:13:12 STEP 467 H= 467:00 +CPU= 19.743 Suspend request received from the BOINC client, suspending the child process Resuming the child process 01:14:58 STEP 468 H= 468:00 +CPU= 35.653 Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000468 Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000468 Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000468 01:15:18 STEP 469 H= 469:00 +CPU= 19.344 Suspend request received from the BOINC client, suspending the child process Resuming the child process 01:18:20 STEP 470 H= 470:00 +CPU= 20.583 01:18:51 STEP 471 H= 471:00 +CPU= 31.711 Suspend request received from the BOINC client, suspending the child process Resuming the child process 01:21:24 STEP 472 H= 472:00 +CPU= 20.772 01:21:44 STEP 473 H= 473:00 +CPU= 19.691 01:22:16 STEP 474 H= 474:00 +CPU= 32.247 Suspend request received from the BOINC client, suspending the child process Resuming the child process 01:24:57 STEP 475 H= 475:00 +CPU= 19.103 01:25:17 STEP 476 H= 476:00 +CPU= 20.401 Suspend request received from the BOINC client, suspending the child process Resuming the child process 01:27:50 STEP 477 H= 477:00 +CPU= 31.648 01:28:10 STEP 478 H= 478:00 +CPU= 19.784 01:28:30 STEP 479 H= 479:00 +CPU= 20.695 Suspend request received from the BOINC client, suspending the child process Resuming the child process 01:31:57 STEP 480 H= 480:00 +CPU= 36.691 Abort request received from BOINC client, ending the child process ..Failed, model did not complete successfully </stderr_txt> ]]> |
No trickles! |
---|
©2024 cpdn.org