climateprediction.net home page
Task 22270883

Task 22270883

Name oifs_43r3_ps_0906_1999050100_123_968_12185550_0
Workunit 12185550
Created 22 Dec 2022, 20:02:15 UTC
Sent 11 Jan 2023, 18:38:14 UTC
Report deadline 10 Feb 2023, 18:38:14 UTC
Received 12 Jan 2023, 23:10:54 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 196 (0x000000C4) EXIT_DISK_LIMIT_EXCEEDED
Computer ID 1519251
Run time 4 hours 43 min 14 sec
CPU time 4 hours 34 min 52 sec
Validate state Invalid
Credit 0.00
Device peak FLOPS 4.94 GFLOPS
Application version OpenIFS 43r3 Perturbed Surface v1.05
x86_64-pc-linux-gnu
Peak working set size 4,416.18 MB
Peak swap size 4,978.07 MB
Peak disk usage 7,802.60 MB
Stderr
<core_client_version>7.16.6</core_client_version>
<![CDATA[
<message>
Disk usage limit exceeded</message>
<stderr_txt>
boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185550/ICMUAhq0f+000336
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185550/ICMGGhq0f+000348
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185550/ICMSHhq0f+000348
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185550/ICMUAhq0f+000348
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0906_1999050100_123_968_12185550_0_r674103759_14.zip
Uploading the intermediate file: upload_file_14.zip
  00:53:47 STEP  361 H= 361:00 +CPU= 20.839
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:55:50 STEP  362 H= 362:00 +CPU= 18.968
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:57:34 STEP  363 H= 363:00 +CPU= 31.861
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:59:25 STEP  364 H= 364:00 +CPU= 21.008
  00:59:46 STEP  365 H= 365:00 +CPU= 20.381
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:03:22 STEP  366 H= 366:00 +CPU= 33.554
  01:03:41 STEP  367 H= 367:00 +CPU= 19.519
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:05:23 STEP  368 H= 368:00 +CPU= 20.521
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:07:17 STEP  369 H= 369:00 +CPU= 33.341
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:09:09 STEP  370 H= 370:00 +CPU= 20.926
  01:09:30 STEP  371 H= 371:00 +CPU= 20.791
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:11:27 STEP  372 H= 372:00 +CPU= 35.313
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+000372
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+000372
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+000372
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:13:19 STEP  373 H= 373:00 +CPU= 21.354
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:15:53 STEP  374 H= 374:00 +CPU= 22.062
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:17:29 STEP  375 H= 375:00 +CPU= 33.344
  01:17:50 STEP  376 H= 376:00 +CPU= 20.985
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:19:52 STEP  377 H= 377:00 +CPU= 21.047
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:21:57 STEP  378 H= 378:00 +CPU= 34.267
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 1999050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0906
(argv4) batchid: 968
(argv5) wuid: 12185550
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/3
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185550
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/3/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/3/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_f5738850212c523619ff94fd32207772 to: /var/lib/boinc-client/slots/3/oifs_43r3_ps_0906_1999050100_123_968_12185550.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/3/oifs_43r3_ps_0906_1999050100_123_968_12185550.zip
ic_ancil_file: ic_ancil_12185550
ifsdata_file: ifsdata_12185550
climate_data_file: clim_data_12185550
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_300ad98e3e727469f31e1eaa7b8b5640 to: /var/lib/boinc-client/slots/3/ic_ancil_12185550.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/3/ic_ancil_12185550.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_a56e8d6a7590f3f94f9bcd052b16959f to: /var/lib/boinc-client/slots/3/ifsdata/ifsdata_12185550.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/3/ifsdata/ifsdata_12185550.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/3/159l_2/clim_data_12185550.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/3/159l_2/clim_data_12185550.zip
Checking for progress XML file: /var/lib/boinc-client/slots/3/progress_file_12185550.xml
Opened progress file ok : /var/lib/boinc-client/slots/3/progress_file_12185550.xml
-- Model is restarting --
Adjusting last_iter, 379, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/3/progress_file_12185550.xml
last_cpu_time: 11647
upload_file_number: 15
last_iter: 359
last_upload: 1296000
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0906_1999050100_123_968_12185550_0_r674103759
The child process has been launched with process id: 1633732
Executing the command: /var/lib/boinc-client/slots/3/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:p500:1:1:1633732:1633732] [20230113:012523:1673540723:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:p500:1:1:1633732:1633732] [20230113:012523:1673540723:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:p500:1:1:1633732:1633732] [20230113:012523:1673540723:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:p500:1:1:1633732:1633732] [20230113:012523:1673540723:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:p500:1:1:1633732:1633732] [20230113:012523:1673540723:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:p500:1:1:1633732:1633732] [20230113:012523:1673540723:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:p500:1:1:1633732:1633732] [20230113:012523:1673540723:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:p500:1:1:1633732:1633732] [20230113:012523:1673540723:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:p500:1:1:1633732:1633732] [20230113:012523:1673540723:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:p500:1:1:1633732:1633732] [20230113:012523:1673540723:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:p500:1:1:1633732:1633732] [20230113:012523:1673540723:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:p500:1:1:1633732:1633732] [20230113:012523:1673540723:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:p500:1:1:1633732:1633732] [20230113:012523:1673540723:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:p500:1:1:1633732:1633732] [20230113:012523:1673540723:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:p500:1:1:1633732:1633732] [20230113:012523:1673540723:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:p500:1:1:1633732:1633732] [20230113:012523:1673540723:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1633732:1633732] [20230113:012523:1673540723:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1633732:1633732] [20230113:012523:1673540723:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1633732:1633732] [20230113:012523:1673540723:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1633732:1633732] [20230113:012523:1673540723:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:p500:1:1:1633732:1633732] [20230113:012523:1673540723:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1633732:1633732] [20230113:012523:1673540723:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1633732:1633732] [20230113:012523:1673540723:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1633732:1633732] [20230113:012523:1673540723:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1633732:1633732] [20230113:012523:1673540723:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1633732:1633732] [20230113:012523:1673540723:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1633732:1633732] [20230113:012523:1673540723:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1633732:1633732] [20230113:012523:1673540723:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1633732:1633732] [20230113:012523:1673540723:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1633732:1633732] [20230113:012523:1673540723:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:27:11 STEP  360 H= 360:00 +CPU= 36.698
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:29:03 STEP  361 H= 361:00 +CPU= 20.819
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:30:53 STEP  362 H= 362:00 +CPU= 20.683
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:32:58 STEP  363 H= 363:00 +CPU= 32.629
  01:33:19 STEP  364 H= 364:00 +CPU= 20.955
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:35:11 STEP  365 H= 365:00 +CPU= 20.979
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:37:04 STEP  366 H= 366:00 +CPU= 32.846
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:38:36 STEP  367 H= 367:00 +CPU= 20.697
  01:38:58 STEP  368 H= 368:00 +CPU= 21.553
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:40:42 STEP  369 H= 369:00 +CPU= 32.805
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:42:33 STEP  370 H= 370:00 +CPU= 20.549
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:44:14 STEP  371 H= 371:00 +CPU= 20.299
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:46:11 STEP  372 H= 372:00 +CPU= 35.482
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+000372
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+000372
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+000372
  01:46:32 STEP  373 H= 373:00 +CPU= 20.974
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:48:15 STEP  374 H= 374:00 +CPU= 21.687
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:50:09 STEP  375 H= 375:00 +CPU= 32.454
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:52:01 STEP  376 H= 376:00 +CPU= 21.461
  01:52:22 STEP  377 H= 377:00 +CPU= 20.575
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:55:47 STEP  378 H= 378:00 +CPU= 32.738
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:57:29 STEP  379 H= 379:00 +CPU= 20.917
  01:57:50 STEP  380 H= 380:00 +CPU= 20.491
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:59:54 STEP  381 H= 381:00 +CPU= 32.859
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:02:07 STEP  382 H= 382:00 +CPU= 20.940
  02:02:27 STEP  383 H= 383:00 +CPU= 20.583
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:05:37 STEP  384 H= 384:00 +CPU= 38.169
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+000384
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+000384
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+000384
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185550/ICMGGhq0f+000360
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185550/ICMSHhq0f+000360
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185550/ICMUAhq0f+000360
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185550/ICMGGhq0f+000372
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185550/ICMSHhq0f+000372
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185550/ICMUAhq0f+000372
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0906_1999050100_123_968_12185550_0_r674103759_15.zip
Uploading the intermediate file: upload_file_15.zip
  02:05:57 STEP  385 H= 385:00 +CPU= 20.212
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:07:50 STEP  386 H= 386:00 +CPU= 20.253
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:09:35 STEP  387 H= 387:00 +CPU= 32.667
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:10:58 STEP  388 H= 388:00 +CPU= 20.743
  02:11:19 STEP  389 H= 389:00 +CPU= 20.837
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:13:22 STEP  390 H= 390:00 +CPU= 33.039
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:15:14 STEP  391 H= 391:00 +CPU= 20.415
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:17:17 STEP  392 H= 392:00 +CPU= 21.850
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:19:20 STEP  393 H= 393:00 +CPU= 31.888
  02:19:41 STEP  394 H= 394:00 +CPU= 21.552
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:21:32 STEP  395 H= 395:00 +CPU= 20.656
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:23:10 STEP  396 H= 396:00 +CPU= 36.188
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+000396
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+000396
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+000396
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:25:04 STEP  397 H= 397:00 +CPU= 21.304
  02:25:25 STEP  398 H= 398:00 +CPU= 21.097
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:27:20 STEP  399 H= 399:00 +CPU= 32.445
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:29:12 STEP  400 H= 400:00 +CPU= 20.968
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:31:03 STEP  401 H= 401:00 +CPU= 20.347
  02:31:36 STEP  402 H= 402:00 +CPU= 32.593
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:33:57 STEP  403 H= 403:00 +CPU= 20.542
  02:34:20 STEP  404 H= 404:00 +CPU= 22.300
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:36:33 STEP  405 H= 405:00 +CPU= 32.217
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:38:15 STEP  406 H= 406:00 +CPU= 20.711
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:39:56 STEP  407 H= 407:00 +CPU= 20.410
  02:40:34 STEP  408 H= 408:00 +CPU= 37.946
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+000408
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+000408
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+000408
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185550/ICMGGhq0f+000384
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185550/ICMSHhq0f+000384
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185550/ICMUAhq0f+000384
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185550/ICMGGhq0f+000396
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185550/ICMSHhq0f+000396
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185550/ICMUAhq0f+000396
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0906_1999050100_123_968_12185550_0_r674103759_16.zip
Uploading the intermediate file: upload_file_16.zip
  02:40:52 STEP  409 H= 409:00 +CPU= 17.968
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:42:44 STEP  410 H= 410:00 +CPU= 19.956
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:44:48 STEP  411 H= 411:00 +CPU= 31.717
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:46:32 STEP  412 H= 412:00 +CPU= 22.201
  02:46:52 STEP  413 H= 413:00 +CPU= 20.304
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:48:46 STEP  414 H= 414:00 +CPU= 32.704
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:50:27 STEP  415 H= 415:00 +CPU= 20.219
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:52:10 STEP  416 H= 416:00 +CPU= 20.961
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:54:23 STEP  417 H= 417:00 +CPU= 31.932
  02:54:43 STEP  418 H= 418:00 +CPU= 20.174
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:56:33 STEP  419 H= 419:00 +CPU= 19.755
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:58:31 STEP  420 H= 420:00 +CPU= 35.936
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+000420
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+000420
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+000420
  02:58:51 STEP  421 H= 421:00 +CPU= 20.397
  02:59:12 STEP  422 H= 422:00 +CPU= 20.340
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  03:01:23 STEP  423 H= 423:00 +CPU= 31.207
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  03:03:35 STEP  424 H= 424:00 +CPU= 20.392
  03:03:55 STEP  425 H= 425:00 +CPU= 20.654
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  03:05:49 STEP  426 H= 426:00 +CPU= 32.549
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  03:08:10 STEP  427 H= 427:00 +CPU= 20.617
  03:08:31 STEP  428 H= 428:00 +CPU= 21.018
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  03:10:34 STEP  429 H= 429:00 +CPU= 31.673
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  03:12:57 STEP  430 H= 430:00 +CPU= 20.906
  03:13:19 STEP  431 H= 431:00 +CPU= 22.225
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  03:15:19 STEP  432 H= 432:00 +CPU= 38.524
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+000432
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+000432
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+000432
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185550/ICMGGhq0f+000408
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185550/ICMSHhq0f+000408
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185550/ICMUAhq0f+000408
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185550/ICMGGhq0f+000420
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185550/ICMSHhq0f+000420
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185550/ICMUAhq0f+000420
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0906_1999050100_123_968_12185550_0_r674103759_17.zip
Uploading the intermediate file: upload_file_17.zip
  03:15:38 STEP  433 H= 433:00 +CPU= 19.741
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  03:17:14 STEP  434 H= 434:00 +CPU= 19.819
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  03:19:28 STEP  435 H= 435:00 +CPU= 32.762
  03:19:50 STEP  436 H= 436:00 +CPU= 21.677
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  03:21:21 STEP  437 H= 437:00 +CPU= 20.237
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  03:23:17 STEP  438 H= 438:00 +CPU= 33.749
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  03:24:29 STEP  439 H= 439:00 +CPU= 20.011
  03:24:51 STEP  440 H= 440:00 +CPU= 21.108
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Quit request received from BOINC client, ending the child process
..Failed, model did not complete successfully
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 1999050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0906
(argv4) batchid: 968
(argv5) wuid: 12185550
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/3
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185550
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/3/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/3/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_f5738850212c523619ff94fd32207772 to: /var/lib/boinc-client/slots/3/oifs_43r3_ps_0906_1999050100_123_968_12185550.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/3/oifs_43r3_ps_0906_1999050100_123_968_12185550.zip
ic_ancil_file: ic_ancil_12185550
ifsdata_file: ifsdata_12185550
climate_data_file: clim_data_12185550
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_300ad98e3e727469f31e1eaa7b8b5640 to: /var/lib/boinc-client/slots/3/ic_ancil_12185550.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/3/ic_ancil_12185550.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_a56e8d6a7590f3f94f9bcd052b16959f to: /var/lib/boinc-client/slots/3/ifsdata/ifsdata_12185550.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/3/ifsdata/ifsdata_12185550.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/3/159l_2/clim_data_12185550.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/3/159l_2/clim_data_12185550.zip
Checking for progress XML file: /var/lib/boinc-client/slots/3/progress_file_12185550.xml
Opened progress file ok : /var/lib/boinc-client/slots/3/progress_file_12185550.xml
-- Model is restarting --
Adjusting last_iter, 441, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/3/progress_file_12185550.xml
last_cpu_time: 13686
upload_file_number: 18
last_iter: 431
last_upload: 1555200
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0906_1999050100_123_968_12185550_0_r674103759
The child process has been launched with process id: 1651365
Executing the command: /var/lib/boinc-client/slots/3/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:p500:1:1:1651365:1651365] [20230113:032828:1673548108:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:p500:1:1:1651365:1651365] [20230113:032828:1673548108:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:p500:1:1:1651365:1651365] [20230113:032828:1673548108:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:p500:1:1:1651365:1651365] [20230113:032828:1673548108:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:p500:1:1:1651365:1651365] [20230113:032828:1673548108:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:p500:1:1:1651365:1651365] [20230113:032828:1673548108:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:p500:1:1:1651365:1651365] [20230113:032828:1673548108:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:p500:1:1:1651365:1651365] [20230113:032828:1673548108:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:p500:1:1:1651365:1651365] [20230113:032828:1673548108:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:p500:1:1:1651365:1651365] [20230113:032828:1673548108:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:p500:1:1:1651365:1651365] [20230113:032828:1673548108:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:p500:1:1:1651365:1651365] [20230113:032828:1673548108:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:p500:1:1:1651365:1651365] [20230113:032828:1673548108:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:p500:1:1:1651365:1651365] [20230113:032828:1673548108:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:p500:1:1:1651365:1651365] [20230113:032828:1673548108:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:p500:1:1:1651365:1651365] [20230113:032828:1673548108:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1651365:1651365] [20230113:032828:1673548108:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1651365:1651365] [20230113:032828:1673548108:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1651365:1651365] [20230113:032828:1673548108:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1651365:1651365] [20230113:032828:1673548108:0.001] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:p500:1:1:1651365:1651365] [20230113:032828:1673548108:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1651365:1651365] [20230113:032828:1673548108:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1651365:1651365] [20230113:032828:1673548108:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1651365:1651365] [20230113:032828:1673548108:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1651365:1651365] [20230113:032828:1673548108:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1651365:1651365] [20230113:032828:1673548108:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1651365:1651365] [20230113:032828:1673548108:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1651365:1651365] [20230113:032828:1673548108:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1651365:1651365] [20230113:032828:1673548108:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1651365:1651365] [20230113:032828:1673548108:0.001] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  03:30:39 STEP  432 H= 432:00 +CPU= 37.163
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  03:32:11 STEP  433 H= 433:00 +CPU= 20.394
  03:32:32 STEP  434 H= 434:00 +CPU= 21.223
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  03:34:05 STEP  435 H= 435:00 +CPU= 32.144
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  03:35:36 STEP  436 H= 436:00 +CPU= 21.139
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  03:37:07 STEP  437 H= 437:00 +CPU= 20.546
  03:37:41 STEP  438 H= 438:00 +CPU= 33.326
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  03:39:32 STEP  439 H= 439:00 +CPU= 20.096
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  03:41:14 STEP  440 H= 440:00 +CPU= 20.896
  03:41:46 STEP  441 H= 441:00 +CPU= 32.577
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  03:43:38 STEP  442 H= 442:00 +CPU= 20.524
  03:43:59 STEP  443 H= 443:00 +CPU= 20.489
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  03:46:06 STEP  444 H= 444:00 +CPU= 36.504
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+000444
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+000444
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+000444
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  03:47:58 STEP  445 H= 445:00 +CPU= 20.614
  03:48:19 STEP  446 H= 446:00 +CPU= 21.362
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  03:50:14 STEP  447 H= 447:00 +CPU= 33.015
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  03:51:46 STEP  448 H= 448:00 +CPU= 21.460
  03:52:07 STEP  449 H= 449:00 +CPU= 20.538
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  03:54:41 STEP  450 H= 450:00 +CPU= 33.462
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  03:56:44 STEP  451 H= 451:00 +CPU= 21.002
  03:57:05 STEP  452 H= 452:00 +CPU= 21.341
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  03:58:59 STEP  453 H= 453:00 +CPU= 32.653
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:00:40 STEP  454 H= 454:00 +CPU= 20.794
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:02:22 STEP  455 H= 455:00 +CPU= 20.983
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:04:21 STEP  456 H= 456:00 +CPU= 38.280
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+000456
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+000456
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+000456
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185550/ICMGGhq0f+000432
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185550/ICMSHhq0f+000432
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185550/ICMUAhq0f+000432
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185550/ICMGGhq0f+000444
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185550/ICMSHhq0f+000444
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185550/ICMUAhq0f+000444
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0906_1999050100_123_968_12185550_0_r674103759_18.zip
Uploading the intermediate file: upload_file_18.zip
  04:04:42 STEP  457 H= 457:00 +CPU= 20.233
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:06:44 STEP  458 H= 458:00 +CPU= 20.900
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:08:49 STEP  459 H= 459:00 +CPU= 32.960
  04:09:10 STEP  460 H= 460:00 +CPU= 21.602
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:11:12 STEP  461 H= 461:00 +CPU= 20.411
  04:11:45 STEP  462 H= 462:00 +CPU= 32.863
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:13:27 STEP  463 H= 463:00 +CPU= 20.987
  04:13:48 STEP  464 H= 464:00 +CPU= 21.292
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:15:52 STEP  465 H= 465:00 +CPU= 32.705
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:17:44 STEP  466 H= 466:00 +CPU= 20.934
  04:18:05 STEP  467 H= 467:00 +CPU= 20.861
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:20:22 STEP  468 H= 468:00 +CPU= 35.831
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+000468
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+000468
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+000468
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:21:54 STEP  469 H= 469:00 +CPU= 20.728
  04:22:15 STEP  470 H= 470:00 +CPU= 21.092
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:24:21 STEP  471 H= 471:00 +CPU= 33.639
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:25:55 STEP  472 H= 472:00 +CPU= 21.783
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:27:26 STEP  473 H= 473:00 +CPU= 20.167
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:29:10 STEP  474 H= 474:00 +CPU= 32.754
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:30:32 STEP  475 H= 475:00 +CPU= 19.815
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:32:05 STEP  476 H= 476:00 +CPU= 20.809
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:33:39 STEP  477 H= 477:00 +CPU= 32.376
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:35:20 STEP  478 H= 478:00 +CPU= 20.107
  04:35:41 STEP  479 H= 479:00 +CPU= 21.458
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:38:01 STEP  480 H= 480:00 +CPU= 38.882
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+000480
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+000480
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+000480
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185550/ICMGGhq0f+000456
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185550/ICMSHhq0f+000456
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185550/ICMUAhq0f+000456
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185550/ICMGGhq0f+000468
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185550/ICMSHhq0f+000468
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185550/ICMUAhq0f+000468
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0906_1999050100_123_968_12185550_0_r674103759_19.zip
Uploading the intermediate file: upload_file_19.zip
  04:38:22 STEP  481 H= 481:00 +CPU= 20.440
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:40:54 STEP  482 H= 482:00 +CPU= 20.140
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:43:00 STEP  483 H= 483:00 +CPU= 33.923
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:47:04 STEP  484 H= 484:00 +CPU= 22.496
  04:47:25 STEP  485 H= 485:00 +CPU= 20.629
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:49:52 STEP  486 H= 486:00 +CPU= 34.410
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:51:36 STEP  487 H= 487:00 +CPU= 21.634
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:54:09 STEP  488 H= 488:00 +CPU= 21.525
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:56:55 STEP  489 H= 489:00 +CPU= 34.519
  04:57:17 STEP  490 H= 490:00 +CPU= 22.157
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:59:50 STEP  491 H= 491:00 +CPU= 21.175
  05:00:27 STEP  492 H= 492:00 +CPU= 36.937
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+000492
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+000492
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+000492
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:03:00 STEP  493 H= 493:00 +CPU= 21.046
  05:03:22 STEP  494 H= 494:00 +CPU= 21.606
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:06:28 STEP  495 H= 495:00 +CPU= 33.578
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:09:10 STEP  496 H= 496:00 +CPU= 21.144
  05:09:32 STEP  497 H= 497:00 +CPU= 21.616
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:12:29 STEP  498 H= 498:00 +CPU= 34.829
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:15:03 STEP  499 H= 499:00 +CPU= 20.925
  05:15:25 STEP  500 H= 500:00 +CPU= 21.985
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:20:23 STEP  501 H= 501:00 +CPU= 34.702
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:22:27 STEP  502 H= 502:00 +CPU= 22.578
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:24:31 STEP  503 H= 503:00 +CPU= 22.506
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:26:51 STEP  504 H= 504:00 +CPU= 39.252
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+000504
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+000504
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+000504
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185550/ICMGGhq0f+000480
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185550/ICMSHhq0f+000480
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185550/ICMUAhq0f+000480
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185550/ICMGGhq0f+000492
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185550/ICMSHhq0f+000492
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185550/ICMUAhq0f+000492
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0906_1999050100_123_968_12185550_0_r674103759_20.zip
Uploading the intermediate file: upload_file_20.zip
  05:27:12 STEP  505 H= 505:00 +CPU= 20.096
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 1999050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0906
(argv4) batchid: 968
(argv5) wuid: 12185550
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/3
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185550
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/3/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/3/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_f5738850212c523619ff94fd32207772 to: /var/lib/boinc-client/slots/3/oifs_43r3_ps_0906_1999050100_123_968_12185550.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/3/oifs_43r3_ps_0906_1999050100_123_968_12185550.zip
ic_ancil_file: ic_ancil_12185550
ifsdata_file: ifsdata_12185550
climate_data_file: clim_data_12185550
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_300ad98e3e727469f31e1eaa7b8b5640 to: /var/lib/boinc-client/slots/3/ic_ancil_12185550.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/3/ic_ancil_12185550.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_a56e8d6a7590f3f94f9bcd052b16959f to: /var/lib/boinc-client/slots/3/ifsdata/ifsdata_12185550.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/3/ifsdata/ifsdata_12185550.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/3/159l_2/clim_data_12185550.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/3/159l_2/clim_data_12185550.zip
Checking for progress XML file: /var/lib/boinc-client/slots/3/progress_file_12185550.xml
Opened progress file ok : /var/lib/boinc-client/slots/3/progress_file_12185550.xml
-- Model is restarting --
Adjusting last_iter, 506, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/3/progress_file_12185550.xml
last_cpu_time: 15600
upload_file_number: 21
last_iter: 503
last_upload: 1814400
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0906_1999050100_123_968_12185550_0_r674103759
The child process has been launched with process id: 1672907
Executing the command: /var/lib/boinc-client/slots/3/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:p500:1:1:1672907:1672907] [20230113:053112:1673555472:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:p500:1:1:1672907:1672907] [20230113:053112:1673555472:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:p500:1:1:1672907:1672907] [20230113:053112:1673555472:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:p500:1:1:1672907:1672907] [20230113:053112:1673555472:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:p500:1:1:1672907:1672907] [20230113:053112:1673555472:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:p500:1:1:1672907:1672907] [20230113:053112:1673555472:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:p500:1:1:1672907:1672907] [20230113:053112:1673555472:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:p500:1:1:1672907:1672907] [20230113:053112:1673555472:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:p500:1:1:1672907:1672907] [20230113:053112:1673555472:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:p500:1:1:1672907:1672907] [20230113:053112:1673555472:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:p500:1:1:1672907:1672907] [20230113:053112:1673555472:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:p500:1:1:1672907:1672907] [20230113:053112:1673555472:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:p500:1:1:1672907:1672907] [20230113:053112:1673555472:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:p500:1:1:1672907:1672907] [20230113:053112:1673555472:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:p500:1:1:1672907:1672907] [20230113:053112:1673555472:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:p500:1:1:1672907:1672907] [20230113:053112:1673555472:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1672907:1672907] [20230113:053112:1673555472:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1672907:1672907] [20230113:053112:1673555472:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1672907:1672907] [20230113:053112:1673555472:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1672907:1672907] [20230113:053112:1673555472:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:p500:1:1:1672907:1672907] [20230113:053112:1673555472:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1672907:1672907] [20230113:053112:1673555472:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1672907:1672907] [20230113:053112:1673555472:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1672907:1672907] [20230113:053112:1673555472:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1672907:1672907] [20230113:053112:1673555472:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1672907:1672907] [20230113:053112:1673555472:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:1672907:1672907] [20230113:053112:1673555472:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1672907:1672907] [20230113:053112:1673555472:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1672907:1672907] [20230113:053112:1673555472:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:1672907:1672907] [20230113:053112:1673555472:0.001] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:33:12 STEP  504 H= 504:00 +CPU= 37.800
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:34:43 STEP  505 H= 505:00 +CPU= 20.645
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:36:26 STEP  506 H= 506:00 +CPU= 21.363
  05:36:59 STEP  507 H= 507:00 +CPU= 32.635
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:39:02 STEP  508 H= 508:00 +CPU= 21.332
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:41:25 STEP  509 H= 509:00 +CPU= 21.365
  05:41:59 STEP  510 H= 510:00 +CPU= 33.491
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:44:21 STEP  511 H= 511:00 +CPU= 21.097
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:46:13 STEP  512 H= 512:00 +CPU= 21.715
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:48:09 STEP  513 H= 513:00 +CPU= 33.193
  05:48:30 STEP  514 H= 514:00 +CPU= 21.091
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:50:12 STEP  515 H= 515:00 +CPU= 20.954
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:53:12 STEP  516 H= 516:00 +CPU= 36.113
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+000516
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+000516
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+000516
  05:53:33 STEP  517 H= 517:00 +CPU= 20.538
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:55:25 STEP  518 H= 518:00 +CPU= 21.414
  05:55:57 STEP  519 H= 519:00 +CPU= 31.798
  05:56:18 STEP  520 H= 520:00 +CPU= 20.919
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:58:11 STEP  521 H= 521:00 +CPU= 21.143
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:59:56 STEP  522 H= 522:00 +CPU= 34.004
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:01:48 STEP  523 H= 523:00 +CPU= 20.849
  06:02:09 STEP  524 H= 524:00 +CPU= 21.491
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:04:12 STEP  525 H= 525:00 +CPU= 32.287
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:05:34 STEP  526 H= 526:00 +CPU= 21.530
  06:05:57 STEP  527 H= 527:00 +CPU= 22.020
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:07:56 STEP  528 H= 528:00 +CPU= 38.334
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+000528
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+000528
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+000528
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185550/ICMGGhq0f+000504
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185550/ICMSHhq0f+000504
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185550/ICMUAhq0f+000504
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185550/ICMGGhq0f+000516
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185550/ICMSHhq0f+000516
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12185550/ICMUAhq0f+000516
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0906_1999050100_123_968_12185550_0_r674103759_21.zip
Uploading the intermediate file: upload_file_21.zip
  06:09:48 STEP  529 H= 529:00 +CPU= 20.840
  06:10:10 STEP  530 H= 530:00 +CPU= 21.448
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:12:13 STEP  531 H= 531:00 +CPU= 32.291
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:14:15 STEP  532 H= 532:00 +CPU= 20.981
  06:14:36 STEP  533 H= 533:00 +CPU= 21.006
  06:15:09 STEP  534 H= 534:00 +CPU= 32.931
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:17:02 STEP  535 H= 535:00 +CPU= 20.755
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:19:05 STEP  536 H= 536:00 +CPU= 21.571
  06:19:38 STEP  537 H= 537:00 +CPU= 33.301
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Abort request received from BOINC client, ending the child process
..Failed, model did not complete successfully

</stderr_txt>
]]>
No trickles!


©2024 cpdn.org