climateprediction.net home page
Task 22442231

Task 22442231

Name oifs_43r3_bl_a09h_2016092300_20_1018_12288668_0
Workunit 12288668
Created 12 Jun 2024, 20:28:13 UTC
Sent 12 Jun 2024, 20:54:24 UTC
Report deadline 11 Aug 2024, 20:54:24 UTC
Received 11 Jul 2024, 18:49:30 UTC
Server state Over
Outcome Success
Client state Done
Exit status 0 (0x00000000)
Computer ID 1541083
Run time 2 days 3 hours 27 min 5 sec
CPU time 2 days 3 hours 27 min 5 sec
Validate state Valid
Credit 2,635.55
Device peak FLOPS 2.64 GFLOPS
Application version OpenIFS 43r3 Baroclinic Lifecycle v1.13
x86_64-pc-linux-gnu
Peak working set size 5,023.25 MB
Peak swap size 5,981.08 MB
Peak disk usage 2,234.25 MB
Stderr
<core_client_version>8.0.2</core_client_version>
<![CDATA[
<stderr_txt>
 H= 395:00 +CPU= 92.312
  05:56:33 STEP 1581 H= 395:15 +CPU=103.278
  05:57:54 STEP 1582 H= 395:30 +CPU= 79.119
  05:59:08 STEP 1583 H= 395:45 +CPU= 74.177
  06:01:27 STEP 1584 H= 396:00 +CPU=133.888
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+001584
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+001584
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+001584
  06:03:06 STEP 1585 H= 396:15 +CPU= 98.516
  06:04:35 STEP 1586 H= 396:30 +CPU= 88.201
  06:06:16 STEP 1587 H= 396:45 +CPU= 97.619
  06:07:47 STEP 1588 H= 397:00 +CPU= 89.689
  06:09:25 STEP 1589 H= 397:15 +CPU= 95.971
  06:10:34 STEP 1590 H= 397:30 +CPU= 68.225
  06:11:49 STEP 1591 H= 397:45 +CPU= 74.520
  06:13:40 STEP 1592 H= 398:00 +CPU=108.170
  06:15:23 STEP 1593 H= 398:15 +CPU=101.473
  06:16:48 STEP 1594 H= 398:30 +CPU= 85.266
  06:17:50 STEP 1595 H= 398:45 +CPU= 61.244
  06:19:52 STEP 1596 H= 399:00 +CPU=121.029
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+001596
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+001596
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+001596
  06:21:26 STEP 1597 H= 399:15 +CPU= 92.665
  06:22:49 STEP 1598 H= 399:30 +CPU= 81.594
  06:24:03 STEP 1599 H= 399:45 +CPU= 73.876
  06:26:20 STEP 1600 H= 400:00 +CPU=131.647
  06:27:38 STEP 1601 H= 400:15 +CPU= 77.203
  06:28:57 STEP 1602 H= 400:30 +CPU= 78.823
  06:30:09 STEP 1603 H= 400:45 +CPU= 71.425
  06:31:49 STEP 1604 H= 401:00 +CPU= 97.185
  06:33:18 STEP 1605 H= 401:15 +CPU= 87.621
  06:34:28 STEP 1606 H= 401:30 +CPU= 68.998
  06:35:59 STEP 1607 H= 401:45 +CPU= 90.332
  06:38:03 STEP 1608 H= 402:00 +CPU=121.904
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+001608
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+001608
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+001608
  06:39:34 STEP 1609 H= 402:15 +CPU= 90.155
  06:40:45 STEP 1610 H= 402:30 +CPU= 70.860
  06:42:31 STEP 1611 H= 402:45 +CPU=101.894
  06:44:32 STEP 1612 H= 403:00 +CPU=120.028
  06:46:20 STEP 1613 H= 403:15 +CPU=106.619
  06:48:06 STEP 1614 H= 403:30 +CPU=103.350
  06:50:00 STEP 1615 H= 403:45 +CPU=112.652
  06:51:40 STEP 1616 H= 404:00 +CPU= 98.872
  06:53:28 STEP 1617 H= 404:15 +CPU=106.813
  06:55:20 STEP 1618 H= 404:30 +CPU=109.616
  06:57:16 STEP 1619 H= 404:45 +CPU=114.756
  06:59:16 STEP 1620 H= 405:00 +CPU=118.939
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+001620
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+001620
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+001620
  07:01:09 STEP 1621 H= 405:15 +CPU=111.230
  07:02:52 STEP 1622 H= 405:30 +CPU=102.607
  07:04:50 STEP 1623 H= 405:45 +CPU=116.239
  07:06:26 STEP 1624 H= 406:00 +CPU= 94.606
  07:08:29 STEP 1625 H= 406:15 +CPU=119.885
  07:10:17 STEP 1626 H= 406:30 +CPU=107.526
  07:12:02 STEP 1627 H= 406:45 +CPU=104.094
  07:13:32 STEP 1628 H= 407:00 +CPU= 88.925
  07:15:41 STEP 1629 H= 407:15 +CPU=128.123
  07:17:16 STEP 1630 H= 407:30 +CPU= 94.104
  07:18:51 STEP 1631 H= 407:45 +CPU= 93.610
  07:20:38 STEP 1632 H= 408:00 +CPU=104.488
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+001632
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+001632
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+001632
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMGGh7zg+001536
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMSHh7zg+001536
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMUAh7zg+001536
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMGGh7zg+001548
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMSHh7zg+001548
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMUAh7zg+001548
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMGGh7zg+001560
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMSHh7zg+001560
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMUAh7zg+001560
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMGGh7zg+001572
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMSHh7zg+001572
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMUAh7zg+001572
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMGGh7zg+001584
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMSHh7zg+001584
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMUAh7zg+001584
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMGGh7zg+001596
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMSHh7zg+001596
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMUAh7zg+001596
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMGGh7zg+001608
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMSHh7zg+001608
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMUAh7zg+001608
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMGGh7zg+001620
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMSHh7zg+001620
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMUAh7zg+001620
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a09h_2016092300_20_1018_12288668_0_r1508782734_16.zip
Uploading the intermediate file: upload_file_16.zip
  07:22:20 STEP 1633 H= 408:15 +CPU= 99.026
  07:23:45 STEP 1634 H= 408:30 +CPU= 84.120
  07:25:05 STEP 1635 H= 408:45 +CPU= 79.029
  07:26:30 STEP 1636 H= 409:00 +CPU= 83.522
  07:28:04 STEP 1637 H= 409:15 +CPU= 90.980
  07:29:37 STEP 1638 H= 409:30 +CPU= 91.696
  07:31:07 STEP 1639 H= 409:45 +CPU= 88.536
  07:32:21 STEP 1640 H= 410:00 +CPU= 72.529
  07:33:45 STEP 1641 H= 410:15 +CPU= 81.379
  07:35:21 STEP 1642 H= 410:30 +CPU= 94.742
  07:36:58 STEP 1643 H= 410:45 +CPU= 96.376
  07:38:32 STEP 1644 H= 411:00 +CPU= 93.334
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+001644
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+001644
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+001644
  07:39:48 STEP 1645 H= 411:15 +CPU= 75.245
  07:41:11 STEP 1646 H= 411:30 +CPU= 80.556
  07:42:47 STEP 1647 H= 411:45 +CPU= 94.453
  07:44:13 STEP 1648 H= 412:00 +CPU= 86.007
  07:45:28 STEP 1649 H= 412:15 +CPU= 74.052
  07:46:47 STEP 1650 H= 412:30 +CPU= 78.684
  07:48:19 STEP 1651 H= 412:45 +CPU= 89.151
  07:50:00 STEP 1652 H= 413:00 +CPU= 99.961
  07:51:16 STEP 1653 H= 413:15 +CPU= 74.405
  07:52:37 STEP 1654 H= 413:30 +CPU= 80.713
  07:53:51 STEP 1655 H= 413:45 +CPU= 73.282
  07:55:57 STEP 1656 H= 414:00 +CPU=123.841
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+001656
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+001656
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+001656
  07:57:25 STEP 1657 H= 414:15 +CPU= 86.585
  07:58:40 STEP 1658 H= 414:30 +CPU= 74.048
(argv0) ../../projects/climateprediction.net/oifs_43r3_bl_1.13_x86_64-pc-linux-gnu
(argv1) start_date: 2016092300
(argv2) exptid: h7zg
(argv3) unique_member_id: a09h
(argv4) batchid: 1018
(argv5) wuid: 12288668
(argv6) fclen: 20
(argv7) app_name: oifs_43r3_bl
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/8
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_bl
version: 1.13
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/8/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/8/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_eb1b6c1e46b8b06039c8c9121c2685b0 to: /var/lib/boinc-client/slots/8/oifs_43r3_bl_a09h_2016092300_20_1018_12288668.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/8/oifs_43r3_bl_a09h_2016092300_20_1018_12288668.zip
ic_ancil_file: ic_ancil_12288668
ifsdata_file: ifsdata_12288668
climate_data_file: clim_data_12288668
horiz_resolution: 159
vert_resolution: 91
grid_type: l_2
upload_interval: 96
utstep: 900
nfrres: restart dump frequency (steps) 48
Copying IC ancils from: ../../projects/climateprediction.net/jf_f7ecfd27aa1e68bbe7e8d19800e25c73 to: /var/lib/boinc-client/slots/8/ic_ancil_12288668.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/8/ic_ancil_12288668.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_265951956d1fd0c788a37ed9a90fce61 to: /var/lib/boinc-client/slots/8/ifsdata/ifsdata_12288668.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/8/ifsdata/ifsdata_12288668.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/8/159l_2/clim_data_12288668.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/8/159l_2/clim_data_12288668.zip
Checking for progress XML file: /var/lib/boinc-client/slots/8/progress_file_12288668.xml
Opened progress file ok : /var/lib/boinc-client/slots/8/progress_file_12288668.xml
-- Model is restarting --
Adjusting last_iter, 1659, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/8/progress_file_12288668.xml
last_cpu_time: 159524
upload_file_number: 17
last_iter: 1633
last_upload: 1468800
model_completed: 0
total_length_of_simulation: 1728000
result_base_name: oifs_43r3_bl_a09h_2016092300_20_1018_12288668_0_r1508782734
The child process has been launched with process id: 1957
Executing the command: /var/lib/boinc-client/slots/8/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:hpc1:1:1:1957:1957] [20240711:223044:1720693844:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2091] fp = 0x314f440
[EC_DRHOOK:hpc1:1:1:1957:1957] [20240711:223044:1720693844:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:hpc1:1:1:1957:1957] [20240711:223044:1720693844:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:hpc1:1:1:1957:1957] [20240711:223044:1720693844:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:hpc1:1:1:1957:1957] [20240711:223044:1720693844:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:hpc1:1:1:1957:1957] [20240711:223044:1720693844:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:hpc1:1:1:1957:1957] [20240711:223044:1720693844:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:hpc1:1:1:1957:1957] [20240711:223044:1720693844:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:hpc1:1:1:1957:1957] [20240711:223044:1720693844:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:hpc1:1:1:1957:1957] [20240711:223044:1720693844:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:hpc1:1:1:1957:1957] [20240711:223044:1720693844:0.001] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:hpc1:1:1:1957:1957] [20240711:223044:1720693844:0.001] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:hpc1:1:1:1957:1957] [20240711:223044:1720693844:0.001] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:hpc1:1:1:1957:1957] [20240711:223044:1720693844:0.001] [ignore_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:hpc1:1:1:1957:1957] [20240711:223044:1720693844:0.001] [restore_default_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:hpc1:1:1:1957:1957] [20240711:223044:1720693844:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:hpc1:1:1:1957:1957] [20240711:223044:1720693844:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:hpc1:1:1:1957:1957] [20240711:223044:1720693844:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:hpc1:1:1:1957:1957] [20240711:223044:1720693844:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x820cf0 (old at (nil))
[EC_DRHOOK:hpc1:1:1:1957:1957] [20240711:223044:1720693844:0.001] [trapfpe_treatment@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:hpc1:1:1:1957:1957] [20240711:223044:1720693844:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:hpc1:1:1:1957:1957] [20240711:223044:1720693844:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:hpc1:1:1:1957:1957] [20240711:223044:1720693844:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x820cf0 (old at (nil))
[EC_DRHOOK:hpc1:1:1:1957:1957] [20240711:223044:1720693844:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:hpc1:1:1:1957:1957] [20240711:223044:1720693844:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:hpc1:1:1:1957:1957] [20240711:223044:1720693844:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:hpc1:1:1:1957:1957] [20240711:223044:1720693844:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x820cf0 (old at (nil))
[EC_DRHOOK:hpc1:1:1:1957:1957] [20240711:223044:1720693844:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x820cf0 (old at (nil))
[EC_DRHOOK:hpc1:1:1:1957:1957] [20240711:223044:1720693844:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:hpc1:1:1:1957:1957] [20240711:223044:1720693844:0.001] [catch_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  22:33:19 STEP 1632 H= 408:00 +CPU=129.047
  22:35:50 STEP 1633 H= 408:15 +CPU=145.660
  22:38:03 STEP 1634 H= 408:30 +CPU=129.931
  22:40:21 STEP 1635 H= 408:45 +CPU=128.680
  22:42:31 STEP 1636 H= 409:00 +CPU=100.122
  22:44:27 STEP 1637 H= 409:15 +CPU=111.573
  22:45:57 STEP 1638 H= 409:30 +CPU= 87.718
  22:47:32 STEP 1639 H= 409:45 +CPU= 91.143
  22:49:12 STEP 1640 H= 410:00 +CPU= 97.331
  22:50:41 STEP 1641 H= 410:15 +CPU= 87.414
  22:52:13 STEP 1642 H= 410:30 +CPU= 90.482
  22:53:25 STEP 1643 H= 410:45 +CPU= 70.962
  22:55:05 STEP 1644 H= 411:00 +CPU= 99.026
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+001644
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+001644
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+001644
  22:56:20 STEP 1645 H= 411:15 +CPU= 73.282
  22:57:40 STEP 1646 H= 411:30 +CPU= 78.495
  22:58:46 STEP 1647 H= 411:45 +CPU= 64.998
  23:00:05 STEP 1648 H= 412:00 +CPU= 77.933
  23:01:22 STEP 1649 H= 412:15 +CPU= 74.722
  23:02:50 STEP 1650 H= 412:30 +CPU= 87.369
  23:04:02 STEP 1651 H= 412:45 +CPU= 70.708
  23:05:22 STEP 1652 H= 413:00 +CPU= 78.660
  23:06:58 STEP 1653 H= 413:15 +CPU= 94.411
  23:08:29 STEP 1654 H= 413:30 +CPU= 89.455
  23:09:42 STEP 1655 H= 413:45 +CPU= 72.111
  23:11:39 STEP 1656 H= 414:00 +CPU=115.315
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+001656
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+001656
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+001656
  23:13:00 STEP 1657 H= 414:15 +CPU= 79.699
  23:14:22 STEP 1658 H= 414:30 +CPU= 82.125
  23:15:32 STEP 1659 H= 414:45 +CPU= 69.491
  23:17:02 STEP 1660 H= 415:00 +CPU= 89.200
  23:18:40 STEP 1661 H= 415:15 +CPU= 96.842
  23:20:30 STEP 1662 H= 415:30 +CPU=108.576
  23:22:22 STEP 1663 H= 415:45 +CPU=111.945
  23:24:34 STEP 1664 H= 416:00 +CPU=128.983
  23:26:41 STEP 1665 H= 416:15 +CPU=125.714
  23:28:28 STEP 1666 H= 416:30 +CPU=105.342
  23:30:27 STEP 1667 H= 416:45 +CPU=117.681
  23:32:46 STEP 1668 H= 417:00 +CPU=138.080
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+001668
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+001668
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+001668
  23:34:24 STEP 1669 H= 417:15 +CPU= 96.086
  23:36:50 STEP 1670 H= 417:30 +CPU=138.964
  23:38:16 STEP 1671 H= 417:45 +CPU= 83.574
  23:40:08 STEP 1672 H= 418:00 +CPU=108.733
  23:41:55 STEP 1673 H= 418:15 +CPU=106.220
  23:43:21 STEP 1674 H= 418:30 +CPU= 85.162
  23:44:44 STEP 1675 H= 418:45 +CPU= 82.331
  23:46:35 STEP 1676 H= 419:00 +CPU=109.115
  23:48:18 STEP 1677 H= 419:15 +CPU=103.011
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:50:08 STEP 1678 H= 419:30 +CPU= 87.711
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:51:50 STEP 1679 H= 419:45 +CPU= 79.036
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:55:24 STEP 1680 H= 420:00 +CPU=148.665
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+001680
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+001680
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+001680
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:57:26 STEP 1681 H= 420:15 +CPU= 97.940
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:59:38 STEP 1682 H= 420:30 +CPU= 89.214
  00:01:01 STEP 1683 H= 420:45 +CPU= 81.731
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:03:08 STEP 1684 H= 421:00 +CPU=104.488
  00:04:31 STEP 1685 H= 421:15 +CPU= 82.104
  00:05:49 STEP 1686 H= 421:30 +CPU= 76.808
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:07:28 STEP 1687 H= 421:45 +CPU= 78.259
  00:09:25 STEP 1688 H= 422:00 +CPU=115.042
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:11:20 STEP 1689 H= 422:15 +CPU= 92.326
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:13:18 STEP 1690 H= 422:30 +CPU= 75.666
  00:14:31 STEP 1691 H= 422:45 +CPU= 71.959
  00:16:03 STEP 1692 H= 423:00 +CPU= 90.781
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+001692
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+001692
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+001692
  00:17:51 STEP 1693 H= 423:15 +CPU=107.347
  00:19:20 STEP 1694 H= 423:30 +CPU= 86.126
  00:20:28 STEP 1695 H= 423:45 +CPU= 68.099
  00:21:37 STEP 1696 H= 424:00 +CPU= 68.172
  00:23:02 STEP 1697 H= 424:15 +CPU= 84.419
  00:24:44 STEP 1698 H= 424:30 +CPU=100.610
  00:26:21 STEP 1699 H= 424:45 +CPU= 95.213
  00:27:46 STEP 1700 H= 425:00 +CPU= 83.742
  00:29:04 STEP 1701 H= 425:15 +CPU= 77.733
  00:30:32 STEP 1702 H= 425:30 +CPU= 86.958
  00:31:59 STEP 1703 H= 425:45 +CPU= 86.699
  00:34:04 STEP 1704 H= 426:00 +CPU=122.650
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+001704
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+001704
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+001704
  00:35:11 STEP 1705 H= 426:15 +CPU= 66.662
  00:36:16 STEP 1706 H= 426:30 +CPU= 64.490
  00:37:49 STEP 1707 H= 426:45 +CPU= 92.058
  00:39:20 STEP 1708 H= 427:00 +CPU= 89.974
  00:40:38 STEP 1709 H= 427:15 +CPU= 77.579
  00:41:59 STEP 1710 H= 427:30 +CPU= 80.800
  00:43:18 STEP 1711 H= 427:45 +CPU= 78.341
  00:44:53 STEP 1712 H= 428:00 +CPU= 92.959
  00:46:29 STEP 1713 H= 428:15 +CPU= 95.852
  00:48:02 STEP 1714 H= 428:30 +CPU= 91.612
  00:49:10 STEP 1715 H= 428:45 +CPU= 67.831
  00:51:00 STEP 1716 H= 429:00 +CPU=108.640
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+001716
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+001716
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+001716
  00:52:39 STEP 1717 H= 429:15 +CPU= 97.030
  00:54:14 STEP 1718 H= 429:30 +CPU= 94.695
  00:55:45 STEP 1719 H= 429:45 +CPU= 89.907
  00:57:08 STEP 1720 H= 430:00 +CPU= 82.065
  00:58:26 STEP 1721 H= 430:15 +CPU= 77.350
  01:00:08 STEP 1722 H= 430:30 +CPU=100.746
  01:01:41 STEP 1723 H= 430:45 +CPU= 91.100
  01:03:05 STEP 1724 H= 431:00 +CPU= 82.690
  01:04:21 STEP 1725 H= 431:15 +CPU= 75.672
  01:05:47 STEP 1726 H= 431:30 +CPU= 85.562
  01:07:28 STEP 1727 H= 431:45 +CPU= 99.643
  01:09:42 STEP 1728 H= 432:00 +CPU=131.734
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+001728
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+001728
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+001728
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMGGh7zg+001632
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMSHh7zg+001632
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMUAh7zg+001632
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMGGh7zg+001644
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMSHh7zg+001644
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMUAh7zg+001644
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMGGh7zg+001656
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMSHh7zg+001656
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMUAh7zg+001656
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMGGh7zg+001668
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMSHh7zg+001668
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMUAh7zg+001668
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMGGh7zg+001680
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMSHh7zg+001680
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMUAh7zg+001680
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMGGh7zg+001692
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMSHh7zg+001692
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMUAh7zg+001692
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMGGh7zg+001704
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMSHh7zg+001704
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMUAh7zg+001704
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMGGh7zg+001716
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMSHh7zg+001716
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMUAh7zg+001716
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a09h_2016092300_20_1018_12288668_0_r1508782734_17.zip
Uploading the intermediate file: upload_file_17.zip
  01:11:02 STEP 1729 H= 432:15 +CPU= 77.137
  01:12:15 STEP 1730 H= 432:30 +CPU= 72.353
  01:14:08 STEP 1731 H= 432:45 +CPU=109.108
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:16:05 STEP 1732 H= 433:00 +CPU= 92.664
  01:17:36 STEP 1733 H= 433:15 +CPU= 91.050
  01:18:43 STEP 1734 H= 433:30 +CPU= 66.089
  01:19:50 STEP 1735 H= 433:45 +CPU= 66.780
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:22:23 STEP 1736 H= 434:00 +CPU=108.752
  01:23:39 STEP 1737 H= 434:15 +CPU= 75.558
  01:25:04 STEP 1738 H= 434:30 +CPU= 82.904
  01:26:30 STEP 1739 H= 434:45 +CPU= 86.170
  01:28:07 STEP 1740 H= 435:00 +CPU= 91.415
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+001740
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+001740
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+001740
  01:29:50 STEP 1741 H= 435:15 +CPU=101.740
  01:31:12 STEP 1742 H= 435:30 +CPU= 80.707
  01:32:32 STEP 1743 H= 435:45 +CPU= 79.341
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:34:15 STEP 1744 H= 436:00 +CPU= 81.772
  01:36:01 STEP 1745 H= 436:15 +CPU=104.493
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:38:25 STEP 1746 H= 436:30 +CPU=121.837
  01:40:03 STEP 1747 H= 436:45 +CPU= 95.029
  01:41:22 STEP 1748 H= 437:00 +CPU= 78.472
  01:42:43 STEP 1749 H= 437:15 +CPU= 80.520
  01:44:29 STEP 1750 H= 437:30 +CPU=104.173
  01:46:04 STEP 1751 H= 437:45 +CPU= 94.283
  01:48:03 STEP 1752 H= 438:00 +CPU=118.026
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+001752
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+001752
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+001752
  01:49:15 STEP 1753 H= 438:15 +CPU= 71.316
  01:51:08 STEP 1754 H= 438:30 +CPU=111.801
  01:52:47 STEP 1755 H= 438:45 +CPU= 97.836
  01:54:19 STEP 1756 H= 439:00 +CPU= 90.537
  01:55:43 STEP 1757 H= 439:15 +CPU= 83.619
  01:57:16 STEP 1758 H= 439:30 +CPU= 91.978
  01:59:12 STEP 1759 H= 439:45 +CPU=113.505
  02:00:55 STEP 1760 H= 440:00 +CPU=102.075
  02:02:25 STEP 1761 H= 440:15 +CPU= 89.367
  02:04:17 STEP 1762 H= 440:30 +CPU=110.395
  02:05:48 STEP 1763 H= 440:45 +CPU= 90.929
  02:07:46 STEP 1764 H= 441:00 +CPU=116.641
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+001764
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+001764
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+001764
  02:09:02 STEP 1765 H= 441:15 +CPU= 74.559
  02:10:29 STEP 1766 H= 441:30 +CPU= 85.883
  02:12:11 STEP 1767 H= 441:45 +CPU=101.453
  02:13:33 STEP 1768 H= 442:00 +CPU= 80.438
  02:14:47 STEP 1769 H= 442:15 +CPU= 73.216
  02:16:00 STEP 1770 H= 442:30 +CPU= 72.441
  02:17:22 STEP 1771 H= 442:45 +CPU= 81.432
  02:19:02 STEP 1772 H= 443:00 +CPU= 97.939
  02:20:23 STEP 1773 H= 443:15 +CPU= 79.653
  02:21:36 STEP 1774 H= 443:30 +CPU= 71.886
  02:22:46 STEP 1775 H= 443:45 +CPU= 69.099
  02:25:00 STEP 1776 H= 444:00 +CPU=133.218
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+001776
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+001776
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+001776
  02:26:18 STEP 1777 H= 444:15 +CPU= 75.011
  02:27:50 STEP 1778 H= 444:30 +CPU= 90.922
  02:29:12 STEP 1779 H= 444:45 +CPU= 81.116
  02:30:19 STEP 1780 H= 445:00 +CPU= 66.370
  02:31:42 STEP 1781 H= 445:15 +CPU= 81.771
  02:33:24 STEP 1782 H= 445:30 +CPU=100.435
  02:34:41 STEP 1783 H= 445:45 +CPU= 76.357
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_bl_1.13_x86_64-pc-linux-gnu
(argv1) start_date: 2016092300
(argv2) exptid: h7zg
(argv3) unique_member_id: a09h
(argv4) batchid: 1018
(argv5) wuid: 12288668
(argv6) fclen: 20
(argv7) app_name: oifs_43r3_bl
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/8
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_bl
version: 1.13
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/8/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/8/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_eb1b6c1e46b8b06039c8c9121c2685b0 to: /var/lib/boinc-client/slots/8/oifs_43r3_bl_a09h_2016092300_20_1018_12288668.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/8/oifs_43r3_bl_a09h_2016092300_20_1018_12288668.zip
ic_ancil_file: ic_ancil_12288668
ifsdata_file: ifsdata_12288668
climate_data_file: clim_data_12288668
horiz_resolution: 159
vert_resolution: 91
grid_type: l_2
upload_interval: 96
utstep: 900
nfrres: restart dump frequency (steps) 48
Copying IC ancils from: ../../projects/climateprediction.net/jf_f7ecfd27aa1e68bbe7e8d19800e25c73 to: /var/lib/boinc-client/slots/8/ic_ancil_12288668.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/8/ic_ancil_12288668.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_265951956d1fd0c788a37ed9a90fce61 to: /var/lib/boinc-client/slots/8/ifsdata/ifsdata_12288668.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/8/ifsdata/ifsdata_12288668.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/8/159l_2/clim_data_12288668.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/8/159l_2/clim_data_12288668.zip
Checking for progress XML file: /var/lib/boinc-client/slots/8/progress_file_12288668.xml
Opened progress file ok : /var/lib/boinc-client/slots/8/progress_file_12288668.xml
-- Model is restarting --
Adjusting last_iter, 1784, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/8/progress_file_12288668.xml
last_cpu_time: 173528
upload_file_number: 18
last_iter: 1777
last_upload: 1555200
model_completed: 0
total_length_of_simulation: 1728000
result_base_name: oifs_43r3_bl_a09h_2016092300_20_1018_12288668_0_r1508782734
The child process has been launched with process id: 2958
Executing the command: /var/lib/boinc-client/slots/8/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:hpc1:1:1:2958:2958] [20240712:023617:1720708577:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2091] fp = 0x314f440
[EC_DRHOOK:hpc1:1:1:2958:2958] [20240712:023617:1720708577:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:hpc1:1:1:2958:2958] [20240712:023617:1720708577:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:hpc1:1:1:2958:2958] [20240712:023617:1720708577:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:hpc1:1:1:2958:2958] [20240712:023617:1720708577:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:hpc1:1:1:2958:2958] [20240712:023617:1720708577:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:hpc1:1:1:2958:2958] [20240712:023617:1720708577:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:hpc1:1:1:2958:2958] [20240712:023617:1720708577:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:hpc1:1:1:2958:2958] [20240712:023617:1720708577:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:hpc1:1:1:2958:2958] [20240712:023617:1720708577:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:hpc1:1:1:2958:2958] [20240712:023617:1720708577:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:hpc1:1:1:2958:2958] [20240712:023617:1720708577:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:hpc1:1:1:2958:2958] [20240712:023617:1720708577:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:hpc1:1:1:2958:2958] [20240712:023617:1720708577:0.001] [ignore_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:hpc1:1:1:2958:2958] [20240712:023617:1720708577:0.001] [restore_default_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:hpc1:1:1:2958:2958] [20240712:023617:1720708577:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:hpc1:1:1:2958:2958] [20240712:023617:1720708577:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:hpc1:1:1:2958:2958] [20240712:023617:1720708577:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:hpc1:1:1:2958:2958] [20240712:023617:1720708577:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x820cf0 (old at (nil))
[EC_DRHOOK:hpc1:1:1:2958:2958] [20240712:023617:1720708577:0.001] [trapfpe_treatment@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:hpc1:1:1:2958:2958] [20240712:023617:1720708577:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:hpc1:1:1:2958:2958] [20240712:023617:1720708577:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:hpc1:1:1:2958:2958] [20240712:023617:1720708577:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x820cf0 (old at (nil))
[EC_DRHOOK:hpc1:1:1:2958:2958] [20240712:023617:1720708577:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:hpc1:1:1:2958:2958] [20240712:023617:1720708577:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:hpc1:1:1:2958:2958] [20240712:023617:1720708577:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:hpc1:1:1:2958:2958] [20240712:023617:1720708577:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x820cf0 (old at (nil))
[EC_DRHOOK:hpc1:1:1:2958:2958] [20240712:023617:1720708577:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x820cf0 (old at (nil))
[EC_DRHOOK:hpc1:1:1:2958:2958] [20240712:023617:1720708577:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:hpc1:1:1:2958:2958] [20240712:023617:1720708577:0.001] [catch_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:38:27 STEP 1776 H= 444:00 +CPU= 89.864
  02:39:43 STEP 1777 H= 444:15 +CPU= 76.131
  02:41:43 STEP 1778 H= 444:30 +CPU=119.335
  02:43:00 STEP 1779 H= 444:45 +CPU= 76.075
  02:44:04 STEP 1780 H= 445:00 +CPU= 63.860
  02:45:06 STEP 1781 H= 445:15 +CPU= 62.043
  02:46:08 STEP 1782 H= 445:30 +CPU= 61.291
  02:47:42 STEP 1783 H= 445:45 +CPU= 92.518
  02:49:10 STEP 1784 H= 446:00 +CPU= 86.988
  02:50:23 STEP 1785 H= 446:15 +CPU= 71.753
  02:51:23 STEP 1786 H= 446:30 +CPU= 59.726
  02:52:41 STEP 1787 H= 446:45 +CPU= 77.788
  02:54:37 STEP 1788 H= 447:00 +CPU=115.489
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+001788
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+001788
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+001788
  02:55:58 STEP 1789 H= 447:15 +CPU= 80.204
  02:57:20 STEP 1790 H= 447:30 +CPU= 81.290
  02:58:29 STEP 1791 H= 447:45 +CPU= 68.136
  02:59:40 STEP 1792 H= 448:00 +CPU= 70.791
  03:01:24 STEP 1793 H= 448:15 +CPU=103.051
  03:02:41 STEP 1794 H= 448:30 +CPU= 75.960
  03:04:09 STEP 1795 H= 448:45 +CPU= 86.729
  03:05:34 STEP 1796 H= 449:00 +CPU= 84.680
  03:07:01 STEP 1797 H= 449:15 +CPU= 85.918
  03:08:39 STEP 1798 H= 449:30 +CPU= 97.167
  03:09:55 STEP 1799 H= 449:45 +CPU= 75.715
  03:11:23 STEP 1800 H= 450:00 +CPU= 87.244
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+001800
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+001800
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+001800
  03:12:37 STEP 1801 H= 450:15 +CPU= 73.525
  03:14:26 STEP 1802 H= 450:30 +CPU=107.042
  03:15:48 STEP 1803 H= 450:45 +CPU= 80.976
  03:17:01 STEP 1804 H= 451:00 +CPU= 72.678
  03:18:20 STEP 1805 H= 451:15 +CPU= 78.235
  03:19:33 STEP 1806 H= 451:30 +CPU= 72.912
  03:20:56 STEP 1807 H= 451:45 +CPU= 81.390
  03:22:39 STEP 1808 H= 452:00 +CPU=101.405
  03:24:05 STEP 1809 H= 452:15 +CPU= 86.198
  03:25:22 STEP 1810 H= 452:30 +CPU= 76.732
  03:26:31 STEP 1811 H= 452:45 +CPU= 68.550
  03:28:25 STEP 1812 H= 453:00 +CPU=112.171
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+001812
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+001812
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+001812
  03:30:06 STEP 1813 H= 453:15 +CPU= 99.248
  03:31:24 STEP 1814 H= 453:30 +CPU= 77.992
  03:32:41 STEP 1815 H= 453:45 +CPU= 76.329
  03:33:47 STEP 1816 H= 454:00 +CPU= 65.623
  03:35:21 STEP 1817 H= 454:15 +CPU= 93.701
  03:36:51 STEP 1818 H= 454:30 +CPU= 89.121
  03:38:06 STEP 1819 H= 454:45 +CPU= 72.047
  03:39:15 STEP 1820 H= 455:00 +CPU= 68.292
  03:40:29 STEP 1821 H= 455:15 +CPU= 73.100
  03:42:19 STEP 1822 H= 455:30 +CPU=108.820
  03:43:44 STEP 1823 H= 455:45 +CPU= 84.071
  03:45:51 STEP 1824 H= 456:00 +CPU=123.413
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+001824
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+001824
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+001824
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMGGh7zg+001728
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMSHh7zg+001728
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMUAh7zg+001728
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMGGh7zg+001740
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMSHh7zg+001740
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMUAh7zg+001740
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMGGh7zg+001752
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMSHh7zg+001752
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMUAh7zg+001752
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMGGh7zg+001764
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMSHh7zg+001764
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMUAh7zg+001764
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMGGh7zg+001776
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMSHh7zg+001776
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMUAh7zg+001776
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMGGh7zg+001788
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMSHh7zg+001788
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMUAh7zg+001788
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMGGh7zg+001800
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMSHh7zg+001800
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMUAh7zg+001800
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMGGh7zg+001812
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMSHh7zg+001812
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMUAh7zg+001812
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a09h_2016092300_20_1018_12288668_0_r1508782734_18.zip
Uploading the intermediate file: upload_file_18.zip
  03:47:05 STEP 1825 H= 456:15 +CPU= 73.760
  03:48:02 STEP 1826 H= 456:30 +CPU= 56.879
  03:49:32 STEP 1827 H= 456:45 +CPU= 87.890
  03:51:22 STEP 1828 H= 457:00 +CPU=107.976
  03:52:44 STEP 1829 H= 457:15 +CPU= 80.292
  03:53:58 STEP 1830 H= 457:30 +CPU= 73.176
  03:55:21 STEP 1831 H= 457:45 +CPU= 82.339
  03:56:54 STEP 1832 H= 458:00 +CPU= 92.259
  03:58:30 STEP 1833 H= 458:15 +CPU= 94.907
  04:00:04 STEP 1834 H= 458:30 +CPU= 91.367
  04:01:19 STEP 1835 H= 458:45 +CPU= 75.089
  04:03:08 STEP 1836 H= 459:00 +CPU=107.563
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+001836
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+001836
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+001836
  04:04:30 STEP 1837 H= 459:15 +CPU= 80.885
  04:05:53 STEP 1838 H= 459:30 +CPU= 81.671
  04:07:27 STEP 1839 H= 459:45 +CPU= 93.563
  04:08:31 STEP 1840 H= 460:00 +CPU= 62.935
  04:09:43 STEP 1841 H= 460:15 +CPU= 71.901
  04:11:29 STEP 1842 H= 460:30 +CPU=105.465
  04:12:48 STEP 1843 H= 460:45 +CPU= 77.259
  04:14:05 STEP 1844 H= 461:00 +CPU= 75.986
  04:15:28 STEP 1845 H= 461:15 +CPU= 82.568
  04:16:37 STEP 1846 H= 461:30 +CPU= 68.212
  04:18:18 STEP 1847 H= 461:45 +CPU=100.087
  04:20:07 STEP 1848 H= 462:00 +CPU=108.047
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+001848
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+001848
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+001848
  04:21:34 STEP 1849 H= 462:15 +CPU= 85.517
  04:22:53 STEP 1850 H= 462:30 +CPU= 79.023
  04:23:58 STEP 1851 H= 462:45 +CPU= 64.686
  04:25:20 STEP 1852 H= 463:00 +CPU= 80.739
  04:27:08 STEP 1853 H= 463:15 +CPU=106.885
  04:28:23 STEP 1854 H= 463:30 +CPU= 73.806
  04:29:25 STEP 1855 H= 463:45 +CPU= 60.811
  04:30:42 STEP 1856 H= 464:00 +CPU= 76.858
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:32:17 STEP 1857 H= 464:15 +CPU= 72.981
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:34:51 STEP 1858 H= 464:30 +CPU=109.956
  04:36:09 STEP 1859 H= 464:45 +CPU= 76.502
  04:37:47 STEP 1860 H= 465:00 +CPU= 96.068
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+001860
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+001860
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+001860
  04:38:55 STEP 1861 H= 465:15 +CPU= 67.986
  04:40:00 STEP 1862 H= 465:30 +CPU= 64.889
  04:41:26 STEP 1863 H= 465:45 +CPU= 84.801
  04:43:04 STEP 1864 H= 466:00 +CPU= 96.913
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:44:41 STEP 1865 H= 466:15 +CPU= 74.896
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:46:46 STEP 1866 H= 466:30 +CPU= 80.953
  04:48:00 STEP 1867 H= 466:45 +CPU= 73.992
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:49:41 STEP 1868 H= 467:00 +CPU= 79.616
  04:51:26 STEP 1869 H= 467:15 +CPU=103.780
  04:52:39 STEP 1870 H= 467:30 +CPU= 73.333
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:54:29 STEP 1871 H= 467:45 +CPU= 87.915
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:57:02 STEP 1872 H= 468:00 +CPU=110.434
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+001872
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+001872
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+001872
  04:58:30 STEP 1873 H= 468:15 +CPU= 87.053
  05:00:02 STEP 1874 H= 468:30 +CPU= 92.249
  05:01:24 STEP 1875 H= 468:45 +CPU= 79.977
  05:02:40 STEP 1876 H= 469:00 +CPU= 75.556
  05:04:01 STEP 1877 H= 469:15 +CPU= 80.966
  05:05:17 STEP 1878 H= 469:30 +CPU= 74.676
  05:06:52 STEP 1879 H= 469:45 +CPU= 94.224
  05:08:02 STEP 1880 H= 470:00 +CPU= 69.708
  05:09:19 STEP 1881 H= 470:15 +CPU= 77.364
  05:10:29 STEP 1882 H= 470:30 +CPU= 69.515
  05:11:22 STEP 1883 H= 470:45 +CPU= 52.714
  05:13:05 STEP 1884 H= 471:00 +CPU=101.669
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+001884
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+001884
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+001884
  05:14:19 STEP 1885 H= 471:15 +CPU= 73.227
  05:15:24 STEP 1886 H= 471:30 +CPU= 64.403
  05:16:43 STEP 1887 H= 471:45 +CPU= 77.937
  05:17:39 STEP 1888 H= 472:00 +CPU= 55.705
  05:18:45 STEP 1889 H= 472:15 +CPU= 66.148
  05:20:04 STEP 1890 H= 472:30 +CPU= 78.116
  05:21:17 STEP 1891 H= 472:45 +CPU= 72.140
  05:22:23 STEP 1892 H= 473:00 +CPU= 66.102
  05:23:24 STEP 1893 H= 473:15 +CPU= 59.901
  05:24:30 STEP 1894 H= 473:30 +CPU= 66.585
  05:25:23 STEP 1895 H= 473:45 +CPU= 52.275
  05:26:56 STEP 1896 H= 474:00 +CPU= 92.960
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+001896
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+001896
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+001896
  05:28:00 STEP 1897 H= 474:15 +CPU= 62.970
  05:29:15 STEP 1898 H= 474:30 +CPU= 74.661
  05:30:24 STEP 1899 H= 474:45 +CPU= 68.317
  05:31:33 STEP 1900 H= 475:00 +CPU= 69.066
  05:32:37 STEP 1901 H= 475:15 +CPU= 63.701
  05:33:56 STEP 1902 H= 475:30 +CPU= 78.318
  05:35:11 STEP 1903 H= 475:45 +CPU= 74.912
  05:36:16 STEP 1904 H= 476:00 +CPU= 64.970
  05:37:26 STEP 1905 H= 476:15 +CPU= 69.374
  05:38:38 STEP 1906 H= 476:30 +CPU= 72.282
  05:39:37 STEP 1907 H= 476:45 +CPU= 58.832
  05:41:16 STEP 1908 H= 477:00 +CPU= 97.305
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+001908
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+001908
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+001908
  05:42:16 STEP 1909 H= 477:15 +CPU= 60.680
  05:43:34 STEP 1910 H= 477:30 +CPU= 77.282
  05:44:42 STEP 1911 H= 477:45 +CPU= 67.153
  05:45:42 STEP 1912 H= 478:00 +CPU= 59.578
  05:47:00 STEP 1913 H= 478:15 +CPU= 77.420
  05:48:11 STEP 1914 H= 478:30 +CPU= 70.901
  05:49:24 STEP 1915 H= 478:45 +CPU= 72.104
  05:50:48 STEP 1916 H= 479:00 +CPU= 81.974
  05:51:59 STEP 1917 H= 479:15 +CPU= 70.891
  05:53:06 STEP 1918 H= 479:30 +CPU= 66.181
  05:54:36 STEP 1919 H= 479:45 +CPU= 86.741
  05:56:56 STEP 1920 H= 480:00 +CPU=138.099
..The child process terminated with status: 0
>>> Printing last 70 lines from file: NODE.001_01

 
DDH------------------------------------------------------------------------
I                                                                         I
I TIME =  480.000H DDH * Diagnostic Var.,     AREA                GLOBAL  I
I                                                                         I
I-------------------------------------------------------------------------I
I   VARIABLE   I      Initial value         I        Current value        I
I-------------------------------------------------------------------------I
I  OMEGA (P/S) I      0.2485569908E-21      I       0.9116754932E-08      I
ICloud fractionI      0.0000000000E+00      I       0.2127846557E-01      I
I  Relat. Hum. I      0.3700783164E+00      I       0.4098653327E+00      I
I  PASS 01     I      0.0000000000E+00      I       0.9818023879E+02      I
DDH------------------------------------------------------------------------
  
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
 NSTEP =  1920 SCAN2M_HPOS  P
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ICMGGh7zg+001920 MODE=a
 IO-STREAM CLOSED - ICMGGh7zg+001920
 MPI-TASK:   1 -    308683428 BYTES IN      7 RECORDS TRANSFERRED IN    0.0044 SECONDS 70155.3 Mbytes/s, TOTAL TIME=    0.0380(11.6%)
 05:56:56 STEP 1920 H= 480:00 +CPU=138.099
 END CNT3
 NSTEP =  1920 CNT0   000000000  93.641  93.641  94.895   0.987               0               0               0               0               0
IO-STREAM STATISTICS, TOTAL NO OF REC/BYTES READ           46/  1069125708 WRITTEN         4290/************
 -TASK-OPENED-OPEN-RECS IN -KBYTE IN    -RECS OUT-KBYTE OUT-WALL   -WALL IN-WALL
  OU-TOT IN -TOT OUT
   1     78    0       46      1044068     4290 ************    50.3    11.0    39.3    18.5    62.4
===-=== START OF TIMING STATISTICS ===-===
 
STATS FOR ALL TASKS
 NUM ROUTINE                                     CALLS  MEAN(ms)   MAX(ms)   FRAC(%)  UNBAL(%)
   0 CNT0     - COMPLETE EXECUTION                   1 ********* *********    100.00      0.00
   1 CNT4     - FORWARD INTEGRATION                  1 ********* *********     99.67      0.00
   8 SCAN2M - GRID-POINT DYNAMICS                  157   12591.1   12591.1     16.42      0.00
   9 SPCM     - SPECTRAL COMP.                     144    1878.0    1878.0      2.25      0.00
  10 SCAN2M - PHYSICS                              145   37633.1   37633.1     45.33      0.00
  11 IOPACK   - OUTPUT P.P. RESULTS                 12    4204.6    4204.6      0.42      0.00
  12 SPNORM   - SPECTRAL NORM COMP.                  7    3194.3    3194.3      0.19      0.00
  17 GRIDFPOS IN CNT4                               12     131.6     131.6      0.01      0.00
  27 RESTART FILES - WRITING                         3   16817.5   16817.5      0.42      0.00
  28 RESTART FILES - READING                         1   22142.8   22142.8      0.18      0.00
  29 SU4FPOS IN CNT4                                12       0.9       0.9      0.00      0.00
  30 DYNFPOS IN CNT4                                12   16404.9   16404.9      1.64      0.00
  31 POSDDH IN STEPO                                 1       9.4       9.4      0.00      0.00
  37 CPGLAG   - SL COMPUTATIONS                    145   18780.7   18780.7     22.62      0.00
  39 SU0YOMB                                         1   16889.2   16889.2      0.14      0.00
  51 SCAN2M   - SL COMM. PART 1                    145     464.6     464.6      0.56      0.00
  54 SPCM     - M TO S/S TO M TRANSP.              144     865.5     865.5      1.04      0.00
  55 SPCIMPF  - S TO M/M TO S TRANSP.              144     231.2     231.2      0.28      0.00
  56 SPNORM   - SPECTRAL NORM COMM.                  7       1.5       1.5      0.00      0.00
 102 LTINV_CTL   - INVERSE LEGENDRE TRANSFORM      316    1174.0    1174.0      3.08      0.00
 103 LTDIR_CTL   - DIRECT LEGENDRE TRANSFORM       300     623.4     623.4      1.55      0.00
 106 FTDIR_CTL   - DIRECT FOURIER TRANSFORM        300     342.4     342.4      0.85      0.00
 107 FTINV_CTL   - INVERSE FOURIER TRANSFORM       316     328.9     328.9      0.86      0.00
 140 SULEG       - COMP. OF LEGENDRE POL.            1      63.5      63.5      0.00      0.00
 152 LTINV_CTL   - M TO L TRANSPOSITION            316     280.9     280.9      0.74      0.00
 153 LTDIR_CTL   - L TO M TRANSPOSITION            300     137.1     137.1      0.34      0.00
 157 FTINV_CTL   - L TO G TRANSPOSITION            316     221.2     221.2      0.58      0.00
 158 FTDIR_CTL   - G TO L TRANSPOSITION            300     185.1     185.1      0.46      0.00
 400 GSTATS                                      17763       0.0       0.0      0.00      0.00
 401 GSTATS HOOK                                 16473       0.0       0.0      0.01      0.00
TOTAL MEASURED IMBALANCE =       0.0 SECONDS,  0.0 PERCENT
TOTAL WALLCLOCK TIME    12039.1 CPU TIME   11697.2 VECTOR TIME    11697.2


===-=== END   OF TIMING STATISTICS ===-===
FORECAST DAYS PER DAY      144.0
  *** END CNT0 *** 
------------------------------------------------
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+001920
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+001920
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+001920
Adding to the zip: /var/lib/boinc-client/slots/8/NODE.001_01
Adding to the zip: /var/lib/boinc-client/slots/8/ifs.stat
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMGGh7zg+001848
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMGGh7zg+001860
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMGGh7zg+001920
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMUAh7zg+001836
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMUAh7zg+001848
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMUAh7zg+001884
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMGGh7zg+001884
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMUAh7zg+001920
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMSHh7zg+001824
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMSHh7zg+001908
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMGGh7zg+001908
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMGGh7zg+001872
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMGGh7zg+001836
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMSHh7zg+001872
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMSHh7zg+001884
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMUAh7zg+001872
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMSHh7zg+001896
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMSHh7zg+001836
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMSHh7zg+001860
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMGGh7zg+001824
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMGGh7zg+001896
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMUAh7zg+001824
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMUAh7zg+001896
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMSHh7zg+001848
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMUAh7zg+001860
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMUAh7zg+001908
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMSHh7zg+001920
Zipping up the final file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a09h_2016092300_20_1018_12288668_0_r1508782734_19.zip
Uploading the final file: upload_file_19.zip
Uploading trickle at timestep: 1727100
06:00:42 (2955): called boinc_finish(0)

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
11 Jul 2024 18:50:50 1541083 22442231 oifs_43r3_bl_a09h_2016092300_20_1018_12288668_0 1,727,100 185,225 0.1072


©2024 cpdn.org