Name | oifs_43r3_bl_a09h_2016092300_20_1018_12288668_0 |
Workunit | 12288668 |
Created | 12 Jun 2024, 20:28:13 UTC |
Sent | 12 Jun 2024, 20:54:24 UTC |
Report deadline | 11 Aug 2024, 20:54:24 UTC |
Received | 11 Jul 2024, 18:49:30 UTC |
Server state | Over |
Outcome | Success |
Client state | Done |
Exit status | 0 (0x00000000) |
Computer ID | 1541083 |
Run time | 2 days 3 hours 27 min 5 sec |
CPU time | 2 days 3 hours 27 min 5 sec |
Validate state | Valid |
Credit | 2,635.55 |
Device peak FLOPS | 2.64 GFLOPS |
Application version | OpenIFS 43r3 Baroclinic Lifecycle v1.13 x86_64-pc-linux-gnu |
Peak working set size | 5,023.25 MB |
Peak swap size | 5,981.08 MB |
Peak disk usage | 2,234.25 MB |
Stderr | <core_client_version>8.0.2</core_client_version> <![CDATA[ <stderr_txt> H= 395:00 +CPU= 92.312 05:56:33 STEP 1581 H= 395:15 +CPU=103.278 05:57:54 STEP 1582 H= 395:30 +CPU= 79.119 05:59:08 STEP 1583 H= 395:45 +CPU= 74.177 06:01:27 STEP 1584 H= 396:00 +CPU=133.888 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+001584 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+001584 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+001584 06:03:06 STEP 1585 H= 396:15 +CPU= 98.516 06:04:35 STEP 1586 H= 396:30 +CPU= 88.201 06:06:16 STEP 1587 H= 396:45 +CPU= 97.619 06:07:47 STEP 1588 H= 397:00 +CPU= 89.689 06:09:25 STEP 1589 H= 397:15 +CPU= 95.971 06:10:34 STEP 1590 H= 397:30 +CPU= 68.225 06:11:49 STEP 1591 H= 397:45 +CPU= 74.520 06:13:40 STEP 1592 H= 398:00 +CPU=108.170 06:15:23 STEP 1593 H= 398:15 +CPU=101.473 06:16:48 STEP 1594 H= 398:30 +CPU= 85.266 06:17:50 STEP 1595 H= 398:45 +CPU= 61.244 06:19:52 STEP 1596 H= 399:00 +CPU=121.029 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+001596 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+001596 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+001596 06:21:26 STEP 1597 H= 399:15 +CPU= 92.665 06:22:49 STEP 1598 H= 399:30 +CPU= 81.594 06:24:03 STEP 1599 H= 399:45 +CPU= 73.876 06:26:20 STEP 1600 H= 400:00 +CPU=131.647 06:27:38 STEP 1601 H= 400:15 +CPU= 77.203 06:28:57 STEP 1602 H= 400:30 +CPU= 78.823 06:30:09 STEP 1603 H= 400:45 +CPU= 71.425 06:31:49 STEP 1604 H= 401:00 +CPU= 97.185 06:33:18 STEP 1605 H= 401:15 +CPU= 87.621 06:34:28 STEP 1606 H= 401:30 +CPU= 68.998 06:35:59 STEP 1607 H= 401:45 +CPU= 90.332 06:38:03 STEP 1608 H= 402:00 +CPU=121.904 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+001608 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+001608 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+001608 06:39:34 STEP 1609 H= 402:15 +CPU= 90.155 06:40:45 STEP 1610 H= 402:30 +CPU= 70.860 06:42:31 STEP 1611 H= 402:45 +CPU=101.894 06:44:32 STEP 1612 H= 403:00 +CPU=120.028 06:46:20 STEP 1613 H= 403:15 +CPU=106.619 06:48:06 STEP 1614 H= 403:30 +CPU=103.350 06:50:00 STEP 1615 H= 403:45 +CPU=112.652 06:51:40 STEP 1616 H= 404:00 +CPU= 98.872 06:53:28 STEP 1617 H= 404:15 +CPU=106.813 06:55:20 STEP 1618 H= 404:30 +CPU=109.616 06:57:16 STEP 1619 H= 404:45 +CPU=114.756 06:59:16 STEP 1620 H= 405:00 +CPU=118.939 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+001620 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+001620 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+001620 07:01:09 STEP 1621 H= 405:15 +CPU=111.230 07:02:52 STEP 1622 H= 405:30 +CPU=102.607 07:04:50 STEP 1623 H= 405:45 +CPU=116.239 07:06:26 STEP 1624 H= 406:00 +CPU= 94.606 07:08:29 STEP 1625 H= 406:15 +CPU=119.885 07:10:17 STEP 1626 H= 406:30 +CPU=107.526 07:12:02 STEP 1627 H= 406:45 +CPU=104.094 07:13:32 STEP 1628 H= 407:00 +CPU= 88.925 07:15:41 STEP 1629 H= 407:15 +CPU=128.123 07:17:16 STEP 1630 H= 407:30 +CPU= 94.104 07:18:51 STEP 1631 H= 407:45 +CPU= 93.610 07:20:38 STEP 1632 H= 408:00 +CPU=104.488 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+001632 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+001632 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+001632 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMGGh7zg+001536 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMSHh7zg+001536 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMUAh7zg+001536 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMGGh7zg+001548 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMSHh7zg+001548 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMUAh7zg+001548 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMGGh7zg+001560 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMSHh7zg+001560 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMUAh7zg+001560 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMGGh7zg+001572 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMSHh7zg+001572 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMUAh7zg+001572 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMGGh7zg+001584 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMSHh7zg+001584 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMUAh7zg+001584 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMGGh7zg+001596 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMSHh7zg+001596 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMUAh7zg+001596 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMGGh7zg+001608 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMSHh7zg+001608 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMUAh7zg+001608 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMGGh7zg+001620 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMSHh7zg+001620 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMUAh7zg+001620 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a09h_2016092300_20_1018_12288668_0_r1508782734_16.zip Uploading the intermediate file: upload_file_16.zip 07:22:20 STEP 1633 H= 408:15 +CPU= 99.026 07:23:45 STEP 1634 H= 408:30 +CPU= 84.120 07:25:05 STEP 1635 H= 408:45 +CPU= 79.029 07:26:30 STEP 1636 H= 409:00 +CPU= 83.522 07:28:04 STEP 1637 H= 409:15 +CPU= 90.980 07:29:37 STEP 1638 H= 409:30 +CPU= 91.696 07:31:07 STEP 1639 H= 409:45 +CPU= 88.536 07:32:21 STEP 1640 H= 410:00 +CPU= 72.529 07:33:45 STEP 1641 H= 410:15 +CPU= 81.379 07:35:21 STEP 1642 H= 410:30 +CPU= 94.742 07:36:58 STEP 1643 H= 410:45 +CPU= 96.376 07:38:32 STEP 1644 H= 411:00 +CPU= 93.334 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+001644 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+001644 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+001644 07:39:48 STEP 1645 H= 411:15 +CPU= 75.245 07:41:11 STEP 1646 H= 411:30 +CPU= 80.556 07:42:47 STEP 1647 H= 411:45 +CPU= 94.453 07:44:13 STEP 1648 H= 412:00 +CPU= 86.007 07:45:28 STEP 1649 H= 412:15 +CPU= 74.052 07:46:47 STEP 1650 H= 412:30 +CPU= 78.684 07:48:19 STEP 1651 H= 412:45 +CPU= 89.151 07:50:00 STEP 1652 H= 413:00 +CPU= 99.961 07:51:16 STEP 1653 H= 413:15 +CPU= 74.405 07:52:37 STEP 1654 H= 413:30 +CPU= 80.713 07:53:51 STEP 1655 H= 413:45 +CPU= 73.282 07:55:57 STEP 1656 H= 414:00 +CPU=123.841 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+001656 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+001656 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+001656 07:57:25 STEP 1657 H= 414:15 +CPU= 86.585 07:58:40 STEP 1658 H= 414:30 +CPU= 74.048 (argv0) ../../projects/climateprediction.net/oifs_43r3_bl_1.13_x86_64-pc-linux-gnu (argv1) start_date: 2016092300 (argv2) exptid: h7zg (argv3) unique_member_id: a09h (argv4) batchid: 1018 (argv5) wuid: 12288668 (argv6) fclen: 20 (argv7) app_name: oifs_43r3_bl (argv8) nthreads: 1 Working directory is: /var/lib/boinc-client/slots/8 Project directory is: /var/lib/boinc-client/projects/climateprediction.net/ app name: oifs_43r3_bl version: 1.13 Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668 ..mkdir for temp folder for results failed Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/8/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip Unzipping the app zip file: /var/lib/boinc-client/slots/8/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip Copying the namelist files from: ../../projects/climateprediction.net/jf_eb1b6c1e46b8b06039c8c9121c2685b0 to: /var/lib/boinc-client/slots/8/oifs_43r3_bl_a09h_2016092300_20_1018_12288668.zip Unzipping the namelist zip file: /var/lib/boinc-client/slots/8/oifs_43r3_bl_a09h_2016092300_20_1018_12288668.zip ic_ancil_file: ic_ancil_12288668 ifsdata_file: ifsdata_12288668 climate_data_file: clim_data_12288668 horiz_resolution: 159 vert_resolution: 91 grid_type: l_2 upload_interval: 96 utstep: 900 nfrres: restart dump frequency (steps) 48 Copying IC ancils from: ../../projects/climateprediction.net/jf_f7ecfd27aa1e68bbe7e8d19800e25c73 to: /var/lib/boinc-client/slots/8/ic_ancil_12288668.zip Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/8/ic_ancil_12288668.zip ..mkdir for ifsdata folder failed Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_265951956d1fd0c788a37ed9a90fce61 to: /var/lib/boinc-client/slots/8/ifsdata/ifsdata_12288668.zip Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/8/ifsdata/ifsdata_12288668.zip ..mkdir for the climate data folder failed Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/8/159l_2/clim_data_12288668.zip Unzipping the climate data zip file: /var/lib/boinc-client/slots/8/159l_2/clim_data_12288668.zip Checking for progress XML file: /var/lib/boinc-client/slots/8/progress_file_12288668.xml Opened progress file ok : /var/lib/boinc-client/slots/8/progress_file_12288668.xml -- Model is restarting -- Adjusting last_iter, 1659, to previous model restart step. Creating progress file: /var/lib/boinc-client/slots/8/progress_file_12288668.xml last_cpu_time: 159524 upload_file_number: 17 last_iter: 1633 last_upload: 1468800 model_completed: 0 total_length_of_simulation: 1728000 result_base_name: oifs_43r3_bl_a09h_2016092300_20_1018_12288668_0_r1508782734 The child process has been launched with process id: 1957 Executing the command: /var/lib/boinc-client/slots/8/oifs_43r3_model.exe [EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1 [EC_DRHOOK:hpc1:1:1:1957:1957] [20240711:223044:1720693844:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2091] fp = 0x314f440 [EC_DRHOOK:hpc1:1:1:1957:1957] [20240711:223044:1720693844:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1 [EC_DRHOOK:hpc1:1:1:1957:1957] [20240711:223044:1720693844:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0) [EC_DRHOOK:hpc1:1:1:1957:1957] [20240711:223044:1720693844:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1 [EC_DRHOOK:hpc1:1:1:1957:1957] [20240711:223044:1720693844:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000 [EC_DRHOOK:hpc1:1:1:1957:1957] [20240711:223044:1720693844:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647) [EC_DRHOOK:hpc1:1:1:1957:1957] [20240711:223044:1720693844:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16 [EC_DRHOOK:hpc1:1:1:1957:1957] [20240711:223044:1720693844:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0 [EC_DRHOOK:hpc1:1:1:1957:1957] [20240711:223044:1720693844:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500 [EC_DRHOOK:hpc1:1:1:1957:1957] [20240711:223044:1720693844:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1 [EC_DRHOOK:hpc1:1:1:1957:1957] [20240711:223044:1720693844:0.001] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1 [EC_DRHOOK:hpc1:1:1:1957:1957] [20240711:223044:1720693844:0.001] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1 [EC_DRHOOK:hpc1:1:1:1957:1957] [20240711:223044:1720693844:0.001] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1 [EC_DRHOOK:hpc1:1:1:1957:1957] [20240711:223044:1720693844:0.001] [ignore_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef> [EC_DRHOOK:hpc1:1:1:1957:1957] [20240711:223044:1720693844:0.001] [restore_default_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef> [EC_DRHOOK:hpc1:1:1:1957:1957] [20240711:223044:1720693844:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x820cf0 (old at 0x1cf8da0) [EC_DRHOOK:hpc1:1:1:1957:1957] [20240711:223044:1720693844:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x820cf0 (old at (nil)) [EC_DRHOOK:hpc1:1:1:1957:1957] [20240711:223044:1720693844:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x820cf0 (old at 0x1cf8da0) [EC_DRHOOK:hpc1:1:1:1957:1957] [20240711:223044:1720693844:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x820cf0 (old at (nil)) [EC_DRHOOK:hpc1:1:1:1957:1957] [20240711:223044:1720693844:0.001] [trapfpe_treatment@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1 [EC_DRHOOK:hpc1:1:1:1957:1957] [20240711:223044:1720693844:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x820cf0 (old at 0x1cf8da0) [EC_DRHOOK:hpc1:1:1:1957:1957] [20240711:223044:1720693844:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x820cf0 (old at 0x1cf8da0) [EC_DRHOOK:hpc1:1:1:1957:1957] [20240711:223044:1720693844:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x820cf0 (old at (nil)) [EC_DRHOOK:hpc1:1:1:1957:1957] [20240711:223044:1720693844:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x820cf0 (old at 0x1cf8da0) [EC_DRHOOK:hpc1:1:1:1957:1957] [20240711:223044:1720693844:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x820cf0 (old at 0x1cf8da0) [EC_DRHOOK:hpc1:1:1:1957:1957] [20240711:223044:1720693844:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x820cf0 (old at 0x1cf8da0) [EC_DRHOOK:hpc1:1:1:1957:1957] [20240711:223044:1720693844:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x820cf0 (old at (nil)) [EC_DRHOOK:hpc1:1:1:1957:1957] [20240711:223044:1720693844:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x820cf0 (old at (nil)) [EC_DRHOOK:hpc1:1:1:1957:1957] [20240711:223044:1720693844:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x820cf0 (old at (nil)) [EC_DRHOOK:hpc1:1:1:1957:1957] [20240711:223044:1720693844:0.001] [catch_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef> 22:33:19 STEP 1632 H= 408:00 +CPU=129.047 22:35:50 STEP 1633 H= 408:15 +CPU=145.660 22:38:03 STEP 1634 H= 408:30 +CPU=129.931 22:40:21 STEP 1635 H= 408:45 +CPU=128.680 22:42:31 STEP 1636 H= 409:00 +CPU=100.122 22:44:27 STEP 1637 H= 409:15 +CPU=111.573 22:45:57 STEP 1638 H= 409:30 +CPU= 87.718 22:47:32 STEP 1639 H= 409:45 +CPU= 91.143 22:49:12 STEP 1640 H= 410:00 +CPU= 97.331 22:50:41 STEP 1641 H= 410:15 +CPU= 87.414 22:52:13 STEP 1642 H= 410:30 +CPU= 90.482 22:53:25 STEP 1643 H= 410:45 +CPU= 70.962 22:55:05 STEP 1644 H= 411:00 +CPU= 99.026 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+001644 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+001644 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+001644 22:56:20 STEP 1645 H= 411:15 +CPU= 73.282 22:57:40 STEP 1646 H= 411:30 +CPU= 78.495 22:58:46 STEP 1647 H= 411:45 +CPU= 64.998 23:00:05 STEP 1648 H= 412:00 +CPU= 77.933 23:01:22 STEP 1649 H= 412:15 +CPU= 74.722 23:02:50 STEP 1650 H= 412:30 +CPU= 87.369 23:04:02 STEP 1651 H= 412:45 +CPU= 70.708 23:05:22 STEP 1652 H= 413:00 +CPU= 78.660 23:06:58 STEP 1653 H= 413:15 +CPU= 94.411 23:08:29 STEP 1654 H= 413:30 +CPU= 89.455 23:09:42 STEP 1655 H= 413:45 +CPU= 72.111 23:11:39 STEP 1656 H= 414:00 +CPU=115.315 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+001656 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+001656 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+001656 23:13:00 STEP 1657 H= 414:15 +CPU= 79.699 23:14:22 STEP 1658 H= 414:30 +CPU= 82.125 23:15:32 STEP 1659 H= 414:45 +CPU= 69.491 23:17:02 STEP 1660 H= 415:00 +CPU= 89.200 23:18:40 STEP 1661 H= 415:15 +CPU= 96.842 23:20:30 STEP 1662 H= 415:30 +CPU=108.576 23:22:22 STEP 1663 H= 415:45 +CPU=111.945 23:24:34 STEP 1664 H= 416:00 +CPU=128.983 23:26:41 STEP 1665 H= 416:15 +CPU=125.714 23:28:28 STEP 1666 H= 416:30 +CPU=105.342 23:30:27 STEP 1667 H= 416:45 +CPU=117.681 23:32:46 STEP 1668 H= 417:00 +CPU=138.080 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+001668 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+001668 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+001668 23:34:24 STEP 1669 H= 417:15 +CPU= 96.086 23:36:50 STEP 1670 H= 417:30 +CPU=138.964 23:38:16 STEP 1671 H= 417:45 +CPU= 83.574 23:40:08 STEP 1672 H= 418:00 +CPU=108.733 23:41:55 STEP 1673 H= 418:15 +CPU=106.220 23:43:21 STEP 1674 H= 418:30 +CPU= 85.162 23:44:44 STEP 1675 H= 418:45 +CPU= 82.331 23:46:35 STEP 1676 H= 419:00 +CPU=109.115 23:48:18 STEP 1677 H= 419:15 +CPU=103.011 Suspend request received from the BOINC client, suspending the child process Resuming the child process 23:50:08 STEP 1678 H= 419:30 +CPU= 87.711 Suspend request received from the BOINC client, suspending the child process Resuming the child process 23:51:50 STEP 1679 H= 419:45 +CPU= 79.036 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 23:55:24 STEP 1680 H= 420:00 +CPU=148.665 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+001680 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+001680 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+001680 Suspend request received from the BOINC client, suspending the child process Resuming the child process 23:57:26 STEP 1681 H= 420:15 +CPU= 97.940 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 23:59:38 STEP 1682 H= 420:30 +CPU= 89.214 00:01:01 STEP 1683 H= 420:45 +CPU= 81.731 Suspend request received from the BOINC client, suspending the child process Resuming the child process 00:03:08 STEP 1684 H= 421:00 +CPU=104.488 00:04:31 STEP 1685 H= 421:15 +CPU= 82.104 00:05:49 STEP 1686 H= 421:30 +CPU= 76.808 Suspend request received from the BOINC client, suspending the child process Resuming the child process 00:07:28 STEP 1687 H= 421:45 +CPU= 78.259 00:09:25 STEP 1688 H= 422:00 +CPU=115.042 Suspend request received from the BOINC client, suspending the child process Resuming the child process 00:11:20 STEP 1689 H= 422:15 +CPU= 92.326 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 00:13:18 STEP 1690 H= 422:30 +CPU= 75.666 00:14:31 STEP 1691 H= 422:45 +CPU= 71.959 00:16:03 STEP 1692 H= 423:00 +CPU= 90.781 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+001692 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+001692 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+001692 00:17:51 STEP 1693 H= 423:15 +CPU=107.347 00:19:20 STEP 1694 H= 423:30 +CPU= 86.126 00:20:28 STEP 1695 H= 423:45 +CPU= 68.099 00:21:37 STEP 1696 H= 424:00 +CPU= 68.172 00:23:02 STEP 1697 H= 424:15 +CPU= 84.419 00:24:44 STEP 1698 H= 424:30 +CPU=100.610 00:26:21 STEP 1699 H= 424:45 +CPU= 95.213 00:27:46 STEP 1700 H= 425:00 +CPU= 83.742 00:29:04 STEP 1701 H= 425:15 +CPU= 77.733 00:30:32 STEP 1702 H= 425:30 +CPU= 86.958 00:31:59 STEP 1703 H= 425:45 +CPU= 86.699 00:34:04 STEP 1704 H= 426:00 +CPU=122.650 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+001704 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+001704 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+001704 00:35:11 STEP 1705 H= 426:15 +CPU= 66.662 00:36:16 STEP 1706 H= 426:30 +CPU= 64.490 00:37:49 STEP 1707 H= 426:45 +CPU= 92.058 00:39:20 STEP 1708 H= 427:00 +CPU= 89.974 00:40:38 STEP 1709 H= 427:15 +CPU= 77.579 00:41:59 STEP 1710 H= 427:30 +CPU= 80.800 00:43:18 STEP 1711 H= 427:45 +CPU= 78.341 00:44:53 STEP 1712 H= 428:00 +CPU= 92.959 00:46:29 STEP 1713 H= 428:15 +CPU= 95.852 00:48:02 STEP 1714 H= 428:30 +CPU= 91.612 00:49:10 STEP 1715 H= 428:45 +CPU= 67.831 00:51:00 STEP 1716 H= 429:00 +CPU=108.640 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+001716 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+001716 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+001716 00:52:39 STEP 1717 H= 429:15 +CPU= 97.030 00:54:14 STEP 1718 H= 429:30 +CPU= 94.695 00:55:45 STEP 1719 H= 429:45 +CPU= 89.907 00:57:08 STEP 1720 H= 430:00 +CPU= 82.065 00:58:26 STEP 1721 H= 430:15 +CPU= 77.350 01:00:08 STEP 1722 H= 430:30 +CPU=100.746 01:01:41 STEP 1723 H= 430:45 +CPU= 91.100 01:03:05 STEP 1724 H= 431:00 +CPU= 82.690 01:04:21 STEP 1725 H= 431:15 +CPU= 75.672 01:05:47 STEP 1726 H= 431:30 +CPU= 85.562 01:07:28 STEP 1727 H= 431:45 +CPU= 99.643 01:09:42 STEP 1728 H= 432:00 +CPU=131.734 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+001728 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+001728 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+001728 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMGGh7zg+001632 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMSHh7zg+001632 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMUAh7zg+001632 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMGGh7zg+001644 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMSHh7zg+001644 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMUAh7zg+001644 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMGGh7zg+001656 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMSHh7zg+001656 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMUAh7zg+001656 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMGGh7zg+001668 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMSHh7zg+001668 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMUAh7zg+001668 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMGGh7zg+001680 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMSHh7zg+001680 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMUAh7zg+001680 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMGGh7zg+001692 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMSHh7zg+001692 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMUAh7zg+001692 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMGGh7zg+001704 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMSHh7zg+001704 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMUAh7zg+001704 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMGGh7zg+001716 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMSHh7zg+001716 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMUAh7zg+001716 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a09h_2016092300_20_1018_12288668_0_r1508782734_17.zip Uploading the intermediate file: upload_file_17.zip 01:11:02 STEP 1729 H= 432:15 +CPU= 77.137 01:12:15 STEP 1730 H= 432:30 +CPU= 72.353 01:14:08 STEP 1731 H= 432:45 +CPU=109.108 Suspend request received from the BOINC client, suspending the child process Resuming the child process 01:16:05 STEP 1732 H= 433:00 +CPU= 92.664 01:17:36 STEP 1733 H= 433:15 +CPU= 91.050 01:18:43 STEP 1734 H= 433:30 +CPU= 66.089 01:19:50 STEP 1735 H= 433:45 +CPU= 66.780 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 01:22:23 STEP 1736 H= 434:00 +CPU=108.752 01:23:39 STEP 1737 H= 434:15 +CPU= 75.558 01:25:04 STEP 1738 H= 434:30 +CPU= 82.904 01:26:30 STEP 1739 H= 434:45 +CPU= 86.170 01:28:07 STEP 1740 H= 435:00 +CPU= 91.415 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+001740 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+001740 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+001740 01:29:50 STEP 1741 H= 435:15 +CPU=101.740 01:31:12 STEP 1742 H= 435:30 +CPU= 80.707 01:32:32 STEP 1743 H= 435:45 +CPU= 79.341 Suspend request received from the BOINC client, suspending the child process Resuming the child process 01:34:15 STEP 1744 H= 436:00 +CPU= 81.772 01:36:01 STEP 1745 H= 436:15 +CPU=104.493 Suspend request received from the BOINC client, suspending the child process Resuming the child process 01:38:25 STEP 1746 H= 436:30 +CPU=121.837 01:40:03 STEP 1747 H= 436:45 +CPU= 95.029 01:41:22 STEP 1748 H= 437:00 +CPU= 78.472 01:42:43 STEP 1749 H= 437:15 +CPU= 80.520 01:44:29 STEP 1750 H= 437:30 +CPU=104.173 01:46:04 STEP 1751 H= 437:45 +CPU= 94.283 01:48:03 STEP 1752 H= 438:00 +CPU=118.026 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+001752 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+001752 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+001752 01:49:15 STEP 1753 H= 438:15 +CPU= 71.316 01:51:08 STEP 1754 H= 438:30 +CPU=111.801 01:52:47 STEP 1755 H= 438:45 +CPU= 97.836 01:54:19 STEP 1756 H= 439:00 +CPU= 90.537 01:55:43 STEP 1757 H= 439:15 +CPU= 83.619 01:57:16 STEP 1758 H= 439:30 +CPU= 91.978 01:59:12 STEP 1759 H= 439:45 +CPU=113.505 02:00:55 STEP 1760 H= 440:00 +CPU=102.075 02:02:25 STEP 1761 H= 440:15 +CPU= 89.367 02:04:17 STEP 1762 H= 440:30 +CPU=110.395 02:05:48 STEP 1763 H= 440:45 +CPU= 90.929 02:07:46 STEP 1764 H= 441:00 +CPU=116.641 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+001764 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+001764 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+001764 02:09:02 STEP 1765 H= 441:15 +CPU= 74.559 02:10:29 STEP 1766 H= 441:30 +CPU= 85.883 02:12:11 STEP 1767 H= 441:45 +CPU=101.453 02:13:33 STEP 1768 H= 442:00 +CPU= 80.438 02:14:47 STEP 1769 H= 442:15 +CPU= 73.216 02:16:00 STEP 1770 H= 442:30 +CPU= 72.441 02:17:22 STEP 1771 H= 442:45 +CPU= 81.432 02:19:02 STEP 1772 H= 443:00 +CPU= 97.939 02:20:23 STEP 1773 H= 443:15 +CPU= 79.653 02:21:36 STEP 1774 H= 443:30 +CPU= 71.886 02:22:46 STEP 1775 H= 443:45 +CPU= 69.099 02:25:00 STEP 1776 H= 444:00 +CPU=133.218 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+001776 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+001776 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+001776 02:26:18 STEP 1777 H= 444:15 +CPU= 75.011 02:27:50 STEP 1778 H= 444:30 +CPU= 90.922 02:29:12 STEP 1779 H= 444:45 +CPU= 81.116 02:30:19 STEP 1780 H= 445:00 +CPU= 66.370 02:31:42 STEP 1781 H= 445:15 +CPU= 81.771 02:33:24 STEP 1782 H= 445:30 +CPU=100.435 02:34:41 STEP 1783 H= 445:45 +CPU= 76.357 Suspend request received from the BOINC client, suspending the child process Resuming the child process Quit request received from BOINC client, ending the child process (argv0) ../../projects/climateprediction.net/oifs_43r3_bl_1.13_x86_64-pc-linux-gnu (argv1) start_date: 2016092300 (argv2) exptid: h7zg (argv3) unique_member_id: a09h (argv4) batchid: 1018 (argv5) wuid: 12288668 (argv6) fclen: 20 (argv7) app_name: oifs_43r3_bl (argv8) nthreads: 1 Working directory is: /var/lib/boinc-client/slots/8 Project directory is: /var/lib/boinc-client/projects/climateprediction.net/ app name: oifs_43r3_bl version: 1.13 Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668 ..mkdir for temp folder for results failed Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/8/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip Unzipping the app zip file: /var/lib/boinc-client/slots/8/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip Copying the namelist files from: ../../projects/climateprediction.net/jf_eb1b6c1e46b8b06039c8c9121c2685b0 to: /var/lib/boinc-client/slots/8/oifs_43r3_bl_a09h_2016092300_20_1018_12288668.zip Unzipping the namelist zip file: /var/lib/boinc-client/slots/8/oifs_43r3_bl_a09h_2016092300_20_1018_12288668.zip ic_ancil_file: ic_ancil_12288668 ifsdata_file: ifsdata_12288668 climate_data_file: clim_data_12288668 horiz_resolution: 159 vert_resolution: 91 grid_type: l_2 upload_interval: 96 utstep: 900 nfrres: restart dump frequency (steps) 48 Copying IC ancils from: ../../projects/climateprediction.net/jf_f7ecfd27aa1e68bbe7e8d19800e25c73 to: /var/lib/boinc-client/slots/8/ic_ancil_12288668.zip Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/8/ic_ancil_12288668.zip ..mkdir for ifsdata folder failed Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_265951956d1fd0c788a37ed9a90fce61 to: /var/lib/boinc-client/slots/8/ifsdata/ifsdata_12288668.zip Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/8/ifsdata/ifsdata_12288668.zip ..mkdir for the climate data folder failed Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/8/159l_2/clim_data_12288668.zip Unzipping the climate data zip file: /var/lib/boinc-client/slots/8/159l_2/clim_data_12288668.zip Checking for progress XML file: /var/lib/boinc-client/slots/8/progress_file_12288668.xml Opened progress file ok : /var/lib/boinc-client/slots/8/progress_file_12288668.xml -- Model is restarting -- Adjusting last_iter, 1784, to previous model restart step. Creating progress file: /var/lib/boinc-client/slots/8/progress_file_12288668.xml last_cpu_time: 173528 upload_file_number: 18 last_iter: 1777 last_upload: 1555200 model_completed: 0 total_length_of_simulation: 1728000 result_base_name: oifs_43r3_bl_a09h_2016092300_20_1018_12288668_0_r1508782734 The child process has been launched with process id: 2958 Executing the command: /var/lib/boinc-client/slots/8/oifs_43r3_model.exe [EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1 [EC_DRHOOK:hpc1:1:1:2958:2958] [20240712:023617:1720708577:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2091] fp = 0x314f440 [EC_DRHOOK:hpc1:1:1:2958:2958] [20240712:023617:1720708577:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1 [EC_DRHOOK:hpc1:1:1:2958:2958] [20240712:023617:1720708577:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0) [EC_DRHOOK:hpc1:1:1:2958:2958] [20240712:023617:1720708577:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1 [EC_DRHOOK:hpc1:1:1:2958:2958] [20240712:023617:1720708577:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000 [EC_DRHOOK:hpc1:1:1:2958:2958] [20240712:023617:1720708577:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647) [EC_DRHOOK:hpc1:1:1:2958:2958] [20240712:023617:1720708577:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16 [EC_DRHOOK:hpc1:1:1:2958:2958] [20240712:023617:1720708577:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0 [EC_DRHOOK:hpc1:1:1:2958:2958] [20240712:023617:1720708577:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500 [EC_DRHOOK:hpc1:1:1:2958:2958] [20240712:023617:1720708577:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1 [EC_DRHOOK:hpc1:1:1:2958:2958] [20240712:023617:1720708577:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1 [EC_DRHOOK:hpc1:1:1:2958:2958] [20240712:023617:1720708577:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1 [EC_DRHOOK:hpc1:1:1:2958:2958] [20240712:023617:1720708577:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1 [EC_DRHOOK:hpc1:1:1:2958:2958] [20240712:023617:1720708577:0.001] [ignore_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef> [EC_DRHOOK:hpc1:1:1:2958:2958] [20240712:023617:1720708577:0.001] [restore_default_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef> [EC_DRHOOK:hpc1:1:1:2958:2958] [20240712:023617:1720708577:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x820cf0 (old at 0x1cf8da0) [EC_DRHOOK:hpc1:1:1:2958:2958] [20240712:023617:1720708577:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x820cf0 (old at (nil)) [EC_DRHOOK:hpc1:1:1:2958:2958] [20240712:023617:1720708577:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x820cf0 (old at 0x1cf8da0) [EC_DRHOOK:hpc1:1:1:2958:2958] [20240712:023617:1720708577:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x820cf0 (old at (nil)) [EC_DRHOOK:hpc1:1:1:2958:2958] [20240712:023617:1720708577:0.001] [trapfpe_treatment@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1 [EC_DRHOOK:hpc1:1:1:2958:2958] [20240712:023617:1720708577:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x820cf0 (old at 0x1cf8da0) [EC_DRHOOK:hpc1:1:1:2958:2958] [20240712:023617:1720708577:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x820cf0 (old at 0x1cf8da0) [EC_DRHOOK:hpc1:1:1:2958:2958] [20240712:023617:1720708577:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x820cf0 (old at (nil)) [EC_DRHOOK:hpc1:1:1:2958:2958] [20240712:023617:1720708577:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x820cf0 (old at 0x1cf8da0) [EC_DRHOOK:hpc1:1:1:2958:2958] [20240712:023617:1720708577:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x820cf0 (old at 0x1cf8da0) [EC_DRHOOK:hpc1:1:1:2958:2958] [20240712:023617:1720708577:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x820cf0 (old at 0x1cf8da0) [EC_DRHOOK:hpc1:1:1:2958:2958] [20240712:023617:1720708577:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x820cf0 (old at (nil)) [EC_DRHOOK:hpc1:1:1:2958:2958] [20240712:023617:1720708577:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x820cf0 (old at (nil)) [EC_DRHOOK:hpc1:1:1:2958:2958] [20240712:023617:1720708577:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x820cf0 (old at (nil)) [EC_DRHOOK:hpc1:1:1:2958:2958] [20240712:023617:1720708577:0.001] [catch_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef> Suspend request received from the BOINC client, suspending the child process Resuming the child process 02:38:27 STEP 1776 H= 444:00 +CPU= 89.864 02:39:43 STEP 1777 H= 444:15 +CPU= 76.131 02:41:43 STEP 1778 H= 444:30 +CPU=119.335 02:43:00 STEP 1779 H= 444:45 +CPU= 76.075 02:44:04 STEP 1780 H= 445:00 +CPU= 63.860 02:45:06 STEP 1781 H= 445:15 +CPU= 62.043 02:46:08 STEP 1782 H= 445:30 +CPU= 61.291 02:47:42 STEP 1783 H= 445:45 +CPU= 92.518 02:49:10 STEP 1784 H= 446:00 +CPU= 86.988 02:50:23 STEP 1785 H= 446:15 +CPU= 71.753 02:51:23 STEP 1786 H= 446:30 +CPU= 59.726 02:52:41 STEP 1787 H= 446:45 +CPU= 77.788 02:54:37 STEP 1788 H= 447:00 +CPU=115.489 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+001788 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+001788 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+001788 02:55:58 STEP 1789 H= 447:15 +CPU= 80.204 02:57:20 STEP 1790 H= 447:30 +CPU= 81.290 02:58:29 STEP 1791 H= 447:45 +CPU= 68.136 02:59:40 STEP 1792 H= 448:00 +CPU= 70.791 03:01:24 STEP 1793 H= 448:15 +CPU=103.051 03:02:41 STEP 1794 H= 448:30 +CPU= 75.960 03:04:09 STEP 1795 H= 448:45 +CPU= 86.729 03:05:34 STEP 1796 H= 449:00 +CPU= 84.680 03:07:01 STEP 1797 H= 449:15 +CPU= 85.918 03:08:39 STEP 1798 H= 449:30 +CPU= 97.167 03:09:55 STEP 1799 H= 449:45 +CPU= 75.715 03:11:23 STEP 1800 H= 450:00 +CPU= 87.244 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+001800 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+001800 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+001800 03:12:37 STEP 1801 H= 450:15 +CPU= 73.525 03:14:26 STEP 1802 H= 450:30 +CPU=107.042 03:15:48 STEP 1803 H= 450:45 +CPU= 80.976 03:17:01 STEP 1804 H= 451:00 +CPU= 72.678 03:18:20 STEP 1805 H= 451:15 +CPU= 78.235 03:19:33 STEP 1806 H= 451:30 +CPU= 72.912 03:20:56 STEP 1807 H= 451:45 +CPU= 81.390 03:22:39 STEP 1808 H= 452:00 +CPU=101.405 03:24:05 STEP 1809 H= 452:15 +CPU= 86.198 03:25:22 STEP 1810 H= 452:30 +CPU= 76.732 03:26:31 STEP 1811 H= 452:45 +CPU= 68.550 03:28:25 STEP 1812 H= 453:00 +CPU=112.171 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+001812 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+001812 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+001812 03:30:06 STEP 1813 H= 453:15 +CPU= 99.248 03:31:24 STEP 1814 H= 453:30 +CPU= 77.992 03:32:41 STEP 1815 H= 453:45 +CPU= 76.329 03:33:47 STEP 1816 H= 454:00 +CPU= 65.623 03:35:21 STEP 1817 H= 454:15 +CPU= 93.701 03:36:51 STEP 1818 H= 454:30 +CPU= 89.121 03:38:06 STEP 1819 H= 454:45 +CPU= 72.047 03:39:15 STEP 1820 H= 455:00 +CPU= 68.292 03:40:29 STEP 1821 H= 455:15 +CPU= 73.100 03:42:19 STEP 1822 H= 455:30 +CPU=108.820 03:43:44 STEP 1823 H= 455:45 +CPU= 84.071 03:45:51 STEP 1824 H= 456:00 +CPU=123.413 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+001824 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+001824 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+001824 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMGGh7zg+001728 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMSHh7zg+001728 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMUAh7zg+001728 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMGGh7zg+001740 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMSHh7zg+001740 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMUAh7zg+001740 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMGGh7zg+001752 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMSHh7zg+001752 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMUAh7zg+001752 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMGGh7zg+001764 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMSHh7zg+001764 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMUAh7zg+001764 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMGGh7zg+001776 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMSHh7zg+001776 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMUAh7zg+001776 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMGGh7zg+001788 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMSHh7zg+001788 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMUAh7zg+001788 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMGGh7zg+001800 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMSHh7zg+001800 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMUAh7zg+001800 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMGGh7zg+001812 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMSHh7zg+001812 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMUAh7zg+001812 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a09h_2016092300_20_1018_12288668_0_r1508782734_18.zip Uploading the intermediate file: upload_file_18.zip 03:47:05 STEP 1825 H= 456:15 +CPU= 73.760 03:48:02 STEP 1826 H= 456:30 +CPU= 56.879 03:49:32 STEP 1827 H= 456:45 +CPU= 87.890 03:51:22 STEP 1828 H= 457:00 +CPU=107.976 03:52:44 STEP 1829 H= 457:15 +CPU= 80.292 03:53:58 STEP 1830 H= 457:30 +CPU= 73.176 03:55:21 STEP 1831 H= 457:45 +CPU= 82.339 03:56:54 STEP 1832 H= 458:00 +CPU= 92.259 03:58:30 STEP 1833 H= 458:15 +CPU= 94.907 04:00:04 STEP 1834 H= 458:30 +CPU= 91.367 04:01:19 STEP 1835 H= 458:45 +CPU= 75.089 04:03:08 STEP 1836 H= 459:00 +CPU=107.563 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+001836 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+001836 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+001836 04:04:30 STEP 1837 H= 459:15 +CPU= 80.885 04:05:53 STEP 1838 H= 459:30 +CPU= 81.671 04:07:27 STEP 1839 H= 459:45 +CPU= 93.563 04:08:31 STEP 1840 H= 460:00 +CPU= 62.935 04:09:43 STEP 1841 H= 460:15 +CPU= 71.901 04:11:29 STEP 1842 H= 460:30 +CPU=105.465 04:12:48 STEP 1843 H= 460:45 +CPU= 77.259 04:14:05 STEP 1844 H= 461:00 +CPU= 75.986 04:15:28 STEP 1845 H= 461:15 +CPU= 82.568 04:16:37 STEP 1846 H= 461:30 +CPU= 68.212 04:18:18 STEP 1847 H= 461:45 +CPU=100.087 04:20:07 STEP 1848 H= 462:00 +CPU=108.047 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+001848 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+001848 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+001848 04:21:34 STEP 1849 H= 462:15 +CPU= 85.517 04:22:53 STEP 1850 H= 462:30 +CPU= 79.023 04:23:58 STEP 1851 H= 462:45 +CPU= 64.686 04:25:20 STEP 1852 H= 463:00 +CPU= 80.739 04:27:08 STEP 1853 H= 463:15 +CPU=106.885 04:28:23 STEP 1854 H= 463:30 +CPU= 73.806 04:29:25 STEP 1855 H= 463:45 +CPU= 60.811 04:30:42 STEP 1856 H= 464:00 +CPU= 76.858 Suspend request received from the BOINC client, suspending the child process Resuming the child process 04:32:17 STEP 1857 H= 464:15 +CPU= 72.981 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 04:34:51 STEP 1858 H= 464:30 +CPU=109.956 04:36:09 STEP 1859 H= 464:45 +CPU= 76.502 04:37:47 STEP 1860 H= 465:00 +CPU= 96.068 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+001860 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+001860 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+001860 04:38:55 STEP 1861 H= 465:15 +CPU= 67.986 04:40:00 STEP 1862 H= 465:30 +CPU= 64.889 04:41:26 STEP 1863 H= 465:45 +CPU= 84.801 04:43:04 STEP 1864 H= 466:00 +CPU= 96.913 Suspend request received from the BOINC client, suspending the child process Resuming the child process 04:44:41 STEP 1865 H= 466:15 +CPU= 74.896 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 04:46:46 STEP 1866 H= 466:30 +CPU= 80.953 04:48:00 STEP 1867 H= 466:45 +CPU= 73.992 Suspend request received from the BOINC client, suspending the child process Resuming the child process 04:49:41 STEP 1868 H= 467:00 +CPU= 79.616 04:51:26 STEP 1869 H= 467:15 +CPU=103.780 04:52:39 STEP 1870 H= 467:30 +CPU= 73.333 Suspend request received from the BOINC client, suspending the child process Resuming the child process 04:54:29 STEP 1871 H= 467:45 +CPU= 87.915 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 04:57:02 STEP 1872 H= 468:00 +CPU=110.434 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+001872 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+001872 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+001872 04:58:30 STEP 1873 H= 468:15 +CPU= 87.053 05:00:02 STEP 1874 H= 468:30 +CPU= 92.249 05:01:24 STEP 1875 H= 468:45 +CPU= 79.977 05:02:40 STEP 1876 H= 469:00 +CPU= 75.556 05:04:01 STEP 1877 H= 469:15 +CPU= 80.966 05:05:17 STEP 1878 H= 469:30 +CPU= 74.676 05:06:52 STEP 1879 H= 469:45 +CPU= 94.224 05:08:02 STEP 1880 H= 470:00 +CPU= 69.708 05:09:19 STEP 1881 H= 470:15 +CPU= 77.364 05:10:29 STEP 1882 H= 470:30 +CPU= 69.515 05:11:22 STEP 1883 H= 470:45 +CPU= 52.714 05:13:05 STEP 1884 H= 471:00 +CPU=101.669 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+001884 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+001884 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+001884 05:14:19 STEP 1885 H= 471:15 +CPU= 73.227 05:15:24 STEP 1886 H= 471:30 +CPU= 64.403 05:16:43 STEP 1887 H= 471:45 +CPU= 77.937 05:17:39 STEP 1888 H= 472:00 +CPU= 55.705 05:18:45 STEP 1889 H= 472:15 +CPU= 66.148 05:20:04 STEP 1890 H= 472:30 +CPU= 78.116 05:21:17 STEP 1891 H= 472:45 +CPU= 72.140 05:22:23 STEP 1892 H= 473:00 +CPU= 66.102 05:23:24 STEP 1893 H= 473:15 +CPU= 59.901 05:24:30 STEP 1894 H= 473:30 +CPU= 66.585 05:25:23 STEP 1895 H= 473:45 +CPU= 52.275 05:26:56 STEP 1896 H= 474:00 +CPU= 92.960 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+001896 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+001896 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+001896 05:28:00 STEP 1897 H= 474:15 +CPU= 62.970 05:29:15 STEP 1898 H= 474:30 +CPU= 74.661 05:30:24 STEP 1899 H= 474:45 +CPU= 68.317 05:31:33 STEP 1900 H= 475:00 +CPU= 69.066 05:32:37 STEP 1901 H= 475:15 +CPU= 63.701 05:33:56 STEP 1902 H= 475:30 +CPU= 78.318 05:35:11 STEP 1903 H= 475:45 +CPU= 74.912 05:36:16 STEP 1904 H= 476:00 +CPU= 64.970 05:37:26 STEP 1905 H= 476:15 +CPU= 69.374 05:38:38 STEP 1906 H= 476:30 +CPU= 72.282 05:39:37 STEP 1907 H= 476:45 +CPU= 58.832 05:41:16 STEP 1908 H= 477:00 +CPU= 97.305 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+001908 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+001908 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+001908 05:42:16 STEP 1909 H= 477:15 +CPU= 60.680 05:43:34 STEP 1910 H= 477:30 +CPU= 77.282 05:44:42 STEP 1911 H= 477:45 +CPU= 67.153 05:45:42 STEP 1912 H= 478:00 +CPU= 59.578 05:47:00 STEP 1913 H= 478:15 +CPU= 77.420 05:48:11 STEP 1914 H= 478:30 +CPU= 70.901 05:49:24 STEP 1915 H= 478:45 +CPU= 72.104 05:50:48 STEP 1916 H= 479:00 +CPU= 81.974 05:51:59 STEP 1917 H= 479:15 +CPU= 70.891 05:53:06 STEP 1918 H= 479:30 +CPU= 66.181 05:54:36 STEP 1919 H= 479:45 +CPU= 86.741 05:56:56 STEP 1920 H= 480:00 +CPU=138.099 ..The child process terminated with status: 0 >>> Printing last 70 lines from file: NODE.001_01 DDH------------------------------------------------------------------------ I I I TIME = 480.000H DDH * Diagnostic Var., AREA GLOBAL I I I I-------------------------------------------------------------------------I I VARIABLE I Initial value I Current value I I-------------------------------------------------------------------------I I OMEGA (P/S) I 0.2485569908E-21 I 0.9116754932E-08 I ICloud fractionI 0.0000000000E+00 I 0.2127846557E-01 I I Relat. Hum. I 0.3700783164E+00 I 0.4098653327E+00 I I PASS 01 I 0.0000000000E+00 I 0.9818023879E+02 I DDH------------------------------------------------------------------------ MAXGPFV : MAX. VALUE = 0.000000000000000E+000 MAXGPFV : MAX. VALUE = 0.000000000000000E+000 NSTEP = 1920 SCAN2M_HPOS P IO-STREAM SETUP - IOTYPE = 2 NUMIOPROCS = 1 CPATH = ICMGGh7zg+001920 MODE=a IO-STREAM CLOSED - ICMGGh7zg+001920 MPI-TASK: 1 - 308683428 BYTES IN 7 RECORDS TRANSFERRED IN 0.0044 SECONDS 70155.3 Mbytes/s, TOTAL TIME= 0.0380(11.6%) 05:56:56 STEP 1920 H= 480:00 +CPU=138.099 END CNT3 NSTEP = 1920 CNT0 000000000 93.641 93.641 94.895 0.987 0 0 0 0 0 IO-STREAM STATISTICS, TOTAL NO OF REC/BYTES READ 46/ 1069125708 WRITTEN 4290/************ -TASK-OPENED-OPEN-RECS IN -KBYTE IN -RECS OUT-KBYTE OUT-WALL -WALL IN-WALL OU-TOT IN -TOT OUT 1 78 0 46 1044068 4290 ************ 50.3 11.0 39.3 18.5 62.4 ===-=== START OF TIMING STATISTICS ===-=== STATS FOR ALL TASKS NUM ROUTINE CALLS MEAN(ms) MAX(ms) FRAC(%) UNBAL(%) 0 CNT0 - COMPLETE EXECUTION 1 ********* ********* 100.00 0.00 1 CNT4 - FORWARD INTEGRATION 1 ********* ********* 99.67 0.00 8 SCAN2M - GRID-POINT DYNAMICS 157 12591.1 12591.1 16.42 0.00 9 SPCM - SPECTRAL COMP. 144 1878.0 1878.0 2.25 0.00 10 SCAN2M - PHYSICS 145 37633.1 37633.1 45.33 0.00 11 IOPACK - OUTPUT P.P. RESULTS 12 4204.6 4204.6 0.42 0.00 12 SPNORM - SPECTRAL NORM COMP. 7 3194.3 3194.3 0.19 0.00 17 GRIDFPOS IN CNT4 12 131.6 131.6 0.01 0.00 27 RESTART FILES - WRITING 3 16817.5 16817.5 0.42 0.00 28 RESTART FILES - READING 1 22142.8 22142.8 0.18 0.00 29 SU4FPOS IN CNT4 12 0.9 0.9 0.00 0.00 30 DYNFPOS IN CNT4 12 16404.9 16404.9 1.64 0.00 31 POSDDH IN STEPO 1 9.4 9.4 0.00 0.00 37 CPGLAG - SL COMPUTATIONS 145 18780.7 18780.7 22.62 0.00 39 SU0YOMB 1 16889.2 16889.2 0.14 0.00 51 SCAN2M - SL COMM. PART 1 145 464.6 464.6 0.56 0.00 54 SPCM - M TO S/S TO M TRANSP. 144 865.5 865.5 1.04 0.00 55 SPCIMPF - S TO M/M TO S TRANSP. 144 231.2 231.2 0.28 0.00 56 SPNORM - SPECTRAL NORM COMM. 7 1.5 1.5 0.00 0.00 102 LTINV_CTL - INVERSE LEGENDRE TRANSFORM 316 1174.0 1174.0 3.08 0.00 103 LTDIR_CTL - DIRECT LEGENDRE TRANSFORM 300 623.4 623.4 1.55 0.00 106 FTDIR_CTL - DIRECT FOURIER TRANSFORM 300 342.4 342.4 0.85 0.00 107 FTINV_CTL - INVERSE FOURIER TRANSFORM 316 328.9 328.9 0.86 0.00 140 SULEG - COMP. OF LEGENDRE POL. 1 63.5 63.5 0.00 0.00 152 LTINV_CTL - M TO L TRANSPOSITION 316 280.9 280.9 0.74 0.00 153 LTDIR_CTL - L TO M TRANSPOSITION 300 137.1 137.1 0.34 0.00 157 FTINV_CTL - L TO G TRANSPOSITION 316 221.2 221.2 0.58 0.00 158 FTDIR_CTL - G TO L TRANSPOSITION 300 185.1 185.1 0.46 0.00 400 GSTATS 17763 0.0 0.0 0.00 0.00 401 GSTATS HOOK 16473 0.0 0.0 0.01 0.00 TOTAL MEASURED IMBALANCE = 0.0 SECONDS, 0.0 PERCENT TOTAL WALLCLOCK TIME 12039.1 CPU TIME 11697.2 VECTOR TIME 11697.2 ===-=== END OF TIMING STATISTICS ===-=== FORECAST DAYS PER DAY 144.0 *** END CNT0 *** ------------------------------------------------ Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+001920 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+001920 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+001920 Adding to the zip: /var/lib/boinc-client/slots/8/NODE.001_01 Adding to the zip: /var/lib/boinc-client/slots/8/ifs.stat Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMGGh7zg+001848 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMGGh7zg+001860 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMGGh7zg+001920 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMUAh7zg+001836 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMUAh7zg+001848 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMUAh7zg+001884 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMGGh7zg+001884 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMUAh7zg+001920 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMSHh7zg+001824 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMSHh7zg+001908 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMGGh7zg+001908 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMGGh7zg+001872 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMGGh7zg+001836 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMSHh7zg+001872 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMSHh7zg+001884 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMUAh7zg+001872 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMSHh7zg+001896 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMSHh7zg+001836 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMSHh7zg+001860 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMGGh7zg+001824 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMGGh7zg+001896 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMUAh7zg+001824 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMUAh7zg+001896 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMSHh7zg+001848 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMUAh7zg+001860 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMUAh7zg+001908 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12288668/ICMSHh7zg+001920 Zipping up the final file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a09h_2016092300_20_1018_12288668_0_r1508782734_19.zip Uploading the final file: upload_file_19.zip Uploading trickle at timestep: 1727100 06:00:42 (2955): called boinc_finish(0) </stderr_txt> ]]> |
Latest Trickles Received | ||||||
---|---|---|---|---|---|---|
Time Sent (UTC) | Host ID | Result ID | Result Name | Timestep | CPU Time (sec) | Average (sec/TS) |
11 Jul 2024 18:50:50 | 1541083 | 22442231 | oifs_43r3_bl_a09h_2016092300_20_1018_12288668_0 | 1,727,100 | 185,225 | 0.1072 |
©2024 cpdn.org