Name | oifs_43r3_ps_0625_2018050100_123_987_12204269_2 |
Workunit | 12204269 |
Created | 10 Jan 2023, 22:19:11 UTC |
Sent | 11 Jan 2023, 0:31:31 UTC |
Report deadline | 10 Feb 2023, 0:31:31 UTC |
Received | 13 Jan 2023, 6:12:02 UTC |
Server state | Over |
Outcome | Computation error |
Client state | Compute error |
Exit status | 196 (0x000000C4) EXIT_DISK_LIMIT_EXCEEDED |
Computer ID | 1534673 |
Run time | 21 hours 48 min 57 sec |
CPU time | 13 hours 8 min 5 sec |
Validate state | Invalid |
Credit | 9,434.88 |
Device peak FLOPS | 4.67 GFLOPS |
Application version | OpenIFS 43r3 Perturbed Surface v1.05 x86_64-pc-linux-gnu |
Peak working set size | 4,399.86 MB |
Peak swap size | 4,978.08 MB |
Peak disk usage | 7,548.74 MB |
Stderr | <core_client_version>7.18.1</core_client_version> <![CDATA[ <message> Disk usage limit exceeded</message> <stderr_txt> NC client, suspending the child process Resuming the child process 22:54:04 STEP 1547 H=1547:00 +CPU= 33.391 Suspend request received from the BOINC client, suspending the child process Resuming the child process 22:55:17 STEP 1548 H=1548:00 +CPU= 58.224 Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+001548 Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+001548 Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+001548 22:55:52 STEP 1549 H=1549:00 +CPU= 33.034 22:56:29 STEP 1550 H=1550:00 +CPU= 34.785 22:57:21 STEP 1551 H=1551:00 +CPU= 48.696 22:57:56 STEP 1552 H=1552:00 +CPU= 31.618 22:58:30 STEP 1553 H=1553:00 +CPU= 31.466 22:59:25 STEP 1554 H=1554:00 +CPU= 51.352 22:59:59 STEP 1555 H=1555:00 +CPU= 31.358 23:00:35 STEP 1556 H=1556:00 +CPU= 32.462 Suspend request received from the BOINC client, suspending the child process Resuming the child process 23:01:41 STEP 1557 H=1557:00 +CPU= 50.251 23:02:16 STEP 1558 H=1558:00 +CPU= 32.496 23:02:55 STEP 1559 H=1559:00 +CPU= 35.838 23:03:58 STEP 1560 H=1560:00 +CPU= 58.753 Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+001560 Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+001560 Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+001560 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12204269/ICMGGhq0f+001536 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12204269/ICMSHhq0f+001536 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12204269/ICMUAhq0f+001536 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12204269/ICMGGhq0f+001548 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12204269/ICMSHhq0f+001548 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12204269/ICMUAhq0f+001548 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0625_2018050100_123_987_12204269_2_r1542460059_33.zip Uploading the intermediate file: upload_file_33.zip 23:04:32 STEP 1561 H=1561:00 +CPU= 30.896 23:05:06 STEP 1562 H=1562:00 +CPU= 31.972 23:05:59 STEP 1563 H=1563:00 +CPU= 49.005 23:06:34 STEP 1564 H=1564:00 +CPU= 32.407 23:07:09 STEP 1565 H=1565:00 +CPU= 32.386 23:08:03 STEP 1566 H=1566:00 +CPU= 49.689 23:08:36 STEP 1567 H=1567:00 +CPU= 30.264 23:09:09 STEP 1568 H=1568:00 +CPU= 30.703 23:10:03 STEP 1569 H=1569:00 +CPU= 49.716 23:10:38 STEP 1570 H=1570:00 +CPU= 32.472 23:11:12 STEP 1571 H=1571:00 +CPU= 31.593 Suspend request received from the BOINC client, suspending the child process Resuming the child process 23:12:22 STEP 1572 H=1572:00 +CPU= 55.087 Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+001572 Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+001572 Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+001572 23:13:01 STEP 1573 H=1573:00 +CPU= 36.221 23:13:36 STEP 1574 H=1574:00 +CPU= 32.703 23:14:28 STEP 1575 H=1575:00 +CPU= 48.109 23:15:03 STEP 1576 H=1576:00 +CPU= 31.752 23:15:36 STEP 1577 H=1577:00 +CPU= 30.518 23:16:30 STEP 1578 H=1578:00 +CPU= 50.334 23:17:04 STEP 1579 H=1579:00 +CPU= 31.685 23:17:39 STEP 1580 H=1580:00 +CPU= 32.275 23:18:34 STEP 1581 H=1581:00 +CPU= 51.810 23:19:10 STEP 1582 H=1582:00 +CPU= 32.910 23:19:45 STEP 1583 H=1583:00 +CPU= 32.387 23:20:47 STEP 1584 H=1584:00 +CPU= 57.265 Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+001584 Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+001584 Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+001584 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12204269/ICMGGhq0f+001560 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12204269/ICMSHhq0f+001560 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12204269/ICMUAhq0f+001560 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12204269/ICMGGhq0f+001572 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12204269/ICMSHhq0f+001572 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12204269/ICMUAhq0f+001572 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0625_2018050100_123_987_12204269_2_r1542460059_34.zip Uploading the intermediate file: upload_file_34.zip 23:21:21 STEP 1585 H=1585:00 +CPU= 31.330 23:21:54 STEP 1586 H=1586:00 +CPU= 31.280 23:22:47 STEP 1587 H=1587:00 +CPU= 48.823 23:23:22 STEP 1588 H=1588:00 +CPU= 32.269 23:23:57 STEP 1589 H=1589:00 +CPU= 31.580 23:24:50 STEP 1590 H=1590:00 +CPU= 49.222 23:25:24 STEP 1591 H=1591:00 +CPU= 31.769 23:25:58 STEP 1592 H=1592:00 +CPU= 31.735 23:26:51 STEP 1593 H=1593:00 +CPU= 49.474 23:27:26 STEP 1594 H=1594:00 +CPU= 32.062 23:27:59 STEP 1595 H=1595:00 +CPU= 30.464 23:28:58 STEP 1596 H=1596:00 +CPU= 54.729 Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+001596 Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+001596 Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+001596 23:29:32 STEP 1597 H=1597:00 +CPU= 30.994 23:30:06 STEP 1598 H=1598:00 +CPU= 32.478 23:31:00 STEP 1599 H=1599:00 +CPU= 49.482 23:31:34 STEP 1600 H=1600:00 +CPU= 31.912 23:32:07 STEP 1601 H=1601:00 +CPU= 30.675 23:33:05 STEP 1602 H=1602:00 +CPU= 55.285 23:33:42 STEP 1603 H=1603:00 +CPU= 35.260 23:34:17 STEP 1604 H=1604:00 +CPU= 33.503 23:35:13 STEP 1605 H=1605:00 +CPU= 53.563 23:35:50 STEP 1606 H=1606:00 +CPU= 35.480 23:36:28 STEP 1607 H=1607:00 +CPU= 35.912 23:37:33 STEP 1608 H=1608:00 +CPU= 61.881 Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+001608 Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+001608 Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+001608 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12204269/ICMGGhq0f+001584 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12204269/ICMSHhq0f+001584 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12204269/ICMUAhq0f+001584 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12204269/ICMGGhq0f+001596 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12204269/ICMSHhq0f+001596 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12204269/ICMUAhq0f+001596 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0625_2018050100_123_987_12204269_2_r1542460059_35.zip Uploading the intermediate file: upload_file_35.zip 23:38:07 STEP 1609 H=1609:00 +CPU= 32.696 23:38:44 STEP 1610 H=1610:00 +CPU= 35.119 23:39:39 STEP 1611 H=1611:00 +CPU= 52.325 23:40:15 STEP 1612 H=1612:00 +CPU= 34.507 23:40:51 STEP 1613 H=1613:00 +CPU= 33.769 Suspend request received from the BOINC client, suspending the child process Resuming the child process 23:41:56 STEP 1614 H=1614:00 +CPU= 52.426 23:42:31 STEP 1615 H=1615:00 +CPU= 33.499 23:43:09 STEP 1616 H=1616:00 +CPU= 35.904 23:44:05 STEP 1617 H=1617:00 +CPU= 54.143 23:44:41 STEP 1618 H=1618:00 +CPU= 32.877 23:45:14 STEP 1619 H=1619:00 +CPU= 31.269 23:46:12 STEP 1620 H=1620:00 +CPU= 55.146 Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+001620 Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+001620 Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+001620 23:46:45 STEP 1621 H=1621:00 +CPU= 30.482 23:47:18 STEP 1622 H=1622:00 +CPU= 31.648 23:48:11 STEP 1623 H=1623:00 +CPU= 49.806 23:48:44 STEP 1624 H=1624:00 +CPU= 31.427 23:49:19 STEP 1625 H=1625:00 +CPU= 34.086 23:50:12 STEP 1626 H=1626:00 +CPU= 49.885 23:50:45 STEP 1627 H=1627:00 +CPU= 30.941 23:51:19 STEP 1628 H=1628:00 +CPU= 31.949 23:52:10 STEP 1629 H=1629:00 +CPU= 48.881 23:52:43 STEP 1630 H=1630:00 +CPU= 31.179 23:53:17 STEP 1631 H=1631:00 +CPU= 31.398 23:54:21 STEP 1632 H=1632:00 +CPU= 60.451 Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+001632 Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+001632 Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+001632 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12204269/ICMGGhq0f+001608 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12204269/ICMSHhq0f+001608 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12204269/ICMUAhq0f+001608 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12204269/ICMGGhq0f+001620 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12204269/ICMSHhq0f+001620 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12204269/ICMUAhq0f+001620 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0625_2018050100_123_987_12204269_2_r1542460059_36.zip Uploading the intermediate file: upload_file_36.zip Uploading trickle at timestep: 5875200 23:54:54 STEP 1633 H=1633:00 +CPU= 31.697 23:55:29 STEP 1634 H=1634:00 +CPU= 32.584 23:56:22 STEP 1635 H=1635:00 +CPU= 49.572 23:56:57 STEP 1636 H=1636:00 +CPU= 32.770 23:57:32 STEP 1637 H=1637:00 +CPU= 31.916 23:58:26 STEP 1638 H=1638:00 +CPU= 50.685 Quit request received from BOINC client, ending the child process (argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu (argv1) start_date: 2018050100 (argv2) exptid: hq0f (argv3) unique_member_id: 0625 (argv4) batchid: 987 (argv5) wuid: 12204269 (argv6) fclen: 123 (argv7) app_name: oifs_43r3_ps (argv8) nthreads: 1 Working directory is: /var/lib/boinc-client/slots/5 Project directory is: /var/lib/boinc-client/projects/climateprediction.net/ app name: oifs_43r3_ps version: 1.05 Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12204269 ..mkdir for temp folder for results failed Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/5/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip Unzipping the app zip file: /var/lib/boinc-client/slots/5/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip Copying the namelist files from: ../../projects/climateprediction.net/jf_be72bd4a10f5b751cb0baaa421dc53d9 to: /var/lib/boinc-client/slots/5/oifs_43r3_ps_0625_2018050100_123_987_12204269.zip Unzipping the namelist zip file: /var/lib/boinc-client/slots/5/oifs_43r3_ps_0625_2018050100_123_987_12204269.zip ic_ancil_file: ic_ancil_12204269 ifsdata_file: ifsdata_12204269 climate_data_file: clim_data_12204269 horiz_resolution: 159 vert_resolution: 60 grid_type: l_2 upload_interval: 24 utstep: 3600 nfrres: restart dump frequency (steps) 24 Copying IC ancils from: ../../projects/climateprediction.net/jf_21a5a86d5205f21a37b70596279cc24b to: /var/lib/boinc-client/slots/5/ic_ancil_12204269.zip Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/5/ic_ancil_12204269.zip ..mkdir for ifsdata folder failed Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_78825b502bbae3070cf3e0b5749ec435 to: /var/lib/boinc-client/slots/5/ifsdata/ifsdata_12204269.zip Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/5/ifsdata/ifsdata_12204269.zip ..mkdir for the climate data folder failed Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/5/159l_2/clim_data_12204269.zip Unzipping the climate data zip file: /var/lib/boinc-client/slots/5/159l_2/clim_data_12204269.zip Checking for progress XML file: /var/lib/boinc-client/slots/5/progress_file_12204269.xml Opened progress file ok : /var/lib/boinc-client/slots/5/progress_file_12204269.xml -- Model is restarting -- Adjusting last_iter, 1639, to previous model restart step. Creating progress file: /var/lib/boinc-client/slots/5/progress_file_12204269.xml last_cpu_time: 35685 upload_file_number: 37 last_iter: 1631 last_upload: 5875200 model_completed: 0 total_length_of_simulation: 10627200 result_base_name: oifs_43r3_ps_0625_2018050100_123_987_12204269_2_r1542460059 The child process has been launched with process id: 135121 Executing the command: /var/lib/boinc-client/slots/5/oifs_43r3_model.exe [EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1 [EC_DRHOOK:taman-jurong:1:1:135121:135121] [20230113:060005:1673560805:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0 [EC_DRHOOK:taman-jurong:1:1:135121:135121] [20230113:060005:1673560805:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1 [EC_DRHOOK:taman-jurong:1:1:135121:135121] [20230113:060005:1673560805:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0) [EC_DRHOOK:taman-jurong:1:1:135121:135121] [20230113:060005:1673560805:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1 [EC_DRHOOK:taman-jurong:1:1:135121:135121] [20230113:060005:1673560805:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000 [EC_DRHOOK:taman-jurong:1:1:135121:135121] [20230113:060005:1673560805:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647) [EC_DRHOOK:taman-jurong:1:1:135121:135121] [20230113:060005:1673560805:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16 [EC_DRHOOK:taman-jurong:1:1:135121:135121] [20230113:060005:1673560805:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0 [EC_DRHOOK:taman-jurong:1:1:135121:135121] [20230113:060005:1673560805:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500 [EC_DRHOOK:taman-jurong:1:1:135121:135121] [20230113:060005:1673560805:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1 [EC_DRHOOK:taman-jurong:1:1:135121:135121] [20230113:060005:1673560805:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1 [EC_DRHOOK:taman-jurong:1:1:135121:135121] [20230113:060005:1673560805:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1 [EC_DRHOOK:taman-jurong:1:1:135121:135121] [20230113:060005:1673560805:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1 [EC_DRHOOK:taman-jurong:1:1:135121:135121] [20230113:060005:1673560805:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef> [EC_DRHOOK:taman-jurong:1:1:135121:135121] [20230113:060005:1673560805:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef> [EC_DRHOOK:taman-jurong:1:1:135121:135121] [20230113:060005:1673560805:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:taman-jurong:1:1:135121:135121] [20230113:060005:1673560805:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil)) [EC_DRHOOK:taman-jurong:1:1:135121:135121] [20230113:060005:1673560805:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:taman-jurong:1:1:135121:135121] [20230113:060005:1673560805:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil)) [EC_DRHOOK:taman-jurong:1:1:135121:135121] [20230113:060005:1673560805:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1 [EC_DRHOOK:taman-jurong:1:1:135121:135121] [20230113:060005:1673560805:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:taman-jurong:1:1:135121:135121] [20230113:060005:1673560805:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:taman-jurong:1:1:135121:135121] [20230113:060005:1673560805:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil)) [EC_DRHOOK:taman-jurong:1:1:135121:135121] [20230113:060005:1673560805:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:taman-jurong:1:1:135121:135121] [20230113:060005:1673560805:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:taman-jurong:1:1:135121:135121] [20230113:060005:1673560805:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:taman-jurong:1:1:135121:135121] [20230113:060005:1673560805:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil)) [EC_DRHOOK:taman-jurong:1:1:135121:135121] [20230113:060005:1673560805:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil)) [EC_DRHOOK:taman-jurong:1:1:135121:135121] [20230113:060005:1673560805:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil)) [EC_DRHOOK:taman-jurong:1:1:135121:135121] [20230113:060005:1673560805:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef> 06:01:15 STEP 1632 H=1632:00 +CPU= 60.187 Suspend request received from the BOINC client, suspending the child process Resuming the child process 06:02:15 STEP 1633 H=1633:00 +CPU= 35.243 06:02:51 STEP 1634 H=1634:00 +CPU= 33.430 06:03:45 STEP 1635 H=1635:00 +CPU= 50.916 06:04:21 STEP 1636 H=1636:00 +CPU= 32.952 06:04:57 STEP 1637 H=1637:00 +CPU= 33.269 06:05:54 STEP 1638 H=1638:00 +CPU= 53.348 06:06:29 STEP 1639 H=1639:00 +CPU= 32.880 Suspend request received from the BOINC client, suspending the child process Resuming the child process 06:07:17 STEP 1640 H=1640:00 +CPU= 33.774 06:08:16 STEP 1641 H=1641:00 +CPU= 54.517 06:08:56 STEP 1642 H=1642:00 +CPU= 38.234 06:09:31 STEP 1643 H=1643:00 +CPU= 34.230 06:10:38 STEP 1644 H=1644:00 +CPU= 64.373 Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+001644 Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+001644 Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+001644 06:11:18 STEP 1645 H=1645:00 +CPU= 38.970 06:11:58 STEP 1646 H=1646:00 +CPU= 38.774 06:13:00 STEP 1647 H=1647:00 +CPU= 59.220 06:13:38 STEP 1648 H=1648:00 +CPU= 37.198 06:14:17 STEP 1649 H=1649:00 +CPU= 37.294 Suspend request received from the BOINC client, suspending the child process Resuming the child process 06:15:26 STEP 1650 H=1650:00 +CPU= 55.720 06:16:05 STEP 1651 H=1651:00 +CPU= 37.414 06:16:43 STEP 1652 H=1652:00 +CPU= 36.688 06:17:40 STEP 1653 H=1653:00 +CPU= 55.014 Suspend request received from the BOINC client, suspending the child process Resuming the child process 06:20:29 STEP 1654 H=1654:00 +CPU= 35.849 Suspend request received from the BOINC client, suspending the child process Resuming the child process 06:21:18 STEP 1655 H=1655:00 +CPU= 36.853 Suspend request received from the BOINC client, suspending the child process Resuming the child process 06:22:44 STEP 1656 H=1656:00 +CPU= 69.688 Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+001656 Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+001656 Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+001656 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12204269/ICMGGhq0f+001632 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12204269/ICMSHhq0f+001632 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12204269/ICMUAhq0f+001632 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12204269/ICMGGhq0f+001644 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12204269/ICMSHhq0f+001644 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12204269/ICMUAhq0f+001644 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0625_2018050100_123_987_12204269_2_r1542460059_37.zip Uploading the intermediate file: upload_file_37.zip 06:23:23 STEP 1657 H=1657:00 +CPU= 37.999 06:24:04 STEP 1658 H=1658:00 +CPU= 40.101 06:25:04 STEP 1659 H=1659:00 +CPU= 56.565 06:25:42 STEP 1660 H=1660:00 +CPU= 37.603 06:26:18 STEP 1661 H=1661:00 +CPU= 34.680 06:27:18 STEP 1662 H=1662:00 +CPU= 57.602 06:27:57 STEP 1663 H=1663:00 +CPU= 38.543 06:28:39 STEP 1664 H=1664:00 +CPU= 39.982 06:29:39 STEP 1665 H=1665:00 +CPU= 59.193 06:30:18 STEP 1666 H=1666:00 +CPU= 37.222 06:30:55 STEP 1667 H=1667:00 +CPU= 36.348 06:32:02 STEP 1668 H=1668:00 +CPU= 64.512 Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+001668 Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+001668 Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+001668 06:32:40 STEP 1669 H=1669:00 +CPU= 37.624 06:33:19 STEP 1670 H=1670:00 +CPU= 37.345 06:34:16 STEP 1671 H=1671:00 +CPU= 55.749 06:34:53 STEP 1672 H=1672:00 +CPU= 35.427 06:35:29 STEP 1673 H=1673:00 +CPU= 34.591 06:36:26 STEP 1674 H=1674:00 +CPU= 55.955 06:37:03 STEP 1675 H=1675:00 +CPU= 35.640 06:37:42 STEP 1676 H=1676:00 +CPU= 37.038 06:38:39 STEP 1677 H=1677:00 +CPU= 55.108 Suspend request received from the BOINC client, suspending the child process Resuming the child process 06:39:27 STEP 1678 H=1678:00 +CPU= 35.186 06:40:05 STEP 1679 H=1679:00 +CPU= 37.359 06:41:13 STEP 1680 H=1680:00 +CPU= 65.385 Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+001680 Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+001680 Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+001680 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12204269/ICMGGhq0f+001656 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12204269/ICMSHhq0f+001656 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12204269/ICMUAhq0f+001656 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12204269/ICMGGhq0f+001668 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12204269/ICMSHhq0f+001668 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12204269/ICMUAhq0f+001668 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0625_2018050100_123_987_12204269_2_r1542460059_38.zip Uploading the intermediate file: upload_file_38.zip 06:41:50 STEP 1681 H=1681:00 +CPU= 36.141 06:42:30 STEP 1682 H=1682:00 +CPU= 38.536 06:43:26 STEP 1683 H=1683:00 +CPU= 54.986 06:44:04 STEP 1684 H=1684:00 +CPU= 36.757 06:44:40 STEP 1685 H=1685:00 +CPU= 34.710 06:45:36 STEP 1686 H=1686:00 +CPU= 54.624 06:46:11 STEP 1687 H=1687:00 +CPU= 34.216 06:46:48 STEP 1688 H=1688:00 +CPU= 36.129 06:47:44 STEP 1689 H=1689:00 +CPU= 53.539 06:48:21 STEP 1690 H=1690:00 +CPU= 35.843 06:48:55 STEP 1691 H=1691:00 +CPU= 33.330 06:49:57 STEP 1692 H=1692:00 +CPU= 60.475 Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+001692 Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+001692 Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+001692 Suspend request received from the BOINC client, suspending the child process Resuming the child process 06:50:44 STEP 1693 H=1693:00 +CPU= 34.496 06:51:21 STEP 1694 H=1694:00 +CPU= 35.611 06:52:18 STEP 1695 H=1695:00 +CPU= 54.794 06:52:55 STEP 1696 H=1696:00 +CPU= 36.085 Suspend request received from the BOINC client, suspending the child process Resuming the child process 06:53:51 STEP 1697 H=1697:00 +CPU= 34.189 06:54:48 STEP 1698 H=1698:00 +CPU= 55.403 06:55:24 STEP 1699 H=1699:00 +CPU= 34.499 06:56:02 STEP 1700 H=1700:00 +CPU= 36.266 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 07:02:01 STEP 1701 H=1701:00 +CPU= 52.827 07:02:39 STEP 1702 H=1702:00 +CPU= 36.463 Suspend request received from the BOINC client, suspending the child process Resuming the child process 07:03:30 STEP 1703 H=1703:00 +CPU= 37.413 07:04:35 STEP 1704 H=1704:00 +CPU= 62.174 Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+001704 Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+001704 Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+001704 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12204269/ICMGGhq0f+001680 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12204269/ICMSHhq0f+001680 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12204269/ICMUAhq0f+001680 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12204269/ICMGGhq0f+001692 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12204269/ICMSHhq0f+001692 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12204269/ICMUAhq0f+001692 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0625_2018050100_123_987_12204269_2_r1542460059_39.zip Uploading the intermediate file: upload_file_39.zip 07:05:11 STEP 1705 H=1705:00 +CPU= 33.616 07:05:47 STEP 1706 H=1706:00 +CPU= 34.845 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 07:08:56 STEP 1707 H=1707:00 +CPU= 52.512 Suspend request received from the BOINC client, suspending the child process Resuming the child process 07:10:23 STEP 1708 H=1708:00 +CPU= 35.077 07:10:57 STEP 1709 H=1709:00 +CPU= 32.184 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 07:16:58 STEP 1710 H=1710:00 +CPU= 53.644 07:17:36 STEP 1711 H=1711:00 +CPU= 36.760 Suspend request received from the BOINC client, suspending the child process Resuming the child process 07:18:34 STEP 1712 H=1712:00 +CPU= 35.804 07:19:30 STEP 1713 H=1713:00 +CPU= 54.010 07:20:08 STEP 1714 H=1714:00 +CPU= 36.561 07:20:44 STEP 1715 H=1715:00 +CPU= 34.915 07:21:44 STEP 1716 H=1716:00 +CPU= 56.916 Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+001716 Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+001716 Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+001716 07:22:18 STEP 1717 H=1717:00 +CPU= 31.522 07:22:52 STEP 1718 H=1718:00 +CPU= 32.204 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 07:32:10 STEP 1719 H=1719:00 +CPU= 50.859 07:32:48 STEP 1720 H=1720:00 +CPU= 36.435 Suspend request received from the BOINC client, suspending the child process Resuming the child process 07:35:26 STEP 1721 H=1721:00 +CPU= 33.559 Suspend request received from the BOINC client, suspending the child process Resuming the child process 07:37:14 STEP 1722 H=1722:00 +CPU= 55.024 07:37:50 STEP 1723 H=1723:00 +CPU= 34.910 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 07:40:30 STEP 1724 H=1724:00 +CPU= 35.412 07:41:28 STEP 1725 H=1725:00 +CPU= 55.442 Suspend request received from the BOINC client, suspending the child process Resuming the child process 07:44:29 STEP 1726 H=1726:00 +CPU= 36.548 07:45:06 STEP 1727 H=1727:00 +CPU= 36.122 07:46:15 STEP 1728 H=1728:00 +CPU= 65.708 Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+001728 Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+001728 Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+001728 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12204269/ICMGGhq0f+001704 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12204269/ICMSHhq0f+001704 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12204269/ICMUAhq0f+001704 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12204269/ICMGGhq0f+001716 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12204269/ICMSHhq0f+001716 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12204269/ICMUAhq0f+001716 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0625_2018050100_123_987_12204269_2_r1542460059_40.zip Uploading the intermediate file: upload_file_40.zip Suspend request received from the BOINC client, suspending the child process Resuming the child process 07:47:01 STEP 1729 H=1729:00 +CPU= 33.006 07:47:34 STEP 1730 H=1730:00 +CPU= 30.547 07:48:24 STEP 1731 H=1731:00 +CPU= 46.382 07:48:57 STEP 1732 H=1732:00 +CPU= 31.069 07:49:30 STEP 1733 H=1733:00 +CPU= 31.515 07:50:22 STEP 1734 H=1734:00 +CPU= 48.168 07:50:55 STEP 1735 H=1735:00 +CPU= 30.777 07:51:30 STEP 1736 H=1736:00 +CPU= 33.338 07:52:21 STEP 1737 H=1737:00 +CPU= 47.528 Suspend request received from the BOINC client, suspending the child process Resuming the child process 07:56:46 STEP 1738 H=1738:00 +CPU= 30.764 07:57:18 STEP 1739 H=1739:00 +CPU= 29.903 07:58:18 STEP 1740 H=1740:00 +CPU= 56.553 Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+001740 Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+001740 Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+001740 07:58:52 STEP 1741 H=1741:00 +CPU= 31.909 07:59:27 STEP 1742 H=1742:00 +CPU= 33.108 08:00:20 STEP 1743 H=1743:00 +CPU= 49.932 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 08:08:51 STEP 1744 H=1744:00 +CPU= 32.664 08:09:26 STEP 1745 H=1745:00 +CPU= 33.169 08:10:20 STEP 1746 H=1746:00 +CPU= 50.347 08:10:55 STEP 1747 H=1747:00 +CPU= 33.516 08:11:32 STEP 1748 H=1748:00 +CPU= 35.288 08:12:25 STEP 1749 H=1749:00 +CPU= 49.980 08:13:00 STEP 1750 H=1750:00 +CPU= 32.689 08:13:35 STEP 1751 H=1751:00 +CPU= 32.259 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 08:20:05 STEP 1752 H=1752:00 +CPU= 59.080 Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+001752 Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+001752 Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+001752 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12204269/ICMGGhq0f+001728 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12204269/ICMSHhq0f+001728 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12204269/ICMUAhq0f+001728 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12204269/ICMGGhq0f+001740 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12204269/ICMSHhq0f+001740 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12204269/ICMUAhq0f+001740 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0625_2018050100_123_987_12204269_2_r1542460059_41.zip Uploading the intermediate file: upload_file_41.zip 08:20:39 STEP 1753 H=1753:00 +CPU= 32.516 08:21:14 STEP 1754 H=1754:00 +CPU= 32.250 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 08:26:42 STEP 1755 H=1755:00 +CPU= 49.788 Suspend request received from the BOINC client, suspending the child process Resuming the child process 08:27:50 STEP 1756 H=1756:00 +CPU= 33.810 08:28:23 STEP 1757 H=1757:00 +CPU= 31.198 08:29:15 STEP 1758 H=1758:00 +CPU= 49.795 08:29:48 STEP 1759 H=1759:00 +CPU= 31.041 08:30:22 STEP 1760 H=1760:00 +CPU= 33.352 08:31:16 STEP 1761 H=1761:00 +CPU= 51.541 08:31:50 STEP 1762 H=1762:00 +CPU= 33.839 08:32:26 STEP 1763 H=1763:00 +CPU= 34.478 08:33:27 STEP 1764 H=1764:00 +CPU= 58.792 Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+001764 Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+001764 Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+001764 08:34:02 STEP 1765 H=1765:00 +CPU= 33.186 08:34:37 STEP 1766 H=1766:00 +CPU= 33.938 08:35:36 STEP 1767 H=1767:00 +CPU= 57.876 08:36:14 STEP 1768 H=1768:00 +CPU= 36.324 08:36:52 STEP 1769 H=1769:00 +CPU= 37.110 08:37:50 STEP 1770 H=1770:00 +CPU= 56.591 08:38:26 STEP 1771 H=1771:00 +CPU= 34.048 08:39:02 STEP 1772 H=1772:00 +CPU= 35.030 08:39:57 STEP 1773 H=1773:00 +CPU= 53.066 08:40:33 STEP 1774 H=1774:00 +CPU= 34.673 08:41:10 STEP 1775 H=1775:00 +CPU= 36.064 Suspend request received from the BOINC client, suspending the child process Resuming the child process 08:42:27 STEP 1776 H=1776:00 +CPU= 63.293 Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+001776 Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+001776 Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+001776 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12204269/ICMGGhq0f+001752 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12204269/ICMSHhq0f+001752 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12204269/ICMUAhq0f+001752 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12204269/ICMGGhq0f+001764 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12204269/ICMSHhq0f+001764 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12204269/ICMUAhq0f+001764 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0625_2018050100_123_987_12204269_2_r1542460059_42.zip Uploading the intermediate file: upload_file_42.zip 08:43:02 STEP 1777 H=1777:00 +CPU= 33.767 Suspend request received from the BOINC client, suspending the child process Resuming the child process 08:43:50 STEP 1778 H=1778:00 +CPU= 35.990 Suspend request received from the BOINC client, suspending the child process Resuming the child process 08:45:38 STEP 1779 H=1779:00 +CPU= 53.664 08:46:14 STEP 1780 H=1780:00 +CPU= 34.765 08:46:50 STEP 1781 H=1781:00 +CPU= 34.375 08:47:47 STEP 1782 H=1782:00 +CPU= 54.430 Suspend request received from the BOINC client, suspending the child process Resuming the child process 08:49:04 STEP 1783 H=1783:00 +CPU= 33.383 08:49:40 STEP 1784 H=1784:00 +CPU= 33.925 08:50:37 STEP 1785 H=1785:00 +CPU= 54.243 08:51:13 STEP 1786 H=1786:00 +CPU= 33.408 08:51:48 STEP 1787 H=1787:00 +CPU= 34.121 08:52:49 STEP 1788 H=1788:00 +CPU= 58.016 Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+001788 Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+001788 Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+001788 08:53:24 STEP 1789 H=1789:00 +CPU= 33.151 08:54:01 STEP 1790 H=1790:00 +CPU= 35.235 08:54:55 STEP 1791 H=1791:00 +CPU= 52.247 08:55:33 STEP 1792 H=1792:00 +CPU= 35.646 08:56:08 STEP 1793 H=1793:00 +CPU= 33.686 08:57:07 STEP 1794 H=1794:00 +CPU= 56.040 08:57:42 STEP 1795 H=1795:00 +CPU= 33.923 Suspend request received from the BOINC client, suspending the child process Resuming the child process 08:58:31 STEP 1796 H=1796:00 +CPU= 36.407 Suspend request received from the BOINC client, suspending the child process Resuming the child process 08:59:40 STEP 1797 H=1797:00 +CPU= 55.396 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 09:00:48 STEP 1798 H=1798:00 +CPU= 34.821 09:01:26 STEP 1799 H=1799:00 +CPU= 36.271 Suspend request received from the BOINC client, suspending the child process Resuming the child process 09:02:41 STEP 1800 H=1800:00 +CPU= 61.512 Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+001800 Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+001800 Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+001800 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12204269/ICMGGhq0f+001776 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12204269/ICMSHhq0f+001776 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12204269/ICMUAhq0f+001776 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12204269/ICMGGhq0f+001788 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12204269/ICMSHhq0f+001788 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12204269/ICMUAhq0f+001788 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0625_2018050100_123_987_12204269_2_r1542460059_43.zip Uploading the intermediate file: upload_file_43.zip 09:03:17 STEP 1801 H=1801:00 +CPU= 34.152 09:03:52 STEP 1802 H=1802:00 +CPU= 33.567 09:04:47 STEP 1803 H=1803:00 +CPU= 52.268 09:05:24 STEP 1804 H=1804:00 +CPU= 34.699 09:06:00 STEP 1805 H=1805:00 +CPU= 34.995 09:06:57 STEP 1806 H=1806:00 +CPU= 54.231 09:07:30 STEP 1807 H=1807:00 +CPU= 30.775 09:08:03 STEP 1808 H=1808:00 +CPU= 30.920 09:08:55 STEP 1809 H=1809:00 +CPU= 48.352 09:09:28 STEP 1810 H=1810:00 +CPU= 30.836 09:10:01 STEP 1811 H=1811:00 +CPU= 30.402 09:10:59 STEP 1812 H=1812:00 +CPU= 54.248 Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+001812 Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+001812 Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+001812 09:11:31 STEP 1813 H=1813:00 +CPU= 30.316 SMILAG TRAJECTORY UNDERGROUND 1 TIMES. POINT= 2 MAX ETADOT VERTICAL VEL.= 3.341659593234334E-007 LON = 177.500000000000 degrees LAT = -35.3269057618723 degrees Suspend request received from the BOINC client, suspending the child process Resuming the child process 09:12:15 STEP 1814 H=1814:00 +CPU= 31.224 09:13:06 STEP 1815 H=1815:00 +CPU= 47.660 Suspend request received from the BOINC client, suspending the child process Resuming the child process SMILAG TRAJECTORY UNDERGROUND 1 TIMES. POINT= 3 MAX ETADOT VERTICAL VEL.= 4.335516291625997E-007 LON = 178.750000000000 degrees LAT = -36.4483927977945 degrees Suspend request received from the BOINC client, suspending the child process Resuming the child process 09:14:01 STEP 1816 H=1816:00 +CPU= 30.987 09:14:36 STEP 1817 H=1817:00 +CPU= 32.520 09:15:28 STEP 1818 H=1818:00 +CPU= 49.185 09:16:00 STEP 1819 H=1819:00 +CPU= 30.346 SMILAG TRAJECTORY UNDERGROUND 1 TIMES. POINT= 4 MAX ETADOT VERTICAL VEL.= 1.898447257395042E-007 LON = 180.000000000000 degrees LAT = -38.6913661222016 degrees 09:16:39 STEP 1820 H=1820:00 +CPU= 36.655 09:17:35 STEP 1821 H=1821:00 +CPU= 52.961 SMILAG TRAJECTORY UNDERGROUND 1 TIMES. POINT= 5 MAX ETADOT VERTICAL VEL.= 3.125846168893587E-008 LON = 178.750000000000 degrees LAT = -39.8128523737712 degrees 09:18:10 STEP 1822 H=1822:00 +CPU= 32.082 Suspend request received from the BOINC client, suspending the child process Resuming the child process 09:19:04 STEP 1823 H=1823:00 +CPU= 30.866 09:20:05 STEP 1824 H=1824:00 +CPU= 57.231 Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+001824 Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+001824 Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+001824 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12204269/ICMGGhq0f+001800 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12204269/ICMSHhq0f+001800 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12204269/ICMUAhq0f+001800 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12204269/ICMGGhq0f+001812 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12204269/ICMSHhq0f+001812 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12204269/ICMUAhq0f+001812 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0625_2018050100_123_987_12204269_2_r1542460059_44.zip Uploading the intermediate file: upload_file_44.zip 09:20:36 STEP 1825 H=1825:00 +CPU= 28.974 09:21:08 STEP 1826 H=1826:00 +CPU= 29.896 SMILAG TRAJECTORY UNDERGROUND 1 TIMES. POINT= 22 MAX ETADOT VERTICAL VEL.= 7.357212677974339E-008 LON = 177.187500000000 degrees LAT = -43.1773092288349 degrees 09:21:57 STEP 1827 H=1827:00 +CPU= 45.394 09:22:28 STEP 1828 H=1828:00 +CPU= 29.586 Suspend request received from the BOINC client, suspending the child process Resuming the child process 09:23:24 STEP 1829 H=1829:00 +CPU= 31.988 09:24:15 STEP 1830 H=1830:00 +CPU= 47.684 09:24:46 STEP 1831 H=1831:00 +CPU= 28.439 09:25:18 STEP 1832 H=1832:00 +CPU= 30.058 09:26:08 STEP 1833 H=1833:00 +CPU= 46.145 09:26:40 STEP 1834 H=1834:00 +CPU= 29.972 09:27:12 STEP 1835 H=1835:00 +CPU= 29.555 09:28:07 STEP 1836 H=1836:00 +CPU= 51.069 Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+001836 Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+001836 Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+001836 09:28:38 STEP 1837 H=1837:00 +CPU= 28.854 Suspend request received from the BOINC client, suspending the child process Resuming the child process 09:29:51 STEP 1838 H=1838:00 +CPU= 29.533 09:30:41 STEP 1839 H=1839:00 +CPU= 46.097 09:31:13 STEP 1840 H=1840:00 +CPU= 29.378 09:31:44 STEP 1841 H=1841:00 +CPU= 28.790 Suspend request received from the BOINC client, suspending the child process Resuming the child process 09:33:17 STEP 1842 H=1842:00 +CPU= 46.848 09:33:48 STEP 1843 H=1843:00 +CPU= 28.568 09:34:19 STEP 1844 H=1844:00 +CPU= 28.826 09:35:07 STEP 1845 H=1845:00 +CPU= 45.273 09:35:39 STEP 1846 H=1846:00 +CPU= 29.245 09:36:11 STEP 1847 H=1847:00 +CPU= 29.159 09:37:08 STEP 1848 H=1848:00 +CPU= 52.680 Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+001848 Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+001848 Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+001848 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12204269/ICMGGhq0f+001824 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12204269/ICMSHhq0f+001824 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12204269/ICMUAhq0f+001824 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12204269/ICMGGhq0f+001836 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12204269/ICMSHhq0f+001836 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12204269/ICMUAhq0f+001836 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0625_2018050100_123_987_12204269_2_r1542460059_45.zip Uploading the intermediate file: upload_file_45.zip 09:37:38 STEP 1849 H=1849:00 +CPU= 27.833 09:38:09 STEP 1850 H=1850:00 +CPU= 28.520 09:38:57 STEP 1851 H=1851:00 +CPU= 44.935 09:39:28 STEP 1852 H=1852:00 +CPU= 29.081 09:39:59 STEP 1853 H=1853:00 +CPU= 28.667 09:40:48 STEP 1854 H=1854:00 +CPU= 45.704 09:41:18 STEP 1855 H=1855:00 +CPU= 28.043 09:41:50 STEP 1856 H=1856:00 +CPU= 28.967 09:42:38 STEP 1857 H=1857:00 +CPU= 45.485 09:43:09 STEP 1858 H=1858:00 +CPU= 28.633 09:43:38 STEP 1859 H=1859:00 +CPU= 27.531 09:44:32 STEP 1860 H=1860:00 +CPU= 50.095 Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+001860 Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+001860 Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+001860 09:45:03 STEP 1861 H=1861:00 +CPU= 28.229 09:45:38 STEP 1862 H=1862:00 +CPU= 33.375 09:46:43 STEP 1863 H=1863:00 +CPU= 62.870 09:47:27 STEP 1864 H=1864:00 +CPU= 42.366 09:48:06 STEP 1865 H=1865:00 +CPU= 38.022 09:49:11 STEP 1866 H=1866:00 +CPU= 63.631 09:49:53 STEP 1867 H=1867:00 +CPU= 40.560 09:50:34 STEP 1868 H=1868:00 +CPU= 39.513 09:51:36 STEP 1869 H=1869:00 +CPU= 60.587 09:52:19 STEP 1870 H=1870:00 +CPU= 41.385 09:53:01 STEP 1871 H=1871:00 +CPU= 41.306 09:54:13 STEP 1872 H=1872:00 +CPU= 69.242 Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+001872 Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+001872 Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+001872 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12204269/ICMGGhq0f+001848 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12204269/ICMSHhq0f+001848 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12204269/ICMUAhq0f+001848 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12204269/ICMGGhq0f+001860 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12204269/ICMSHhq0f+001860 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12204269/ICMUAhq0f+001860 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0625_2018050100_123_987_12204269_2_r1542460059_46.zip Uploading the intermediate file: upload_file_46.zip Uploading trickle at timestep: 6739200 09:54:54 STEP 1873 H=1873:00 +CPU= 40.498 09:55:38 STEP 1874 H=1874:00 +CPU= 42.201 09:56:40 STEP 1875 H=1875:00 +CPU= 60.289 09:57:22 STEP 1876 H=1876:00 +CPU= 40.896 09:58:04 STEP 1877 H=1877:00 +CPU= 40.445 09:59:05 STEP 1878 H=1878:00 +CPU= 57.760 09:59:47 STEP 1879 H=1879:00 +CPU= 41.604 10:00:30 STEP 1880 H=1880:00 +CPU= 41.737 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 10:02:36 STEP 1881 H=1881:00 +CPU= 61.778 10:03:20 STEP 1882 H=1882:00 +CPU= 42.196 10:04:00 STEP 1883 H=1883:00 +CPU= 39.483 10:05:11 STEP 1884 H=1884:00 +CPU= 68.475 Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+001884 Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+001884 Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+001884 Suspend request received from the BOINC client, suspending the child process Resuming the child process 10:06:02 STEP 1885 H=1885:00 +CPU= 39.181 10:06:43 STEP 1886 H=1886:00 +CPU= 40.190 10:07:44 STEP 1887 H=1887:00 +CPU= 58.644 10:08:25 STEP 1888 H=1888:00 +CPU= 39.116 Quit request received from BOINC client, ending the child process (argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu (argv1) start_date: 2018050100 (argv2) exptid: hq0f (argv3) unique_member_id: 0625 (argv4) batchid: 987 (argv5) wuid: 12204269 (argv6) fclen: 123 (argv7) app_name: oifs_43r3_ps (argv8) nthreads: 1 Working directory is: /var/lib/boinc-client/slots/5 Project directory is: /var/lib/boinc-client/projects/climateprediction.net/ app name: oifs_43r3_ps version: 1.05 Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12204269 ..mkdir for temp folder for results failed Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/5/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip Unzipping the app zip file: /var/lib/boinc-client/slots/5/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip Copying the namelist files from: ../../projects/climateprediction.net/jf_be72bd4a10f5b751cb0baaa421dc53d9 to: /var/lib/boinc-client/slots/5/oifs_43r3_ps_0625_2018050100_123_987_12204269.zip Unzipping the namelist zip file: /var/lib/boinc-client/slots/5/oifs_43r3_ps_0625_2018050100_123_987_12204269.zip ic_ancil_file: ic_ancil_12204269 ifsdata_file: ifsdata_12204269 climate_data_file: clim_data_12204269 horiz_resolution: 159 vert_resolution: 60 grid_type: l_2 upload_interval: 24 utstep: 3600 nfrres: restart dump frequency (steps) 24 Copying IC ancils from: ../../projects/climateprediction.net/jf_21a5a86d5205f21a37b70596279cc24b to: /var/lib/boinc-client/slots/5/ic_ancil_12204269.zip Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/5/ic_ancil_12204269.zip ..mkdir for ifsdata folder failed Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_78825b502bbae3070cf3e0b5749ec435 to: /var/lib/boinc-client/slots/5/ifsdata/ifsdata_12204269.zip Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/5/ifsdata/ifsdata_12204269.zip ..mkdir for the climate data folder failed Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/5/159l_2/clim_data_12204269.zip Unzipping the climate data zip file: /var/lib/boinc-client/slots/5/159l_2/clim_data_12204269.zip Checking for progress XML file: /var/lib/boinc-client/slots/5/progress_file_12204269.xml Opened progress file ok : /var/lib/boinc-client/slots/5/progress_file_12204269.xml -- Model is restarting -- Adjusting last_iter, 1888, to previous model restart step. Creating progress file: /var/lib/boinc-client/slots/5/progress_file_12204269.xml last_cpu_time: 46282 upload_file_number: 47 last_iter: 1871 last_upload: 6739200 model_completed: 0 total_length_of_simulation: 10627200 result_base_name: oifs_43r3_ps_0625_2018050100_123_987_12204269_2_r1542460059 The child process has been launched with process id: 172516 Executing the command: /var/lib/boinc-client/slots/5/oifs_43r3_model.exe [EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1 [EC_DRHOOK:taman-jurong:1:1:172516:172516] [20230113:124337:1673585017:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0 [EC_DRHOOK:taman-jurong:1:1:172516:172516] [20230113:124337:1673585017:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1 [EC_DRHOOK:taman-jurong:1:1:172516:172516] [20230113:124337:1673585017:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0) [EC_DRHOOK:taman-jurong:1:1:172516:172516] [20230113:124337:1673585017:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1 [EC_DRHOOK:taman-jurong:1:1:172516:172516] [20230113:124337:1673585017:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000 [EC_DRHOOK:taman-jurong:1:1:172516:172516] [20230113:124337:1673585017:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647) [EC_DRHOOK:taman-jurong:1:1:172516:172516] [20230113:124337:1673585017:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16 [EC_DRHOOK:taman-jurong:1:1:172516:172516] [20230113:124337:1673585017:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0 [EC_DRHOOK:taman-jurong:1:1:172516:172516] [20230113:124337:1673585017:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500 [EC_DRHOOK:taman-jurong:1:1:172516:172516] [20230113:124337:1673585017:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1 [EC_DRHOOK:taman-jurong:1:1:172516:172516] [20230113:124337:1673585017:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1 [EC_DRHOOK:taman-jurong:1:1:172516:172516] [20230113:124337:1673585017:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1 [EC_DRHOOK:taman-jurong:1:1:172516:172516] [20230113:124337:1673585017:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1 [EC_DRHOOK:taman-jurong:1:1:172516:172516] [20230113:124337:1673585017:0.001] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef> [EC_DRHOOK:taman-jurong:1:1:172516:172516] [20230113:124337:1673585017:0.001] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef> [EC_DRHOOK:taman-jurong:1:1:172516:172516] [20230113:124337:1673585017:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:taman-jurong:1:1:172516:172516] [20230113:124337:1673585017:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil)) [EC_DRHOOK:taman-jurong:1:1:172516:172516] [20230113:124337:1673585017:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:taman-jurong:1:1:172516:172516] [20230113:124337:1673585017:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil)) [EC_DRHOOK:taman-jurong:1:1:172516:172516] [20230113:124337:1673585017:0.001] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1 [EC_DRHOOK:taman-jurong:1:1:172516:172516] [20230113:124337:1673585017:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:taman-jurong:1:1:172516:172516] [20230113:124337:1673585017:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:taman-jurong:1:1:172516:172516] [20230113:124337:1673585017:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil)) [EC_DRHOOK:taman-jurong:1:1:172516:172516] [20230113:124337:1673585017:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:taman-jurong:1:1:172516:172516] [20230113:124337:1673585017:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:taman-jurong:1:1:172516:172516] [20230113:124337:1673585017:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:taman-jurong:1:1:172516:172516] [20230113:124337:1673585017:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil)) [EC_DRHOOK:taman-jurong:1:1:172516:172516] [20230113:124337:1673585017:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil)) [EC_DRHOOK:taman-jurong:1:1:172516:172516] [20230113:124337:1673585017:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil)) [EC_DRHOOK:taman-jurong:1:1:172516:172516] [20230113:124337:1673585017:0.001] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef> 12:44:42 STEP 1872 H=1872:00 +CPU= 59.028 Suspend request received from the BOINC client, suspending the child process Resuming the child process 12:52:19 STEP 1873 H=1873:00 +CPU= 31.638 12:52:55 STEP 1874 H=1874:00 +CPU= 34.956 12:53:50 STEP 1875 H=1875:00 +CPU= 52.797 12:54:26 STEP 1876 H=1876:00 +CPU= 35.240 12:54:58 STEP 1877 H=1877:00 +CPU= 31.654 12:55:51 STEP 1878 H=1878:00 +CPU= 50.802 12:56:24 STEP 1879 H=1879:00 +CPU= 31.484 12:56:59 STEP 1880 H=1880:00 +CPU= 32.938 Suspend request received from the BOINC client, suspending the child process Resuming the child process 12:59:22 STEP 1881 H=1881:00 +CPU= 47.077 12:59:53 STEP 1882 H=1882:00 +CPU= 28.885 13:00:24 STEP 1883 H=1883:00 +CPU= 29.309 13:01:19 STEP 1884 H=1884:00 +CPU= 51.925 Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+001884 Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+001884 Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+001884 Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:02:22 STEP 1885 H=1885:00 +CPU= 29.528 13:02:56 STEP 1886 H=1886:00 +CPU= 31.462 13:03:47 STEP 1887 H=1887:00 +CPU= 48.266 13:04:20 STEP 1888 H=1888:00 +CPU= 30.613 13:04:53 STEP 1889 H=1889:00 +CPU= 30.934 13:05:44 STEP 1890 H=1890:00 +CPU= 47.624 13:06:16 STEP 1891 H=1891:00 +CPU= 29.436 13:06:49 STEP 1892 H=1892:00 +CPU= 30.445 13:07:40 STEP 1893 H=1893:00 +CPU= 46.852 13:08:12 STEP 1894 H=1894:00 +CPU= 30.546 Suspend request received from the BOINC client, suspending the child process Resuming the child process 13:09:57 STEP 1895 H=1895:00 +CPU= 30.685 13:10:58 STEP 1896 H=1896:00 +CPU= 58.543 Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+001896 Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+001896 Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+001896 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12204269/ICMGGhq0f+001872 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12204269/ICMSHhq0f+001872 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12204269/ICMUAhq0f+001872 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12204269/ICMGGhq0f+001884 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12204269/ICMSHhq0f+001884 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12204269/ICMUAhq0f+001884 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0625_2018050100_123_987_12204269_2_r1542460059_47.zip Uploading the intermediate file: upload_file_47.zip 13:11:32 STEP 1897 H=1897:00 +CPU= 31.639 Abort request received from BOINC client, ending the child process ..Failed, model did not complete successfully </stderr_txt> ]]> |
Latest Trickles Received | ||||||
---|---|---|---|---|---|---|
Time Sent (UTC) | Host ID | Result ID | Result Name | Timestep | CPU Time (sec) | Average (sec/TS) |
09 May 2023 16:17:30 | 1534673 | 22296787 | oifs_43r3_ps_0625_2018050100_123_987_12204269_2 | 6,739,200 | 45,535 | 0.0068 |
09 May 2023 16:17:22 | 1534673 | 22296787 | oifs_43r3_ps_0625_2018050100_123_987_12204269_2 | 5,875,200 | 35,431 | 0.0060 |
09 May 2023 16:17:22 | 1534673 | 22296787 | oifs_43r3_ps_0625_2018050100_123_987_12204269_2 | 5,875,200 | 35,431 | 0.0060 |
09 May 2023 16:17:22 | 1534673 | 22296787 | oifs_43r3_ps_0625_2018050100_123_987_12204269_2 | 5,011,200 | 25,913 | 0.0052 |
09 May 2023 16:16:05 | 1534673 | 22296787 | oifs_43r3_ps_0625_2018050100_123_987_12204269_2 | 1,728,000 | 18,064 | 0.0105 |
09 May 2023 16:16:05 | 1534673 | 22296787 | oifs_43r3_ps_0625_2018050100_123_987_12204269_2 | 864,000 | 9,095 | 0.0105 |
09 May 2023 16:16:05 | 1534673 | 22296787 | oifs_43r3_ps_0625_2018050100_123_987_12204269_2 | 864,000 | 9,095 | 0.0105 |
©2024 cpdn.org