climateprediction.net home page
Task 22448254

Task 22448254

Name oifs_43r3_bl_a4v8_2016092300_20_1018_12294635_0
Workunit 12294635
Created 13 Jun 2024, 5:08:26 UTC
Sent 13 Jun 2024, 17:58:57 UTC
Report deadline 12 Aug 2024, 17:58:57 UTC
Received 15 Jun 2024, 6:40:16 UTC
Server state Over
Outcome Success
Client state Done
Exit status 0 (0x00000000)
Computer ID 1537687
Run time 16 hours 15 min 11 sec
CPU time 15 hours 32 min 41 sec
Validate state Valid
Credit 2,635.55
Device peak FLOPS 7.20 GFLOPS
Application version OpenIFS 43r3 Baroclinic Lifecycle v1.13
x86_64-pc-linux-gnu
Peak working set size 5,020.99 MB
Peak swap size 5,981.13 MB
Peak disk usage 1,368.57 MB
Stderr
<core_client_version>7.18.1</core_client_version>
<![CDATA[
<stderr_txt>
cf8da0)
[EC_DRHOOK:box.zenlime.org:1:1:878624:878624] [20240615:023631:1718418991:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:box.zenlime.org:1:1:878624:878624] [20240615:023631:1718418991:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x820cf0 (old at (nil))
[EC_DRHOOK:box.zenlime.org:1:1:878624:878624] [20240615:023631:1718418991:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x820cf0 (old at (nil))
[EC_DRHOOK:box.zenlime.org:1:1:878624:878624] [20240615:023631:1718418991:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:box.zenlime.org:1:1:878624:878624] [20240615:023631:1718418991:0.000] [catch_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  02:37:00 STEP 1536 H= 384:00 +CPU= 26.098
  02:37:27 STEP 1537 H= 384:15 +CPU= 25.337
  02:37:51 STEP 1538 H= 384:30 +CPU= 24.440
  02:38:20 STEP 1539 H= 384:45 +CPU= 28.341
  02:38:51 STEP 1540 H= 385:00 +CPU= 30.228
  02:39:21 STEP 1541 H= 385:15 +CPU= 29.265
  02:39:51 STEP 1542 H= 385:30 +CPU= 29.169
  02:40:20 STEP 1543 H= 385:45 +CPU= 29.393
  02:40:50 STEP 1544 H= 386:00 +CPU= 29.532
  02:41:20 STEP 1545 H= 386:15 +CPU= 29.834
  02:41:50 STEP 1546 H= 386:30 +CPU= 29.609
  02:42:20 STEP 1547 H= 386:45 +CPU= 29.094
  02:42:57 STEP 1548 H= 387:00 +CPU= 36.620
Moving to projects directory: /var/lib/boinc-client/slots/21/ICMGGh7zg+001548
Moving to projects directory: /var/lib/boinc-client/slots/21/ICMSHh7zg+001548
Moving to projects directory: /var/lib/boinc-client/slots/21/ICMUAh7zg+001548
  02:43:26 STEP 1549 H= 387:15 +CPU= 29.398
  02:43:56 STEP 1550 H= 387:30 +CPU= 28.997
  02:44:25 STEP 1551 H= 387:45 +CPU= 29.323
  02:44:56 STEP 1552 H= 388:00 +CPU= 30.285
  02:45:25 STEP 1553 H= 388:15 +CPU= 29.424
  02:45:56 STEP 1554 H= 388:30 +CPU= 29.456
  02:46:24 STEP 1555 H= 388:45 +CPU= 27.438
  02:46:46 STEP 1556 H= 389:00 +CPU= 22.344
  02:47:07 STEP 1557 H= 389:15 +CPU= 20.990
  02:47:32 STEP 1558 H= 389:30 +CPU= 24.504
  02:47:58 STEP 1559 H= 389:45 +CPU= 25.819
  02:48:29 STEP 1560 H= 390:00 +CPU= 31.232
Moving to projects directory: /var/lib/boinc-client/slots/21/ICMGGh7zg+001560
Moving to projects directory: /var/lib/boinc-client/slots/21/ICMSHh7zg+001560
Moving to projects directory: /var/lib/boinc-client/slots/21/ICMUAh7zg+001560
  02:48:53 STEP 1561 H= 390:15 +CPU= 24.179
  02:49:18 STEP 1562 H= 390:30 +CPU= 24.896
  02:49:48 STEP 1563 H= 390:45 +CPU= 28.879
  02:50:18 STEP 1564 H= 391:00 +CPU= 29.469
  02:50:47 STEP 1565 H= 391:15 +CPU= 28.948
  02:51:16 STEP 1566 H= 391:30 +CPU= 29.292
  02:51:46 STEP 1567 H= 391:45 +CPU= 29.288
  02:52:12 STEP 1568 H= 392:00 +CPU= 26.371
  02:52:34 STEP 1569 H= 392:15 +CPU= 21.490
  02:52:56 STEP 1570 H= 392:30 +CPU= 21.822
  02:53:21 STEP 1571 H= 392:45 +CPU= 24.769
  02:53:50 STEP 1572 H= 393:00 +CPU= 28.830
Moving to projects directory: /var/lib/boinc-client/slots/21/ICMGGh7zg+001572
Moving to projects directory: /var/lib/boinc-client/slots/21/ICMSHh7zg+001572
Moving to projects directory: /var/lib/boinc-client/slots/21/ICMUAh7zg+001572
  02:54:16 STEP 1573 H= 393:15 +CPU= 26.181
  02:54:41 STEP 1574 H= 393:30 +CPU= 25.082
  02:55:08 STEP 1575 H= 393:45 +CPU= 26.165
  02:55:38 STEP 1576 H= 394:00 +CPU= 29.498
  02:56:08 STEP 1577 H= 394:15 +CPU= 28.560
  02:56:38 STEP 1578 H= 394:30 +CPU= 30.121
  02:57:09 STEP 1579 H= 394:45 +CPU= 29.530
  02:57:40 STEP 1580 H= 395:00 +CPU= 30.104
  02:58:10 STEP 1581 H= 395:15 +CPU= 28.922
  02:58:37 STEP 1582 H= 395:30 +CPU= 27.262
  02:58:58 STEP 1583 H= 395:45 +CPU= 20.550
  02:59:30 STEP 1584 H= 396:00 +CPU= 32.626
Moving to projects directory: /var/lib/boinc-client/slots/21/ICMGGh7zg+001584
Moving to projects directory: /var/lib/boinc-client/slots/21/ICMSHh7zg+001584
Moving to projects directory: /var/lib/boinc-client/slots/21/ICMUAh7zg+001584
  02:59:57 STEP 1585 H= 396:15 +CPU= 26.015
  03:00:21 STEP 1586 H= 396:30 +CPU= 24.854
  03:00:45 STEP 1587 H= 396:45 +CPU= 23.932
  03:01:12 STEP 1588 H= 397:00 +CPU= 26.068
  03:01:39 STEP 1589 H= 397:15 +CPU= 27.601
  03:02:09 STEP 1590 H= 397:30 +CPU= 28.977
  03:02:39 STEP 1591 H= 397:45 +CPU= 29.065
  03:03:09 STEP 1592 H= 398:00 +CPU= 29.800
  03:03:38 STEP 1593 H= 398:15 +CPU= 29.097
  03:04:08 STEP 1594 H= 398:30 +CPU= 29.535
  03:04:36 STEP 1595 H= 398:45 +CPU= 27.655
  03:05:05 STEP 1596 H= 399:00 +CPU= 29.111
Moving to projects directory: /var/lib/boinc-client/slots/21/ICMGGh7zg+001596
Moving to projects directory: /var/lib/boinc-client/slots/21/ICMSHh7zg+001596
Moving to projects directory: /var/lib/boinc-client/slots/21/ICMUAh7zg+001596
  03:05:29 STEP 1597 H= 399:15 +CPU= 23.862
  03:05:54 STEP 1598 H= 399:30 +CPU= 25.241
  03:06:21 STEP 1599 H= 399:45 +CPU= 26.991
  03:06:49 STEP 1600 H= 400:00 +CPU= 27.390
  03:07:14 STEP 1601 H= 400:15 +CPU= 24.866
  03:07:41 STEP 1602 H= 400:30 +CPU= 27.025
  03:08:10 STEP 1603 H= 400:45 +CPU= 29.217
  03:08:40 STEP 1604 H= 401:00 +CPU= 29.941
  03:09:11 STEP 1605 H= 401:15 +CPU= 29.404
  03:09:41 STEP 1606 H= 401:30 +CPU= 29.890
  03:10:07 STEP 1607 H= 401:45 +CPU= 26.573
  03:10:33 STEP 1608 H= 402:00 +CPU= 26.063
Moving to projects directory: /var/lib/boinc-client/slots/21/ICMGGh7zg+001608
Moving to projects directory: /var/lib/boinc-client/slots/21/ICMSHh7zg+001608
Moving to projects directory: /var/lib/boinc-client/slots/21/ICMUAh7zg+001608
  03:10:54 STEP 1609 H= 402:15 +CPU= 20.290
  03:11:17 STEP 1610 H= 402:30 +CPU= 23.030
  03:11:36 STEP 1611 H= 402:45 +CPU= 19.072
  03:11:59 STEP 1612 H= 403:00 +CPU= 22.453
  03:12:22 STEP 1613 H= 403:15 +CPU= 23.494
  03:12:48 STEP 1614 H= 403:30 +CPU= 25.826
  03:13:15 STEP 1615 H= 403:45 +CPU= 26.987
  03:13:45 STEP 1616 H= 404:00 +CPU= 29.680
  03:14:14 STEP 1617 H= 404:15 +CPU= 29.217
  03:14:44 STEP 1618 H= 404:30 +CPU= 29.357
  03:15:13 STEP 1619 H= 404:45 +CPU= 29.230
  03:15:49 STEP 1620 H= 405:00 +CPU= 35.611
Moving to projects directory: /var/lib/boinc-client/slots/21/ICMGGh7zg+001620
Moving to projects directory: /var/lib/boinc-client/slots/21/ICMSHh7zg+001620
Moving to projects directory: /var/lib/boinc-client/slots/21/ICMUAh7zg+001620
  03:16:19 STEP 1621 H= 405:15 +CPU= 28.696
  03:16:41 STEP 1622 H= 405:30 +CPU= 22.328
  03:17:03 STEP 1623 H= 405:45 +CPU= 21.715
  03:17:25 STEP 1624 H= 406:00 +CPU= 21.966
  03:17:51 STEP 1625 H= 406:15 +CPU= 26.038
  03:18:17 STEP 1626 H= 406:30 +CPU= 26.096
  03:18:43 STEP 1627 H= 406:45 +CPU= 26.170
  03:19:08 STEP 1628 H= 407:00 +CPU= 25.266
  03:19:36 STEP 1629 H= 407:15 +CPU= 27.252
  03:20:06 STEP 1630 H= 407:30 +CPU= 28.547
  03:20:35 STEP 1631 H= 407:45 +CPU= 29.159
  03:21:13 STEP 1632 H= 408:00 +CPU= 38.228
Moving to projects directory: /var/lib/boinc-client/slots/21/ICMGGh7zg+001632
Moving to projects directory: /var/lib/boinc-client/slots/21/ICMSHh7zg+001632
Moving to projects directory: /var/lib/boinc-client/slots/21/ICMUAh7zg+001632
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294635/ICMGGh7zg+001536
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294635/ICMSHh7zg+001536
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294635/ICMUAh7zg+001536
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294635/ICMGGh7zg+001548
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294635/ICMSHh7zg+001548
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294635/ICMUAh7zg+001548
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294635/ICMGGh7zg+001560
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294635/ICMSHh7zg+001560
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294635/ICMUAh7zg+001560
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294635/ICMGGh7zg+001572
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294635/ICMSHh7zg+001572
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294635/ICMUAh7zg+001572
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294635/ICMGGh7zg+001584
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294635/ICMSHh7zg+001584
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294635/ICMUAh7zg+001584
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294635/ICMGGh7zg+001596
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294635/ICMSHh7zg+001596
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294635/ICMUAh7zg+001596
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294635/ICMGGh7zg+001608
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294635/ICMSHh7zg+001608
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294635/ICMUAh7zg+001608
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294635/ICMGGh7zg+001620
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294635/ICMSHh7zg+001620
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294635/ICMUAh7zg+001620
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a4v8_2016092300_20_1018_12294635_0_r1928167450_16.zip
Uploading the intermediate file: upload_file_16.zip
  03:21:41 STEP 1633 H= 408:15 +CPU= 27.154
  03:22:06 STEP 1634 H= 408:30 +CPU= 25.714
  03:22:29 STEP 1635 H= 408:45 +CPU= 22.929
  03:22:49 STEP 1636 H= 409:00 +CPU= 19.544
  03:23:11 STEP 1637 H= 409:15 +CPU= 21.719
  03:23:33 STEP 1638 H= 409:30 +CPU= 22.733
  03:23:54 STEP 1639 H= 409:45 +CPU= 20.691
  03:24:15 STEP 1640 H= 410:00 +CPU= 21.363
  03:24:39 STEP 1641 H= 410:15 +CPU= 23.947
  03:25:03 STEP 1642 H= 410:30 +CPU= 23.139
  03:25:32 STEP 1643 H= 410:45 +CPU= 28.493
  03:26:09 STEP 1644 H= 411:00 +CPU= 36.266
Moving to projects directory: /var/lib/boinc-client/slots/21/ICMGGh7zg+001644
Moving to projects directory: /var/lib/boinc-client/slots/21/ICMSHh7zg+001644
Moving to projects directory: /var/lib/boinc-client/slots/21/ICMUAh7zg+001644
  03:26:38 STEP 1645 H= 411:15 +CPU= 29.364
  03:27:07 STEP 1646 H= 411:30 +CPU= 28.702
  03:27:36 STEP 1647 H= 411:45 +CPU= 29.360
  03:28:03 STEP 1648 H= 412:00 +CPU= 26.582
  03:28:22 STEP 1649 H= 412:15 +CPU= 19.004
  03:28:43 STEP 1650 H= 412:30 +CPU= 20.849
  03:29:07 STEP 1651 H= 412:45 +CPU= 23.994
  03:29:31 STEP 1652 H= 413:00 +CPU= 23.823
  03:29:53 STEP 1653 H= 413:15 +CPU= 21.756
  03:30:14 STEP 1654 H= 413:30 +CPU= 21.473
  03:30:41 STEP 1655 H= 413:45 +CPU= 26.994
  03:31:16 STEP 1656 H= 414:00 +CPU= 34.481
Moving to projects directory: /var/lib/boinc-client/slots/21/ICMGGh7zg+001656
Moving to projects directory: /var/lib/boinc-client/slots/21/ICMSHh7zg+001656
Moving to projects directory: /var/lib/boinc-client/slots/21/ICMUAh7zg+001656
  03:31:46 STEP 1657 H= 414:15 +CPU= 29.246
  03:32:15 STEP 1658 H= 414:30 +CPU= 29.459
  03:32:45 STEP 1659 H= 414:45 +CPU= 29.279
  03:33:15 STEP 1660 H= 415:00 +CPU= 29.956
  03:33:45 STEP 1661 H= 415:15 +CPU= 29.376
  03:34:14 STEP 1662 H= 415:30 +CPU= 28.799
  03:34:36 STEP 1663 H= 415:45 +CPU= 21.733
  03:35:00 STEP 1664 H= 416:00 +CPU= 24.140
  03:35:24 STEP 1665 H= 416:15 +CPU= 23.814
  03:35:49 STEP 1666 H= 416:30 +CPU= 25.350
  03:36:16 STEP 1667 H= 416:45 +CPU= 27.190
  03:36:48 STEP 1668 H= 417:00 +CPU= 31.342
Moving to projects directory: /var/lib/boinc-client/slots/21/ICMGGh7zg+001668
Moving to projects directory: /var/lib/boinc-client/slots/21/ICMSHh7zg+001668
Moving to projects directory: /var/lib/boinc-client/slots/21/ICMUAh7zg+001668
  03:37:14 STEP 1669 H= 417:15 +CPU= 26.339
  03:37:43 STEP 1670 H= 417:30 +CPU= 28.407
  03:38:14 STEP 1671 H= 417:45 +CPU= 29.122
  03:38:45 STEP 1672 H= 418:00 +CPU= 29.630
  03:39:15 STEP 1673 H= 418:15 +CPU= 29.374
  03:39:44 STEP 1674 H= 418:30 +CPU= 29.371
  03:40:13 STEP 1675 H= 418:45 +CPU= 29.248
  03:40:38 STEP 1676 H= 419:00 +CPU= 24.781
  03:41:04 STEP 1677 H= 419:15 +CPU= 25.868
  03:41:31 STEP 1678 H= 419:30 +CPU= 26.468
  03:41:56 STEP 1679 H= 419:45 +CPU= 24.543
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_bl_1.13_x86_64-pc-linux-gnu
(argv1) start_date: 2016092300
(argv2) exptid: h7zg
(argv3) unique_member_id: a4v8
(argv4) batchid: 1018
(argv5) wuid: 12294635
(argv6) fclen: 20
(argv7) app_name: oifs_43r3_bl
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/21
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_bl
version: 1.13
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294635
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/21/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/21/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_493e35416b989e241911c1281ea272f6 to: /var/lib/boinc-client/slots/21/oifs_43r3_bl_a4v8_2016092300_20_1018_12294635.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/21/oifs_43r3_bl_a4v8_2016092300_20_1018_12294635.zip
ic_ancil_file: ic_ancil_12294635
ifsdata_file: ifsdata_12294635
climate_data_file: clim_data_12294635
horiz_resolution: 159
vert_resolution: 91
grid_type: l_2
upload_interval: 96
utstep: 900
nfrres: restart dump frequency (steps) 48
Copying IC ancils from: ../../projects/climateprediction.net/jf_f7ecfd27aa1e68bbe7e8d19800e25c73 to: /var/lib/boinc-client/slots/21/ic_ancil_12294635.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/21/ic_ancil_12294635.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_45b2ea86ae86cae685a54d991dbb4255 to: /var/lib/boinc-client/slots/21/ifsdata/ifsdata_12294635.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/21/ifsdata/ifsdata_12294635.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/21/159l_2/clim_data_12294635.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/21/159l_2/clim_data_12294635.zip
Checking for progress XML file: /var/lib/boinc-client/slots/21/progress_file_12294635.xml
Opened progress file ok : /var/lib/boinc-client/slots/21/progress_file_12294635.xml
-- Model is restarting --
Adjusting last_iter, 1680, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/21/progress_file_12294635.xml
last_cpu_time: 49291
upload_file_number: 17
last_iter: 1681
last_upload: 1468800
model_completed: 0
total_length_of_simulation: 1728000
result_base_name: oifs_43r3_bl_a4v8_2016092300_20_1018_12294635_0_r1928167450
The child process has been launched with process id: 882746
Executing the command: /var/lib/boinc-client/slots/21/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:box.zenlime.org:1:1:882746:882746] [20240615:034309:1718422989:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2091] fp = 0x314f440
[EC_DRHOOK:box.zenlime.org:1:1:882746:882746] [20240615:034309:1718422989:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:box.zenlime.org:1:1:882746:882746] [20240615:034309:1718422989:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:box.zenlime.org:1:1:882746:882746] [20240615:034309:1718422989:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:box.zenlime.org:1:1:882746:882746] [20240615:034309:1718422989:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:box.zenlime.org:1:1:882746:882746] [20240615:034309:1718422989:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:box.zenlime.org:1:1:882746:882746] [20240615:034309:1718422989:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:box.zenlime.org:1:1:882746:882746] [20240615:034309:1718422989:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:box.zenlime.org:1:1:882746:882746] [20240615:034309:1718422989:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:box.zenlime.org:1:1:882746:882746] [20240615:034309:1718422989:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:box.zenlime.org:1:1:882746:882746] [20240615:034309:1718422989:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:box.zenlime.org:1:1:882746:882746] [20240615:034309:1718422989:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:box.zenlime.org:1:1:882746:882746] [20240615:034309:1718422989:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:box.zenlime.org:1:1:882746:882746] [20240615:034309:1718422989:0.000] [ignore_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:box.zenlime.org:1:1:882746:882746] [20240615:034309:1718422989:0.000] [restore_default_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:box.zenlime.org:1:1:882746:882746] [20240615:034309:1718422989:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:box.zenlime.org:1:1:882746:882746] [20240615:034309:1718422989:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:box.zenlime.org:1:1:882746:882746] [20240615:034309:1718422989:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:box.zenlime.org:1:1:882746:882746] [20240615:034309:1718422989:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x820cf0 (old at (nil))
[EC_DRHOOK:box.zenlime.org:1:1:882746:882746] [20240615:034309:1718422989:0.000] [trapfpe_treatment@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:box.zenlime.org:1:1:882746:882746] [20240615:034309:1718422989:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:box.zenlime.org:1:1:882746:882746] [20240615:034309:1718422989:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:box.zenlime.org:1:1:882746:882746] [20240615:034309:1718422989:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x820cf0 (old at (nil))
[EC_DRHOOK:box.zenlime.org:1:1:882746:882746] [20240615:034309:1718422989:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:box.zenlime.org:1:1:882746:882746] [20240615:034309:1718422989:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:box.zenlime.org:1:1:882746:882746] [20240615:034309:1718422989:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:box.zenlime.org:1:1:882746:882746] [20240615:034309:1718422989:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x820cf0 (old at (nil))
[EC_DRHOOK:box.zenlime.org:1:1:882746:882746] [20240615:034309:1718422989:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x820cf0 (old at (nil))
[EC_DRHOOK:box.zenlime.org:1:1:882746:882746] [20240615:034309:1718422989:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:box.zenlime.org:1:1:882746:882746] [20240615:034309:1718422989:0.000] [catch_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  03:43:41 STEP 1680 H= 420:00 +CPU= 28.676
Moving to projects directory: /var/lib/boinc-client/slots/21/ICMGGh7zg+001680
Moving to projects directory: /var/lib/boinc-client/slots/21/ICMSHh7zg+001680
Moving to projects directory: /var/lib/boinc-client/slots/21/ICMUAh7zg+001680
  03:44:10 STEP 1681 H= 420:15 +CPU= 28.869
  03:44:39 STEP 1682 H= 420:30 +CPU= 28.751
  03:45:08 STEP 1683 H= 420:45 +CPU= 29.188
  03:45:38 STEP 1684 H= 421:00 +CPU= 29.684
  03:46:06 STEP 1685 H= 421:15 +CPU= 28.783
  03:46:27 STEP 1686 H= 421:30 +CPU= 20.540
  03:46:47 STEP 1687 H= 421:45 +CPU= 20.266
  03:47:13 STEP 1688 H= 422:00 +CPU= 25.607
  03:47:38 STEP 1689 H= 422:15 +CPU= 24.724
  03:48:04 STEP 1690 H= 422:30 +CPU= 26.239
  03:48:31 STEP 1691 H= 422:45 +CPU= 26.479
  03:49:02 STEP 1692 H= 423:00 +CPU= 30.906
Moving to projects directory: /var/lib/boinc-client/slots/21/ICMGGh7zg+001692
Moving to projects directory: /var/lib/boinc-client/slots/21/ICMSHh7zg+001692
Moving to projects directory: /var/lib/boinc-client/slots/21/ICMUAh7zg+001692
  03:49:31 STEP 1693 H= 423:15 +CPU= 28.978
  03:50:01 STEP 1694 H= 423:30 +CPU= 29.660
  03:50:31 STEP 1695 H= 423:45 +CPU= 28.982
  03:51:02 STEP 1696 H= 424:00 +CPU= 30.094
  03:51:31 STEP 1697 H= 424:15 +CPU= 29.088
  03:52:01 STEP 1698 H= 424:30 +CPU= 29.087
  03:52:27 STEP 1699 H= 424:45 +CPU= 26.437
  03:52:53 STEP 1700 H= 425:00 +CPU= 25.581
  03:53:19 STEP 1701 H= 425:15 +CPU= 26.192
  03:53:46 STEP 1702 H= 425:30 +CPU= 26.625
  03:54:11 STEP 1703 H= 425:45 +CPU= 25.237
  03:54:43 STEP 1704 H= 426:00 +CPU= 32.114
Moving to projects directory: /var/lib/boinc-client/slots/21/ICMGGh7zg+001704
Moving to projects directory: /var/lib/boinc-client/slots/21/ICMSHh7zg+001704
Moving to projects directory: /var/lib/boinc-client/slots/21/ICMUAh7zg+001704
  03:55:08 STEP 1705 H= 426:15 +CPU= 24.289
  03:55:37 STEP 1706 H= 426:30 +CPU= 28.958
  03:56:06 STEP 1707 H= 426:45 +CPU= 29.312
  03:56:37 STEP 1708 H= 427:00 +CPU= 30.428
  03:57:07 STEP 1709 H= 427:15 +CPU= 29.044
  03:57:37 STEP 1710 H= 427:30 +CPU= 29.402
  03:58:06 STEP 1711 H= 427:45 +CPU= 28.319
  03:58:27 STEP 1712 H= 428:00 +CPU= 21.324
  03:58:53 STEP 1713 H= 428:15 +CPU= 25.860
  03:59:19 STEP 1714 H= 428:30 +CPU= 26.041
  03:59:45 STEP 1715 H= 428:45 +CPU= 26.143
  04:00:17 STEP 1716 H= 429:00 +CPU= 31.601
Moving to projects directory: /var/lib/boinc-client/slots/21/ICMGGh7zg+001716
Moving to projects directory: /var/lib/boinc-client/slots/21/ICMSHh7zg+001716
Moving to projects directory: /var/lib/boinc-client/slots/21/ICMUAh7zg+001716
  04:00:44 STEP 1717 H= 429:15 +CPU= 26.078
  04:01:09 STEP 1718 H= 429:30 +CPU= 25.096
  04:01:38 STEP 1719 H= 429:45 +CPU= 29.332
  04:02:10 STEP 1720 H= 430:00 +CPU= 30.185
  04:02:40 STEP 1721 H= 430:15 +CPU= 29.309
  04:03:09 STEP 1722 H= 430:30 +CPU= 29.646
  04:03:40 STEP 1723 H= 430:45 +CPU= 29.833
  04:04:11 STEP 1724 H= 431:00 +CPU= 29.941
  04:04:42 STEP 1725 H= 431:15 +CPU= 29.145
  04:05:12 STEP 1726 H= 431:30 +CPU= 29.750
  04:05:42 STEP 1727 H= 431:45 +CPU= 29.836
  04:06:21 STEP 1728 H= 432:00 +CPU= 38.480
Moving to projects directory: /var/lib/boinc-client/slots/21/ICMGGh7zg+001728
Moving to projects directory: /var/lib/boinc-client/slots/21/ICMSHh7zg+001728
Moving to projects directory: /var/lib/boinc-client/slots/21/ICMUAh7zg+001728
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294635/ICMGGh7zg+001632
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294635/ICMSHh7zg+001632
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294635/ICMUAh7zg+001632
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294635/ICMGGh7zg+001644
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294635/ICMSHh7zg+001644
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294635/ICMUAh7zg+001644
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294635/ICMGGh7zg+001656
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294635/ICMSHh7zg+001656
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294635/ICMUAh7zg+001656
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294635/ICMGGh7zg+001668
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294635/ICMSHh7zg+001668
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294635/ICMUAh7zg+001668
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294635/ICMGGh7zg+001680
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294635/ICMSHh7zg+001680
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294635/ICMUAh7zg+001680
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294635/ICMGGh7zg+001692
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294635/ICMSHh7zg+001692
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294635/ICMUAh7zg+001692
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294635/ICMGGh7zg+001704
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294635/ICMSHh7zg+001704
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294635/ICMUAh7zg+001704
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294635/ICMGGh7zg+001716
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294635/ICMSHh7zg+001716
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294635/ICMUAh7zg+001716
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a4v8_2016092300_20_1018_12294635_0_r1928167450_17.zip
Uploading the intermediate file: upload_file_17.zip
  04:06:47 STEP 1729 H= 432:15 +CPU= 25.632
  04:07:12 STEP 1730 H= 432:30 +CPU= 25.357
  04:07:38 STEP 1731 H= 432:45 +CPU= 25.827
  04:08:04 STEP 1732 H= 433:00 +CPU= 26.183
  04:08:31 STEP 1733 H= 433:15 +CPU= 26.361
  04:08:57 STEP 1734 H= 433:30 +CPU= 26.448
  04:09:23 STEP 1735 H= 433:45 +CPU= 25.116
  04:09:52 STEP 1736 H= 434:00 +CPU= 29.097
  04:10:21 STEP 1737 H= 434:15 +CPU= 28.744
  04:10:51 STEP 1738 H= 434:30 +CPU= 29.747
  04:11:21 STEP 1739 H= 434:45 +CPU= 28.609
  04:11:57 STEP 1740 H= 435:00 +CPU= 35.473
Moving to projects directory: /var/lib/boinc-client/slots/21/ICMGGh7zg+001740
Moving to projects directory: /var/lib/boinc-client/slots/21/ICMSHh7zg+001740
Moving to projects directory: /var/lib/boinc-client/slots/21/ICMUAh7zg+001740
  04:12:27 STEP 1741 H= 435:15 +CPU= 29.373
  04:12:56 STEP 1742 H= 435:30 +CPU= 29.001
  04:13:25 STEP 1743 H= 435:45 +CPU= 29.154
  04:13:56 STEP 1744 H= 436:00 +CPU= 29.478
  04:14:25 STEP 1745 H= 436:15 +CPU= 28.978
  04:14:55 STEP 1746 H= 436:30 +CPU= 29.043
  04:15:25 STEP 1747 H= 436:45 +CPU= 29.613
  04:15:55 STEP 1748 H= 437:00 +CPU= 29.672
  04:16:25 STEP 1749 H= 437:15 +CPU= 29.381
  04:16:54 STEP 1750 H= 437:30 +CPU= 29.282
  04:17:24 STEP 1751 H= 437:45 +CPU= 28.747
  04:17:59 STEP 1752 H= 438:00 +CPU= 35.047
Moving to projects directory: /var/lib/boinc-client/slots/21/ICMGGh7zg+001752
Moving to projects directory: /var/lib/boinc-client/slots/21/ICMSHh7zg+001752
Moving to projects directory: /var/lib/boinc-client/slots/21/ICMUAh7zg+001752
  04:18:21 STEP 1753 H= 438:15 +CPU= 21.903
  04:18:42 STEP 1754 H= 438:30 +CPU= 21.114
  04:19:04 STEP 1755 H= 438:45 +CPU= 21.140
  04:19:29 STEP 1756 H= 439:00 +CPU= 25.712
  04:19:55 STEP 1757 H= 439:15 +CPU= 25.392
  04:20:18 STEP 1758 H= 439:30 +CPU= 23.605
  04:20:45 STEP 1759 H= 439:45 +CPU= 26.344
  04:21:12 STEP 1760 H= 440:00 +CPU= 27.351
  04:21:42 STEP 1761 H= 440:15 +CPU= 29.145
  04:22:11 STEP 1762 H= 440:30 +CPU= 29.083
  04:22:40 STEP 1763 H= 440:45 +CPU= 28.941
  04:23:16 STEP 1764 H= 441:00 +CPU= 35.600
Moving to projects directory: /var/lib/boinc-client/slots/21/ICMGGh7zg+001764
Moving to projects directory: /var/lib/boinc-client/slots/21/ICMSHh7zg+001764
Moving to projects directory: /var/lib/boinc-client/slots/21/ICMUAh7zg+001764
  04:23:46 STEP 1765 H= 441:15 +CPU= 29.407
  04:24:09 STEP 1766 H= 441:30 +CPU= 23.912
  04:24:36 STEP 1767 H= 441:45 +CPU= 26.903
  04:25:02 STEP 1768 H= 442:00 +CPU= 25.454
  04:25:27 STEP 1769 H= 442:15 +CPU= 25.022
  04:25:54 STEP 1770 H= 442:30 +CPU= 25.909
  04:26:20 STEP 1771 H= 442:45 +CPU= 25.533
  04:26:46 STEP 1772 H= 443:00 +CPU= 26.088
  04:27:14 STEP 1773 H= 443:15 +CPU= 27.714
  04:27:43 STEP 1774 H= 443:30 +CPU= 28.828
  04:28:12 STEP 1775 H= 443:45 +CPU= 28.956
  04:28:51 STEP 1776 H= 444:00 +CPU= 38.846
Moving to projects directory: /var/lib/boinc-client/slots/21/ICMGGh7zg+001776
Moving to projects directory: /var/lib/boinc-client/slots/21/ICMSHh7zg+001776
Moving to projects directory: /var/lib/boinc-client/slots/21/ICMUAh7zg+001776
  04:29:22 STEP 1777 H= 444:15 +CPU= 29.511
  04:29:51 STEP 1778 H= 444:30 +CPU= 29.348
  04:30:19 STEP 1779 H= 444:45 +CPU= 27.568
  04:30:42 STEP 1780 H= 445:00 +CPU= 22.403
  04:31:02 STEP 1781 H= 445:15 +CPU= 20.245
  04:31:25 STEP 1782 H= 445:30 +CPU= 22.895
  04:31:49 STEP 1783 H= 445:45 +CPU= 24.173
  04:32:15 STEP 1784 H= 446:00 +CPU= 25.855
  04:32:40 STEP 1785 H= 446:15 +CPU= 25.155
  04:33:06 STEP 1786 H= 446:30 +CPU= 25.831
  04:33:34 STEP 1787 H= 446:45 +CPU= 27.448
  04:34:10 STEP 1788 H= 447:00 +CPU= 35.492
Moving to projects directory: /var/lib/boinc-client/slots/21/ICMGGh7zg+001788
Moving to projects directory: /var/lib/boinc-client/slots/21/ICMSHh7zg+001788
Moving to projects directory: /var/lib/boinc-client/slots/21/ICMUAh7zg+001788
  04:34:40 STEP 1789 H= 447:15 +CPU= 29.967
  04:35:09 STEP 1790 H= 447:30 +CPU= 28.882
  04:35:39 STEP 1791 H= 447:45 +CPU= 28.871
  04:36:10 STEP 1792 H= 448:00 +CPU= 30.371
  04:36:38 STEP 1793 H= 448:15 +CPU= 27.915
  04:36:57 STEP 1794 H= 448:30 +CPU= 19.306
  04:37:15 STEP 1795 H= 448:45 +CPU= 17.649
  04:37:34 STEP 1796 H= 449:00 +CPU= 19.484
  04:38:01 STEP 1797 H= 449:15 +CPU= 26.404
  04:38:26 STEP 1798 H= 449:30 +CPU= 25.551
  04:38:52 STEP 1799 H= 449:45 +CPU= 25.963
  04:39:23 STEP 1800 H= 450:00 +CPU= 30.540
Moving to projects directory: /var/lib/boinc-client/slots/21/ICMGGh7zg+001800
Moving to projects directory: /var/lib/boinc-client/slots/21/ICMSHh7zg+001800
Moving to projects directory: /var/lib/boinc-client/slots/21/ICMUAh7zg+001800
  04:39:52 STEP 1801 H= 450:15 +CPU= 28.388
  04:40:22 STEP 1802 H= 450:30 +CPU= 29.424
  04:40:52 STEP 1803 H= 450:45 +CPU= 29.753
  04:41:22 STEP 1804 H= 451:00 +CPU= 29.652
  04:41:51 STEP 1805 H= 451:15 +CPU= 29.227
  04:42:17 STEP 1806 H= 451:30 +CPU= 25.882
  04:42:38 STEP 1807 H= 451:45 +CPU= 20.985
  04:43:04 STEP 1808 H= 452:00 +CPU= 25.718
  04:43:31 STEP 1809 H= 452:15 +CPU= 26.647
  04:43:55 STEP 1810 H= 452:30 +CPU= 23.606
  04:44:22 STEP 1811 H= 452:45 +CPU= 26.669
  04:44:54 STEP 1812 H= 453:00 +CPU= 32.201
Moving to projects directory: /var/lib/boinc-client/slots/21/ICMGGh7zg+001812
Moving to projects directory: /var/lib/boinc-client/slots/21/ICMSHh7zg+001812
Moving to projects directory: /var/lib/boinc-client/slots/21/ICMUAh7zg+001812
  04:45:21 STEP 1813 H= 453:15 +CPU= 25.994
  04:45:50 STEP 1814 H= 453:30 +CPU= 28.859
  04:46:20 STEP 1815 H= 453:45 +CPU= 29.338
  04:46:51 STEP 1816 H= 454:00 +CPU= 29.986
  04:47:20 STEP 1817 H= 454:15 +CPU= 29.260
  04:47:50 STEP 1818 H= 454:30 +CPU= 28.845
  04:48:20 STEP 1819 H= 454:45 +CPU= 28.858
  04:48:44 STEP 1820 H= 455:00 +CPU= 23.848
  04:49:07 STEP 1821 H= 455:15 +CPU= 23.108
  04:49:33 STEP 1822 H= 455:30 +CPU= 26.318
  04:49:58 STEP 1823 H= 455:45 +CPU= 24.755
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_bl_1.13_x86_64-pc-linux-gnu
(argv1) start_date: 2016092300
(argv2) exptid: h7zg
(argv3) unique_member_id: a4v8
(argv4) batchid: 1018
(argv5) wuid: 12294635
(argv6) fclen: 20
(argv7) app_name: oifs_43r3_bl
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/21
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_bl
version: 1.13
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294635
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/21/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/21/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_493e35416b989e241911c1281ea272f6 to: /var/lib/boinc-client/slots/21/oifs_43r3_bl_a4v8_2016092300_20_1018_12294635.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/21/oifs_43r3_bl_a4v8_2016092300_20_1018_12294635.zip
ic_ancil_file: ic_ancil_12294635
ifsdata_file: ifsdata_12294635
climate_data_file: clim_data_12294635
horiz_resolution: 159
vert_resolution: 91
grid_type: l_2
upload_interval: 96
utstep: 900
nfrres: restart dump frequency (steps) 48
Copying IC ancils from: ../../projects/climateprediction.net/jf_f7ecfd27aa1e68bbe7e8d19800e25c73 to: /var/lib/boinc-client/slots/21/ic_ancil_12294635.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/21/ic_ancil_12294635.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_45b2ea86ae86cae685a54d991dbb4255 to: /var/lib/boinc-client/slots/21/ifsdata/ifsdata_12294635.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/21/ifsdata/ifsdata_12294635.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/21/159l_2/clim_data_12294635.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/21/159l_2/clim_data_12294635.zip
Checking for progress XML file: /var/lib/boinc-client/slots/21/progress_file_12294635.xml
Opened progress file ok : /var/lib/boinc-client/slots/21/progress_file_12294635.xml
-- Model is restarting --
Adjusting last_iter, 1824, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/21/progress_file_12294635.xml
last_cpu_time: 53259
upload_file_number: 18
last_iter: 1825
last_upload: 1555200
model_completed: 0
total_length_of_simulation: 1728000
result_base_name: oifs_43r3_bl_a4v8_2016092300_20_1018_12294635_0_r1928167450
The child process has been launched with process id: 885388
Executing the command: /var/lib/boinc-client/slots/21/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:box.zenlime.org:1:1:885388:885388] [20240615:045110:1718427070:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2091] fp = 0x314f440
[EC_DRHOOK:box.zenlime.org:1:1:885388:885388] [20240615:045110:1718427070:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:box.zenlime.org:1:1:885388:885388] [20240615:045110:1718427070:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:box.zenlime.org:1:1:885388:885388] [20240615:045110:1718427070:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:box.zenlime.org:1:1:885388:885388] [20240615:045110:1718427070:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:box.zenlime.org:1:1:885388:885388] [20240615:045110:1718427070:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:box.zenlime.org:1:1:885388:885388] [20240615:045110:1718427070:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:box.zenlime.org:1:1:885388:885388] [20240615:045110:1718427070:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:box.zenlime.org:1:1:885388:885388] [20240615:045110:1718427070:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:box.zenlime.org:1:1:885388:885388] [20240615:045110:1718427070:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:box.zenlime.org:1:1:885388:885388] [20240615:045110:1718427070:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:box.zenlime.org:1:1:885388:885388] [20240615:045110:1718427070:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:box.zenlime.org:1:1:885388:885388] [20240615:045110:1718427070:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:box.zenlime.org:1:1:885388:885388] [20240615:045110:1718427070:0.000] [ignore_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:box.zenlime.org:1:1:885388:885388] [20240615:045110:1718427070:0.000] [restore_default_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:box.zenlime.org:1:1:885388:885388] [20240615:045110:1718427070:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:box.zenlime.org:1:1:885388:885388] [20240615:045110:1718427070:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:box.zenlime.org:1:1:885388:885388] [20240615:045110:1718427070:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:box.zenlime.org:1:1:885388:885388] [20240615:045110:1718427070:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x820cf0 (old at (nil))
[EC_DRHOOK:box.zenlime.org:1:1:885388:885388] [20240615:045110:1718427070:0.000] [trapfpe_treatment@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:box.zenlime.org:1:1:885388:885388] [20240615:045110:1718427070:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:box.zenlime.org:1:1:885388:885388] [20240615:045110:1718427070:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:box.zenlime.org:1:1:885388:885388] [20240615:045110:1718427070:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x820cf0 (old at (nil))
[EC_DRHOOK:box.zenlime.org:1:1:885388:885388] [20240615:045110:1718427070:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:box.zenlime.org:1:1:885388:885388] [20240615:045110:1718427070:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:box.zenlime.org:1:1:885388:885388] [20240615:045110:1718427070:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:box.zenlime.org:1:1:885388:885388] [20240615:045110:1718427070:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x820cf0 (old at (nil))
[EC_DRHOOK:box.zenlime.org:1:1:885388:885388] [20240615:045110:1718427070:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x820cf0 (old at (nil))
[EC_DRHOOK:box.zenlime.org:1:1:885388:885388] [20240615:045110:1718427070:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:box.zenlime.org:1:1:885388:885388] [20240615:045110:1718427070:0.000] [catch_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294635/ICMGGh7zg+001728
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294635/ICMSHh7zg+001728
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294635/ICMUAh7zg+001728
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294635/ICMGGh7zg+001740
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294635/ICMSHh7zg+001740
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294635/ICMUAh7zg+001740
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294635/ICMGGh7zg+001752
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294635/ICMSHh7zg+001752
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294635/ICMUAh7zg+001752
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294635/ICMGGh7zg+001764
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294635/ICMSHh7zg+001764
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294635/ICMUAh7zg+001764
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294635/ICMGGh7zg+001776
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294635/ICMSHh7zg+001776
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294635/ICMUAh7zg+001776
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294635/ICMGGh7zg+001788
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294635/ICMSHh7zg+001788
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294635/ICMUAh7zg+001788
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294635/ICMGGh7zg+001800
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294635/ICMSHh7zg+001800
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294635/ICMUAh7zg+001800
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294635/ICMGGh7zg+001812
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294635/ICMSHh7zg+001812
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294635/ICMUAh7zg+001812
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a4v8_2016092300_20_1018_12294635_0_r1928167450_18.zip
Uploading the intermediate file: upload_file_18.zip
  04:51:42 STEP 1824 H= 456:00 +CPU= 29.078
Moving to projects directory: /var/lib/boinc-client/slots/21/ICMGGh7zg+001824
Moving to projects directory: /var/lib/boinc-client/slots/21/ICMSHh7zg+001824
Moving to projects directory: /var/lib/boinc-client/slots/21/ICMUAh7zg+001824
  04:52:13 STEP 1825 H= 456:15 +CPU= 29.680
  04:52:43 STEP 1826 H= 456:30 +CPU= 28.957
  04:53:12 STEP 1827 H= 456:45 +CPU= 29.050
  04:53:42 STEP 1828 H= 457:00 +CPU= 29.207
  04:54:11 STEP 1829 H= 457:15 +CPU= 28.991
  04:54:40 STEP 1830 H= 457:30 +CPU= 29.333
  04:55:10 STEP 1831 H= 457:45 +CPU= 29.500
  04:55:40 STEP 1832 H= 458:00 +CPU= 30.347
  04:56:10 STEP 1833 H= 458:15 +CPU= 29.086
  04:56:39 STEP 1834 H= 458:30 +CPU= 28.421
  04:57:09 STEP 1835 H= 458:45 +CPU= 29.394
  04:57:44 STEP 1836 H= 459:00 +CPU= 34.998
Moving to projects directory: /var/lib/boinc-client/slots/21/ICMGGh7zg+001836
Moving to projects directory: /var/lib/boinc-client/slots/21/ICMSHh7zg+001836
Moving to projects directory: /var/lib/boinc-client/slots/21/ICMUAh7zg+001836
  04:58:14 STEP 1837 H= 459:15 +CPU= 29.126
  04:58:43 STEP 1838 H= 459:30 +CPU= 28.417
  04:59:12 STEP 1839 H= 459:45 +CPU= 28.916
  04:59:41 STEP 1840 H= 460:00 +CPU= 29.154
  05:00:04 STEP 1841 H= 460:15 +CPU= 22.056
  05:00:25 STEP 1842 H= 460:30 +CPU= 21.665
  05:00:50 STEP 1843 H= 460:45 +CPU= 25.029
  05:01:18 STEP 1844 H= 461:00 +CPU= 27.472
  05:01:42 STEP 1845 H= 461:15 +CPU= 24.095
  05:02:06 STEP 1846 H= 461:30 +CPU= 23.862
  05:02:31 STEP 1847 H= 461:45 +CPU= 24.965
  05:03:06 STEP 1848 H= 462:00 +CPU= 34.828
Moving to projects directory: /var/lib/boinc-client/slots/21/ICMGGh7zg+001848
Moving to projects directory: /var/lib/boinc-client/slots/21/ICMSHh7zg+001848
Moving to projects directory: /var/lib/boinc-client/slots/21/ICMUAh7zg+001848
  05:03:35 STEP 1849 H= 462:15 +CPU= 29.071
  05:04:04 STEP 1850 H= 462:30 +CPU= 29.072
  05:04:34 STEP 1851 H= 462:45 +CPU= 29.483
  05:05:04 STEP 1852 H= 463:00 +CPU= 29.797
  05:05:34 STEP 1853 H= 463:15 +CPU= 29.496
  05:06:04 STEP 1854 H= 463:30 +CPU= 29.318
  05:06:34 STEP 1855 H= 463:45 +CPU= 29.293
  05:07:03 STEP 1856 H= 464:00 +CPU= 28.825
  05:07:32 STEP 1857 H= 464:15 +CPU= 29.227
  05:08:02 STEP 1858 H= 464:30 +CPU= 28.897
  05:08:31 STEP 1859 H= 464:45 +CPU= 28.797
  05:09:08 STEP 1860 H= 465:00 +CPU= 35.787
Moving to projects directory: /var/lib/boinc-client/slots/21/ICMGGh7zg+001860
Moving to projects directory: /var/lib/boinc-client/slots/21/ICMSHh7zg+001860
Moving to projects directory: /var/lib/boinc-client/slots/21/ICMUAh7zg+001860
  05:09:37 STEP 1861 H= 465:15 +CPU= 29.443
  05:10:07 STEP 1862 H= 465:30 +CPU= 29.075
  05:10:37 STEP 1863 H= 465:45 +CPU= 29.426
  05:11:07 STEP 1864 H= 466:00 +CPU= 29.564
  05:11:36 STEP 1865 H= 466:15 +CPU= 28.997
  05:12:00 STEP 1866 H= 466:30 +CPU= 23.299
  05:12:20 STEP 1867 H= 466:45 +CPU= 20.820
  05:12:41 STEP 1868 H= 467:00 +CPU= 20.544
  05:13:07 STEP 1869 H= 467:15 +CPU= 26.190
  05:13:34 STEP 1870 H= 467:30 +CPU= 26.663
  05:14:00 STEP 1871 H= 467:45 +CPU= 25.858
  05:14:34 STEP 1872 H= 468:00 +CPU= 33.484
Moving to projects directory: /var/lib/boinc-client/slots/21/ICMGGh7zg+001872
Moving to projects directory: /var/lib/boinc-client/slots/21/ICMSHh7zg+001872
Moving to projects directory: /var/lib/boinc-client/slots/21/ICMUAh7zg+001872
  05:15:00 STEP 1873 H= 468:15 +CPU= 26.194
  05:15:31 STEP 1874 H= 468:30 +CPU= 30.100
  05:16:00 STEP 1875 H= 468:45 +CPU= 28.841
  05:16:30 STEP 1876 H= 469:00 +CPU= 29.892
  05:17:00 STEP 1877 H= 469:15 +CPU= 29.139
  05:17:29 STEP 1878 H= 469:30 +CPU= 28.813
  05:17:59 STEP 1879 H= 469:45 +CPU= 29.513
  05:18:28 STEP 1880 H= 470:00 +CPU= 29.201
  05:18:57 STEP 1881 H= 470:15 +CPU= 28.964
  05:19:27 STEP 1882 H= 470:30 +CPU= 30.057
  05:19:57 STEP 1883 H= 470:45 +CPU= 29.047
  05:20:35 STEP 1884 H= 471:00 +CPU= 36.336
Moving to projects directory: /var/lib/boinc-client/slots/21/ICMGGh7zg+001884
Moving to projects directory: /var/lib/boinc-client/slots/21/ICMSHh7zg+001884
Moving to projects directory: /var/lib/boinc-client/slots/21/ICMUAh7zg+001884
  05:21:04 STEP 1885 H= 471:15 +CPU= 29.202
  05:21:34 STEP 1886 H= 471:30 +CPU= 29.283
  05:22:04 STEP 1887 H= 471:45 +CPU= 29.764
  05:22:34 STEP 1888 H= 472:00 +CPU= 29.904
  05:23:00 STEP 1889 H= 472:15 +CPU= 25.265
  05:23:20 STEP 1890 H= 472:30 +CPU= 20.670
  05:23:42 STEP 1891 H= 472:45 +CPU= 22.042
  05:24:08 STEP 1892 H= 473:00 +CPU= 25.184
  05:24:29 STEP 1893 H= 473:15 +CPU= 21.007
  05:24:52 STEP 1894 H= 473:30 +CPU= 22.864
  05:25:18 STEP 1895 H= 473:45 +CPU= 26.386
  05:25:49 STEP 1896 H= 474:00 +CPU= 30.106
Moving to projects directory: /var/lib/boinc-client/slots/21/ICMGGh7zg+001896
Moving to projects directory: /var/lib/boinc-client/slots/21/ICMSHh7zg+001896
Moving to projects directory: /var/lib/boinc-client/slots/21/ICMUAh7zg+001896
  05:26:18 STEP 1897 H= 474:15 +CPU= 28.860
  05:26:48 STEP 1898 H= 474:30 +CPU= 29.360
  05:27:17 STEP 1899 H= 474:45 +CPU= 29.309
  05:27:48 STEP 1900 H= 475:00 +CPU= 29.903
  05:28:17 STEP 1901 H= 475:15 +CPU= 28.840
  05:28:46 STEP 1902 H= 475:30 +CPU= 28.902
  05:29:12 STEP 1903 H= 475:45 +CPU= 25.978
  05:29:32 STEP 1904 H= 476:00 +CPU= 19.826
  05:29:52 STEP 1905 H= 476:15 +CPU= 19.517
  05:30:11 STEP 1906 H= 476:30 +CPU= 18.961
  05:30:37 STEP 1907 H= 476:45 +CPU= 26.820
  05:31:09 STEP 1908 H= 477:00 +CPU= 31.058
Moving to projects directory: /var/lib/boinc-client/slots/21/ICMGGh7zg+001908
Moving to projects directory: /var/lib/boinc-client/slots/21/ICMSHh7zg+001908
Moving to projects directory: /var/lib/boinc-client/slots/21/ICMUAh7zg+001908
  05:31:36 STEP 1909 H= 477:15 +CPU= 26.945
  05:32:00 STEP 1910 H= 477:30 +CPU= 24.006
  05:32:27 STEP 1911 H= 477:45 +CPU= 26.881
  05:32:57 STEP 1912 H= 478:00 +CPU= 29.593
  05:33:26 STEP 1913 H= 478:15 +CPU= 29.625
  05:33:56 STEP 1914 H= 478:30 +CPU= 29.551
  05:34:26 STEP 1915 H= 478:45 +CPU= 29.431
  05:34:56 STEP 1916 H= 479:00 +CPU= 29.171
  05:35:22 STEP 1917 H= 479:15 +CPU= 25.600
  05:35:43 STEP 1918 H= 479:30 +CPU= 20.503
  05:36:06 STEP 1919 H= 479:45 +CPU= 22.875
  05:36:40 STEP 1920 H= 480:00 +CPU= 33.633
..The child process terminated with status: 0
>>> Printing last 70 lines from file: NODE.001_01

 
DDH------------------------------------------------------------------------
I                                                                         I
I TIME =  480.000H DDH * Diagnostic Var.,     AREA                GLOBAL  I
I                                                                         I
I-------------------------------------------------------------------------I
I   VARIABLE   I      Initial value         I        Current value        I
I-------------------------------------------------------------------------I
I  OMEGA (P/S) I     -0.1648525298E-20      I       0.8135434206E-06      I
ICloud fractionI      0.0000000000E+00      I       0.2998078947E-01      I
I  Relat. Hum. I      0.1097042352E+00      I       0.2143266130E+00      I
I  PASS 01     I      0.0000000000E+00      I       0.3201319502E+03      I
DDH------------------------------------------------------------------------
  
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
 NSTEP =  1920 SCAN2M_HPOS  P
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ICMGGh7zg+001920 MODE=a
 IO-STREAM CLOSED - ICMGGh7zg+001920
 MPI-TASK:   1 -    321010740 BYTES IN      7 RECORDS TRANSFERRED IN    0.0004 SECONDS802526.8 Mbytes/s, TOTAL TIME=    0.0090( 4.4%)
 05:36:40 STEP 1920 H= 480:00 +CPU= 33.633
 END CNT3
 NSTEP =  1920 CNT0   000000000  25.083  25.083  25.129   0.998               0               0               0               0               0
IO-STREAM STATISTICS, TOTAL NO OF REC/BYTES READ           46/  1069125708 WRITTEN         2860/************
 -TASK-OPENED-OPEN-RECS IN -KBYTE IN    -RECS OUT-KBYTE OUT-WALL   -WALL IN-WALL
  OU-TOT IN -TOT OUT
   1     53    0       46      1044068     2860 ************     5.2     1.2     4.0     1.9     8.6
===-=== START OF TIMING STATISTICS ===-===
 
STATS FOR ALL TASKS
 NUM ROUTINE                                     CALLS  MEAN(ms)   MAX(ms)   FRAC(%)  UNBAL(%)
   0 CNT0     - COMPLETE EXECUTION                   1 2730050.9 2730050.9    100.00      0.00
   1 CNT4     - FORWARD INTEGRATION                  1 2726758.9 2726758.9     99.88      0.00
   8 SCAN2M - GRID-POINT DYNAMICS                  105    5208.1    5208.1     20.03      0.00
   9 SPCM     - SPECTRAL COMP.                      96     909.1     909.1      3.20      0.00
  10 SCAN2M - PHYSICS                               97   12290.6   12290.6     43.67      0.00
  11 IOPACK   - OUTPUT P.P. RESULTS                  8     732.0     732.0      0.21      0.00
  12 SPNORM   - SPECTRAL NORM COMP.                  5      78.5      78.5      0.01      0.00
  17 GRIDFPOS IN CNT4                                8      27.8      27.8      0.01      0.00
  27 RESTART FILES - WRITING                         2    2924.0    2924.0      0.21      0.00
  28 RESTART FILES - READING                         1    2819.5    2819.5      0.10      0.00
  29 SU4FPOS IN CNT4                                 8       0.1       0.1      0.00      0.00
  30 DYNFPOS IN CNT4                                 8    5499.5    5499.5      1.61      0.00
  31 POSDDH IN STEPO                                 1      10.1      10.1      0.00      0.00
  37 CPGLAG   - SL COMPUTATIONS                     97    5845.1    5845.1     20.77      0.00
  39 SU0YOMB                                         1     276.4     276.4      0.01      0.00
  51 SCAN2M   - SL COMM. PART 1                     97      28.6      28.6      0.10      0.00
  54 SPCM     - M TO S/S TO M TRANSP.               96     543.7     543.7      1.91      0.00
  55 SPCIMPF  - S TO M/M TO S TRANSP.               96      68.3      68.3      0.24      0.00
  56 SPNORM   - SPECTRAL NORM COMM.                  5       0.3       0.3      0.00      0.00
 102 LTINV_CTL   - INVERSE LEGENDRE TRANSFORM      212     388.1     388.1      3.01      0.00
 103 LTDIR_CTL   - DIRECT LEGENDRE TRANSFORM       200     226.5     226.5      1.66      0.00
 106 FTDIR_CTL   - DIRECT FOURIER TRANSFORM        200      74.8      74.8      0.55      0.00
 107 FTINV_CTL   - INVERSE FOURIER TRANSFORM       212     107.1     107.1      0.83      0.00
 140 SULEG       - COMP. OF LEGENDRE POL.            1      32.6      32.6      0.00      0.00
 152 LTINV_CTL   - M TO L TRANSPOSITION            212      63.0      63.0      0.49      0.00
 153 LTDIR_CTL   - L TO M TRANSPOSITION            200      35.9      35.9      0.26      0.00
 157 FTINV_CTL   - L TO G TRANSPOSITION            212     126.9     126.9      0.99      0.00
 158 FTDIR_CTL   - G TO L TRANSPOSITION            200      58.8      58.8      0.43      0.00
 400 GSTATS                                      12215       0.0       0.0      0.00      0.00
 401 GSTATS HOOK                                 11311       0.0       0.0      0.00      0.00
TOTAL MEASURED IMBALANCE =       0.0 SECONDS,  0.0 PERCENT
TOTAL WALLCLOCK TIME     2730.1 CPU TIME    2702.3 VECTOR TIME     2702.3


===-=== END   OF TIMING STATISTICS ===-===
FORECAST DAYS PER DAY      633.7
  *** END CNT0 *** 
------------------------------------------------
Moving to projects directory: /var/lib/boinc-client/slots/21/ICMGGh7zg+001920
Moving to projects directory: /var/lib/boinc-client/slots/21/ICMSHh7zg+001920
Moving to projects directory: /var/lib/boinc-client/slots/21/ICMUAh7zg+001920
Adding to the zip: /var/lib/boinc-client/slots/21/NODE.001_01
Adding to the zip: /var/lib/boinc-client/slots/21/ifs.stat
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294635/ICMGGh7zg+001884
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294635/ICMGGh7zg+001824
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294635/ICMSHh7zg+001920
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294635/ICMSHh7zg+001884
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294635/ICMUAh7zg+001884
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294635/ICMUAh7zg+001872
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294635/ICMGGh7zg+001896
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294635/ICMUAh7zg+001896
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294635/ICMUAh7zg+001908
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294635/ICMUAh7zg+001860
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294635/ICMSHh7zg+001824
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294635/ICMGGh7zg+001908
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294635/ICMSHh7zg+001908
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294635/ICMUAh7zg+001836
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294635/ICMSHh7zg+001848
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294635/ICMGGh7zg+001848
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294635/ICMGGh7zg+001920
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294635/ICMGGh7zg+001872
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294635/ICMSHh7zg+001860
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294635/ICMSHh7zg+001872
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294635/ICMSHh7zg+001896
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294635/ICMUAh7zg+001824
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294635/ICMUAh7zg+001920
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294635/ICMGGh7zg+001836
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294635/ICMGGh7zg+001860
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294635/ICMSHh7zg+001836
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12294635/ICMUAh7zg+001848
Zipping up the final file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a4v8_2016092300_20_1018_12294635_0_r1928167450_19.zip
Uploading the final file: upload_file_19.zip
Uploading trickle at timestep: 1727100
05:40:08 (885385): called boinc_finish(0)

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
15 Jun 2024 05:39:19 1537687 22448254 oifs_43r3_bl_a4v8_2016092300_20_1018_12294635_0 1,727,100 55,961 0.0324


©2024 cpdn.org