climateprediction.net home page
Task 22443417

Task 22443417

Name oifs_43r3_bl_a15q_2016092300_20_1018_12289829_0
Workunit 12289829
Created 12 Jun 2024, 22:08:14 UTC
Sent 14 Jun 2024, 9:07:27 UTC
Report deadline 13 Aug 2024, 9:07:27 UTC
Received 15 Jun 2024, 9:49:26 UTC
Server state Over
Outcome Success
Client state Done
Exit status 0 (0x00000000)
Computer ID 1541000
Run time 14 hours 54 min 10 sec
CPU time 14 hours 52 min 25 sec
Validate state Valid
Credit 2,635.55
Device peak FLOPS 4.91 GFLOPS
Application version OpenIFS 43r3 Baroclinic Lifecycle v1.13
x86_64-pc-linux-gnu
Peak working set size 5,023.55 MB
Peak swap size 5,981.07 MB
Peak disk usage 1,408.41 MB
Stderr
<core_client_version>8.0.2</core_client_version>
<![CDATA[
<stderr_txt>
1.951
  13:57:25 STEP 1512 H= 378:00 +CPU= 52.580
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001512
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001512
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001512
  13:58:07 STEP 1513 H= 378:15 +CPU= 41.880
  13:58:49 STEP 1514 H= 378:30 +CPU= 41.851
  13:59:31 STEP 1515 H= 378:45 +CPU= 41.791
  14:00:13 STEP 1516 H= 379:00 +CPU= 42.460
  14:00:55 STEP 1517 H= 379:15 +CPU= 41.839
  14:01:37 STEP 1518 H= 379:30 +CPU= 41.873
  14:02:19 STEP 1519 H= 379:45 +CPU= 41.756
  14:03:02 STEP 1520 H= 380:00 +CPU= 42.504
  14:03:43 STEP 1521 H= 380:15 +CPU= 41.818
  14:04:25 STEP 1522 H= 380:30 +CPU= 41.839
  14:05:07 STEP 1523 H= 380:45 +CPU= 41.913
  14:06:00 STEP 1524 H= 381:00 +CPU= 52.552
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001524
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001524
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001524
  14:06:42 STEP 1525 H= 381:15 +CPU= 41.871
  14:07:24 STEP 1526 H= 381:30 +CPU= 41.893
  14:08:06 STEP 1527 H= 381:45 +CPU= 41.928
  14:08:48 STEP 1528 H= 382:00 +CPU= 42.559
  14:09:30 STEP 1529 H= 382:15 +CPU= 41.882
  14:10:12 STEP 1530 H= 382:30 +CPU= 41.904
  14:10:54 STEP 1531 H= 382:45 +CPU= 41.897
  14:11:37 STEP 1532 H= 383:00 +CPU= 42.524
  14:12:19 STEP 1533 H= 383:15 +CPU= 41.985
  14:13:01 STEP 1534 H= 383:30 +CPU= 41.958
  14:13:43 STEP 1535 H= 383:45 +CPU= 41.885
  14:14:41 STEP 1536 H= 384:00 +CPU= 58.187
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001536
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001536
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001536
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMGGh7zg+001440
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMSHh7zg+001440
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMUAh7zg+001440
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMGGh7zg+001452
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMSHh7zg+001452
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMUAh7zg+001452
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMGGh7zg+001464
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMSHh7zg+001464
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMUAh7zg+001464
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMGGh7zg+001476
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMSHh7zg+001476
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMUAh7zg+001476
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMGGh7zg+001488
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMSHh7zg+001488
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMUAh7zg+001488
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMGGh7zg+001500
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMSHh7zg+001500
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMUAh7zg+001500
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMGGh7zg+001512
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMSHh7zg+001512
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMUAh7zg+001512
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMGGh7zg+001524
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMSHh7zg+001524
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMUAh7zg+001524
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a15q_2016092300_20_1018_12289829_0_r1746830500_15.zip
Uploading the intermediate file: upload_file_15.zip
Uploading trickle at timestep: 1382400
  14:15:23 STEP 1537 H= 384:15 +CPU= 42.021
  14:16:05 STEP 1538 H= 384:30 +CPU= 41.946
  14:16:47 STEP 1539 H= 384:45 +CPU= 41.960
  14:17:30 STEP 1540 H= 385:00 +CPU= 42.600
  14:18:12 STEP 1541 H= 385:15 +CPU= 41.947
  14:18:54 STEP 1542 H= 385:30 +CPU= 41.936
  14:19:36 STEP 1543 H= 385:45 +CPU= 41.912
  14:20:18 STEP 1544 H= 386:00 +CPU= 42.568
  14:21:00 STEP 1545 H= 386:15 +CPU= 41.873
  14:21:42 STEP 1546 H= 386:30 +CPU= 41.963
  14:22:24 STEP 1547 H= 386:45 +CPU= 41.944
  14:23:17 STEP 1548 H= 387:00 +CPU= 52.744
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001548
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001548
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001548
  14:23:59 STEP 1549 H= 387:15 +CPU= 41.849
  14:24:41 STEP 1550 H= 387:30 +CPU= 42.043
  14:25:23 STEP 1551 H= 387:45 +CPU= 41.855
  14:26:05 STEP 1552 H= 388:00 +CPU= 42.447
  14:26:47 STEP 1553 H= 388:15 +CPU= 41.830
  14:27:29 STEP 1554 H= 388:30 +CPU= 41.855
  14:28:11 STEP 1555 H= 388:45 +CPU= 41.804
  14:28:54 STEP 1556 H= 389:00 +CPU= 42.487
  14:29:35 STEP 1557 H= 389:15 +CPU= 41.829
  14:30:17 STEP 1558 H= 389:30 +CPU= 41.780
  14:30:59 STEP 1559 H= 389:45 +CPU= 41.812
  14:31:52 STEP 1560 H= 390:00 +CPU= 52.808
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001560
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001560
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001560
  14:32:34 STEP 1561 H= 390:15 +CPU= 41.909
  14:33:16 STEP 1562 H= 390:30 +CPU= 41.930
  14:33:58 STEP 1563 H= 390:45 +CPU= 41.986
  14:34:41 STEP 1564 H= 391:00 +CPU= 42.573
  14:35:23 STEP 1565 H= 391:15 +CPU= 41.834
  14:36:05 STEP 1566 H= 391:30 +CPU= 41.926
  14:36:47 STEP 1567 H= 391:45 +CPU= 41.915
  14:37:29 STEP 1568 H= 392:00 +CPU= 42.544
  14:38:11 STEP 1569 H= 392:15 +CPU= 41.838
  14:38:53 STEP 1570 H= 392:30 +CPU= 41.877
  14:39:35 STEP 1571 H= 392:45 +CPU= 41.871
  14:40:28 STEP 1572 H= 393:00 +CPU= 52.669
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001572
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001572
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001572
  14:41:09 STEP 1573 H= 393:15 +CPU= 41.767
  14:41:51 STEP 1574 H= 393:30 +CPU= 41.912
  14:42:33 STEP 1575 H= 393:45 +CPU= 41.947
  14:43:16 STEP 1576 H= 394:00 +CPU= 42.626
  14:43:58 STEP 1577 H= 394:15 +CPU= 42.026
  14:44:40 STEP 1578 H= 394:30 +CPU= 42.019
  14:45:22 STEP 1579 H= 394:45 +CPU= 42.077
  14:46:05 STEP 1580 H= 395:00 +CPU= 42.662
  14:46:47 STEP 1581 H= 395:15 +CPU= 42.016
  14:47:29 STEP 1582 H= 395:30 +CPU= 42.070
  14:48:11 STEP 1583 H= 395:45 +CPU= 41.968
  14:49:09 STEP 1584 H= 396:00 +CPU= 58.060
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001584
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001584
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001584
  14:49:51 STEP 1585 H= 396:15 +CPU= 42.024
  14:50:34 STEP 1586 H= 396:30 +CPU= 42.030
  14:51:16 STEP 1587 H= 396:45 +CPU= 42.068
  14:51:58 STEP 1588 H= 397:00 +CPU= 42.677
  14:52:40 STEP 1589 H= 397:15 +CPU= 41.898
  14:53:22 STEP 1590 H= 397:30 +CPU= 41.980
  14:54:04 STEP 1591 H= 397:45 +CPU= 41.945
  14:54:47 STEP 1592 H= 398:00 +CPU= 42.649
  14:55:29 STEP 1593 H= 398:15 +CPU= 42.056
  14:56:11 STEP 1594 H= 398:30 +CPU= 41.993
  14:56:53 STEP 1595 H= 398:45 +CPU= 41.952
  14:57:46 STEP 1596 H= 399:00 +CPU= 52.742
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001596
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001596
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001596
  14:58:28 STEP 1597 H= 399:15 +CPU= 41.819
  14:59:10 STEP 1598 H= 399:30 +CPU= 41.889
  14:59:52 STEP 1599 H= 399:45 +CPU= 41.939
  15:00:09 STEP 1600 H= 400:00 +CPU= 17.520
  15:00:27 STEP 1601 H= 400:15 +CPU= 17.907
  15:00:45 STEP 1602 H= 400:30 +CPU= 17.562
  15:01:03 STEP 1603 H= 400:45 +CPU= 18.172
  15:01:20 STEP 1604 H= 401:00 +CPU= 17.417
  15:01:38 STEP 1605 H= 401:15 +CPU= 17.474
  15:01:56 STEP 1606 H= 401:30 +CPU= 17.634
  15:02:13 STEP 1607 H= 401:45 +CPU= 17.901
  15:02:36 STEP 1608 H= 402:00 +CPU= 22.647
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001608
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001608
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001608
  15:02:54 STEP 1609 H= 402:15 +CPU= 18.279
  15:03:13 STEP 1610 H= 402:30 +CPU= 18.626
  15:03:31 STEP 1611 H= 402:45 +CPU= 18.028
  15:03:49 STEP 1612 H= 403:00 +CPU= 18.263
  15:04:07 STEP 1613 H= 403:15 +CPU= 17.932
  15:04:24 STEP 1614 H= 403:30 +CPU= 16.994
  15:04:41 STEP 1615 H= 403:45 +CPU= 16.710
  15:04:58 STEP 1616 H= 404:00 +CPU= 16.968
  15:05:15 STEP 1617 H= 404:15 +CPU= 16.629
  15:05:31 STEP 1618 H= 404:30 +CPU= 16.664
  15:05:48 STEP 1619 H= 404:45 +CPU= 16.646
  15:06:09 STEP 1620 H= 405:00 +CPU= 21.008
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001620
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001620
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001620
  15:06:26 STEP 1621 H= 405:15 +CPU= 16.777
  15:06:44 STEP 1622 H= 405:30 +CPU= 17.973
  15:07:02 STEP 1623 H= 405:45 +CPU= 17.890
  15:07:19 STEP 1624 H= 406:00 +CPU= 17.657
  15:07:37 STEP 1625 H= 406:15 +CPU= 17.428
  15:07:55 STEP 1626 H= 406:30 +CPU= 17.770
  15:08:14 STEP 1627 H= 406:45 +CPU= 18.783
  15:08:31 STEP 1628 H= 407:00 +CPU= 17.486
  15:08:49 STEP 1629 H= 407:15 +CPU= 17.790
  15:09:06 STEP 1630 H= 407:30 +CPU= 17.480
  15:09:24 STEP 1631 H= 407:45 +CPU= 17.450
  15:09:49 STEP 1632 H= 408:00 +CPU= 25.636
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001632
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001632
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001632
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMGGh7zg+001536
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMSHh7zg+001536
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMUAh7zg+001536
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMGGh7zg+001548
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMSHh7zg+001548
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMUAh7zg+001548
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMGGh7zg+001560
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMSHh7zg+001560
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMUAh7zg+001560
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMGGh7zg+001572
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMSHh7zg+001572
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMUAh7zg+001572
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMGGh7zg+001584
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMSHh7zg+001584
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMUAh7zg+001584
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMGGh7zg+001596
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMSHh7zg+001596
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMUAh7zg+001596
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMGGh7zg+001608
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMSHh7zg+001608
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMUAh7zg+001608
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMGGh7zg+001620
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMSHh7zg+001620
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMUAh7zg+001620
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a15q_2016092300_20_1018_12289829_0_r1746830500_16.zip
Uploading the intermediate file: upload_file_16.zip
  15:10:07 STEP 1633 H= 408:15 +CPU= 17.872
Suspend request received from the BOINC client, suspending the child process
[EC_DRHOOK:Rig-05:1:1:2977:2977] [20240614:153930:1718404770:34497.868] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1538] Received signal#15 (SIGTERM) :: 3958MB (heap), 4797MB (maxrss), 0MB (maxstack), 0 (paging), nsigs = 1
[EC_DRHOOK:Rig-05:1:1:2977:2977] [20240614:153930:1718404770:34497.868] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1542] Also activating Harakiri-alarm (SIGALRM=14) to expire after 500s elapsed to prevent hangs, nsigs = 1
[EC_DRHOOK:Rig-05:1:1:2977:2977] [20240614:153930:1718404770:34497.868] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1544] Harakiri signal handler 'signal_harakiri' for signal#14 (SIGALRM) installed at 0x820890 (old at (nil))
[EC_DRHOOK:Rig-05:1:1:2977:2977] [20240614:153930:1718404770:34497.868] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1617] Signal#15 was caused by unrecognized si_code [memaddr=0x1], nsigs = 1
[EC_DRHOOK:Rig-05:1:1:2977:2977] [20240614:153930:1718404770:34497.868] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1686] Starting DrHook backtrace for signal#15, nsigs = 1
[EC_DRHOOK:Rig-05:1:1:2977:2977] [20240614:153930:1718404770:34497.868] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3843] 3958 MB (maxheap), 4797 MB (maxrss), 0 MB (maxstack)
[EC_DRHOOK:Rig-05:1:1:2977:2977] [20240614:153930:1718404770:34497.868] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :  MASTER 
[EC_DRHOOK:Rig-05:1:1:2977:2977] [20240614:153930:1718404770:34497.868] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :   CNT0 
[EC_DRHOOK:Rig-05:1:1:2977:2977] [20240614:153930:1718404770:34497.868] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :    CNT1 
[EC_DRHOOK:Rig-05:1:1:2977:2977] [20240614:153930:1718404770:34497.868] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :     CNT2 
[EC_DRHOOK:Rig-05:1:1:2977:2977] [20240614:153930:1718404770:34497.868] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :      CNT3 
[EC_DRHOOK:Rig-05:1:1:2977:2977] [20240614:153930:1718404770:34497.868] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :       CNT4 
[EC_DRHOOK:Rig-05:1:1:2977:2977] [20240614:153930:1718404770:34497.868] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :        STEPO 
[EC_DRHOOK:Rig-05:1:1:2977:2977] [20240614:153930:1718404770:34497.868] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :         SCAN2M 
[EC_DRHOOK:Rig-05:1:1:2977:2977] [20240614:153930:1718404770:34497.868] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :          GP_MODEL_HEAP 
[EC_DRHOOK:Rig-05:1:1:2977:2977] [20240614:153930:1718404770:34497.868] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :           GP_MODEL 
[EC_DRHOOK:Rig-05:1:1:2977:2977] [20240614:153930:1718404770:34497.868] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :            EC_PHYS_DRV 
[EC_DRHOOK:Rig-05:1:1:2977:2977] [20240614:153930:1718404770:34497.868] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :             >OMP-PHYSICS              (1001) 
[EC_DRHOOK:Rig-05:1:1:2977:2977] [20240614:153930:1718404770:34497.868] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :              EC_PHYS 
[EC_DRHOOK:Rig-05:1:1:2977:2977] [20240614:153930:1718404770:34497.868] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :               CALLPAR 
[EC_DRHOOK:Rig-05:1:1:2977:2977] [20240614:153930:1718404770:34497.868] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :                CONVECTION_LAYER 
[EC_DRHOOK:Rig-05:1:1:2977:2977] [20240614:153930:1718404770:34497.868] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :                 CUCALLN 
[EC_DRHOOK:Rig-05:1:1:2977:2977] [20240614:153930:1718404770:34497.868] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :                  CUMASTRN 
[EC_DRHOOK:Rig-05:1:1:2977:2977] [20240614:153930:1718404770:34497.868] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :                   CUASCN 
[EC_DRHOOK:Rig-05:1:1:2977:2977] [20240614:153931:1718404771:34498.868] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1734] DrHook backtrace done for signal#15, nsigs = 1
[EC_DRHOOK:Rig-05:1:1:2977:2977] [20240614:153931:1718404771:34498.868] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1785] Calling previous signal handler at 0x1cf8da0 for signal#15, nsigs = 1
forrtl: error (78): process killed (SIGTERM)
Image              PC                Routine            Line        Source             
oifs_43r3_model.e  0000000001CF935B  Unknown               Unknown  Unknown
oifs_43r3_model.e  00000000008214C1  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001DD5490  Unknown               Unknown  Unknown
oifs_43r3_model.e  00000000011F0F9E  cuascn_                   680  cuascn.F90
oifs_43r3_model.e  00000000011D5A22  cumastrn_                 537  cumastrn.F90
oifs_43r3_model.e  00000000011D02C2  cucalln_                  352  cucalln.F90
oifs_43r3_model.e  00000000011CD745  convection_layer_         121  convection_layer.F90
oifs_43r3_model.e  0000000000FB288B  callpar_                 1064  callpar.F90
oifs_43r3_model.e  0000000000F5FA6B  ec_phys_                  670  ec_phys.F90
oifs_43r3_model.e  0000000000F4DBC4  ec_phys_drv_              599  ec_phys_drv.F90
oifs_43r3_model.e  0000000000BD7131  gp_model_                 613  gp_model.F90
oifs_43r3_model.e  00000000012C5002  gp_model_heap_             74  gp_model_heap.F90
oifs_43r3_model.e  0000000000BACBAE  scan2m_                   535  scan2m.F90
oifs_43r3_model.e  0000000000688153  stepo_                    327  stepo.F90
oifs_43r3_model.e  00000000004CD1DC  cnt4_                    1133  cnt4.F90
oifs_43r3_model.e  00000000004B5D85  cnt3_                     267  cnt3.F90
oifs_43r3_model.e  00000000004B4F52  cnt2_                      88  cnt2.F90
oifs_43r3_model.e  000000000049D638  cnt1_                      92  cnt1.F90
oifs_43r3_model.e  0000000000406FBD  cnt0_                     146  cnt0.F90
oifs_43r3_model.e  0000000000401E7F  MAIN__                     96  master.F90
oifs_43r3_model.e  0000000000401E12  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001DCC34A  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001DCDBE7  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000000401CD5  Unknown               Unknown  Unknown
(argv0) ../../projects/climateprediction.net/oifs_43r3_bl_1.13_x86_64-pc-linux-gnu
(argv1) start_date: 2016092300
(argv2) exptid: h7zg
(argv3) unique_member_id: a15q
(argv4) batchid: 1018
(argv5) wuid: 12289829
(argv6) fclen: 20
(argv7) app_name: oifs_43r3_bl
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/6
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_bl
version: 1.13
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/6/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/6/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_9a6c963b1260c730250177a7ff0c1d63 to: /var/lib/boinc-client/slots/6/oifs_43r3_bl_a15q_2016092300_20_1018_12289829.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/6/oifs_43r3_bl_a15q_2016092300_20_1018_12289829.zip
ic_ancil_file: ic_ancil_12289829
ifsdata_file: ifsdata_12289829
climate_data_file: clim_data_12289829
horiz_resolution: 159
vert_resolution: 91
grid_type: l_2
upload_interval: 96
utstep: 900
nfrres: restart dump frequency (steps) 48
Copying IC ancils from: ../../projects/climateprediction.net/jf_f7ecfd27aa1e68bbe7e8d19800e25c73 to: /var/lib/boinc-client/slots/6/ic_ancil_12289829.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/6/ic_ancil_12289829.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_a9f34953505f2897976b4d4d4e8af87a to: /var/lib/boinc-client/slots/6/ifsdata/ifsdata_12289829.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/6/ifsdata/ifsdata_12289829.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/6/159l_2/clim_data_12289829.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/6/159l_2/clim_data_12289829.zip
Checking for progress XML file: /var/lib/boinc-client/slots/6/progress_file_12289829.xml
Opened progress file ok : /var/lib/boinc-client/slots/6/progress_file_12289829.xml
-- Model is restarting --
Adjusting last_iter, 1633, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/6/progress_file_12289829.xml
last_cpu_time: 41978
upload_file_number: 17
last_iter: 1633
last_upload: 1468800
model_completed: 0
total_length_of_simulation: 1728000
result_base_name: oifs_43r3_bl_a15q_2016092300_20_1018_12289829_0_r1746830500
The child process has been launched with process id: 3368
Executing the command: /var/lib/boinc-client/slots/6/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:Rig-05:1:1:3368:3368] [20240614:224619:1718430379:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2091] fp = 0x314f440
[EC_DRHOOK:Rig-05:1:1:3368:3368] [20240614:224619:1718430379:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:Rig-05:1:1:3368:3368] [20240614:224619:1718430379:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:Rig-05:1:1:3368:3368] [20240614:224619:1718430379:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:Rig-05:1:1:3368:3368] [20240614:224619:1718430379:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:Rig-05:1:1:3368:3368] [20240614:224619:1718430379:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:Rig-05:1:1:3368:3368] [20240614:224619:1718430379:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:Rig-05:1:1:3368:3368] [20240614:224619:1718430379:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:Rig-05:1:1:3368:3368] [20240614:224619:1718430379:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:Rig-05:1:1:3368:3368] [20240614:224619:1718430379:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:Rig-05:1:1:3368:3368] [20240614:224619:1718430379:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:Rig-05:1:1:3368:3368] [20240614:224619:1718430379:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:Rig-05:1:1:3368:3368] [20240614:224619:1718430379:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:Rig-05:1:1:3368:3368] [20240614:224619:1718430379:0.000] [ignore_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:Rig-05:1:1:3368:3368] [20240614:224619:1718430379:0.000] [restore_default_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:Rig-05:1:1:3368:3368] [20240614:224619:1718430379:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:Rig-05:1:1:3368:3368] [20240614:224619:1718430379:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:Rig-05:1:1:3368:3368] [20240614:224619:1718430379:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:Rig-05:1:1:3368:3368] [20240614:224619:1718430379:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x820cf0 (old at (nil))
[EC_DRHOOK:Rig-05:1:1:3368:3368] [20240614:224619:1718430379:0.000] [trapfpe_treatment@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:Rig-05:1:1:3368:3368] [20240614:224619:1718430379:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:Rig-05:1:1:3368:3368] [20240614:224619:1718430379:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:Rig-05:1:1:3368:3368] [20240614:224619:1718430379:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x820cf0 (old at (nil))
[EC_DRHOOK:Rig-05:1:1:3368:3368] [20240614:224619:1718430379:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:Rig-05:1:1:3368:3368] [20240614:224619:1718430379:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:Rig-05:1:1:3368:3368] [20240614:224619:1718430379:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:Rig-05:1:1:3368:3368] [20240614:224619:1718430379:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x820cf0 (old at (nil))
[EC_DRHOOK:Rig-05:1:1:3368:3368] [20240614:224619:1718430379:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x820cf0 (old at (nil))
[EC_DRHOOK:Rig-05:1:1:3368:3368] [20240614:224619:1718430379:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:Rig-05:1:1:3368:3368] [20240614:224619:1718430379:0.000] [catch_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  22:46:36 STEP 1632 H= 408:00 +CPU= 15.034
  22:46:50 STEP 1633 H= 408:15 +CPU= 13.943
  22:47:04 STEP 1634 H= 408:30 +CPU= 13.894
  22:47:18 STEP 1635 H= 408:45 +CPU= 13.739
  22:47:32 STEP 1636 H= 409:00 +CPU= 14.302
  22:47:46 STEP 1637 H= 409:15 +CPU= 14.034
  22:48:00 STEP 1638 H= 409:30 +CPU= 14.014
  22:48:14 STEP 1639 H= 409:45 +CPU= 13.959
  22:48:28 STEP 1640 H= 410:00 +CPU= 14.134
  22:48:42 STEP 1641 H= 410:15 +CPU= 13.512
  22:48:56 STEP 1642 H= 410:30 +CPU= 13.863
  22:49:10 STEP 1643 H= 410:45 +CPU= 13.932
  22:49:27 STEP 1644 H= 411:00 +CPU= 17.159
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001644
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001644
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001644
  22:49:41 STEP 1645 H= 411:15 +CPU= 14.076
  22:49:55 STEP 1646 H= 411:30 +CPU= 13.611
  22:50:09 STEP 1647 H= 411:45 +CPU= 13.859
  22:50:23 STEP 1648 H= 412:00 +CPU= 14.239
  22:50:37 STEP 1649 H= 412:15 +CPU= 14.243
  22:50:51 STEP 1650 H= 412:30 +CPU= 14.048
  22:51:05 STEP 1651 H= 412:45 +CPU= 13.775
  22:51:19 STEP 1652 H= 413:00 +CPU= 14.087
  22:51:33 STEP 1653 H= 413:15 +CPU= 13.828
  22:51:47 STEP 1654 H= 413:30 +CPU= 13.907
  22:52:01 STEP 1655 H= 413:45 +CPU= 13.924
  22:52:19 STEP 1656 H= 414:00 +CPU= 17.996
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001656
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001656
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001656
  22:52:34 STEP 1657 H= 414:15 +CPU= 15.607
  22:53:14 STEP 1658 H= 414:30 +CPU= 38.930
  22:53:55 STEP 1659 H= 414:45 +CPU= 41.591
  22:54:37 STEP 1660 H= 415:00 +CPU= 42.198
  22:55:19 STEP 1661 H= 415:15 +CPU= 41.523
  22:56:01 STEP 1662 H= 415:30 +CPU= 41.619
  22:56:43 STEP 1663 H= 415:45 +CPU= 41.671
  22:57:25 STEP 1664 H= 416:00 +CPU= 42.159
  22:58:07 STEP 1665 H= 416:15 +CPU= 41.648
  22:58:48 STEP 1666 H= 416:30 +CPU= 41.546
  22:59:30 STEP 1667 H= 416:45 +CPU= 41.550
  23:00:22 STEP 1668 H= 417:00 +CPU= 52.323
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001668
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001668
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001668
  23:01:04 STEP 1669 H= 417:15 +CPU= 41.723
  23:01:46 STEP 1670 H= 417:30 +CPU= 41.797
  23:02:28 STEP 1671 H= 417:45 +CPU= 41.859
  23:03:10 STEP 1672 H= 418:00 +CPU= 42.445
  23:03:52 STEP 1673 H= 418:15 +CPU= 41.843
  23:04:34 STEP 1674 H= 418:30 +CPU= 41.845
  23:05:16 STEP 1675 H= 418:45 +CPU= 41.893
  23:05:59 STEP 1676 H= 419:00 +CPU= 42.452
  23:06:41 STEP 1677 H= 419:15 +CPU= 41.872
  23:07:23 STEP 1678 H= 419:30 +CPU= 41.857
  23:08:05 STEP 1679 H= 419:45 +CPU= 41.845
  23:09:02 STEP 1680 H= 420:00 +CPU= 57.429
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001680
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001680
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001680
  23:09:44 STEP 1681 H= 420:15 +CPU= 41.635
  23:10:26 STEP 1682 H= 420:30 +CPU= 41.619
  23:11:07 STEP 1683 H= 420:45 +CPU= 41.730
  23:11:50 STEP 1684 H= 421:00 +CPU= 42.374
  23:12:32 STEP 1685 H= 421:15 +CPU= 41.776
  23:13:14 STEP 1686 H= 421:30 +CPU= 41.735
  23:13:55 STEP 1687 H= 421:45 +CPU= 41.810
  23:14:38 STEP 1688 H= 422:00 +CPU= 42.482
  23:15:20 STEP 1689 H= 422:15 +CPU= 41.751
  23:16:02 STEP 1690 H= 422:30 +CPU= 41.784
  23:16:43 STEP 1691 H= 422:45 +CPU= 41.795
  23:17:36 STEP 1692 H= 423:00 +CPU= 52.394
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001692
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001692
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001692
  23:18:18 STEP 1693 H= 423:15 +CPU= 41.669
  23:18:59 STEP 1694 H= 423:30 +CPU= 41.632
  23:19:41 STEP 1695 H= 423:45 +CPU= 41.690
  23:20:23 STEP 1696 H= 424:00 +CPU= 42.258
  23:21:05 STEP 1697 H= 424:15 +CPU= 41.705
  23:21:47 STEP 1698 H= 424:30 +CPU= 41.581
  23:22:29 STEP 1699 H= 424:45 +CPU= 41.698
  23:23:11 STEP 1700 H= 425:00 +CPU= 42.281
  23:23:53 STEP 1701 H= 425:15 +CPU= 41.574
  23:24:34 STEP 1702 H= 425:30 +CPU= 41.640
  23:25:16 STEP 1703 H= 425:45 +CPU= 41.581
  23:26:08 STEP 1704 H= 426:00 +CPU= 52.276
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001704
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001704
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001704
  23:26:50 STEP 1705 H= 426:15 +CPU= 41.731
  23:27:32 STEP 1706 H= 426:30 +CPU= 41.547
  23:28:13 STEP 1707 H= 426:45 +CPU= 41.609
  23:28:56 STEP 1708 H= 427:00 +CPU= 42.305
  23:29:37 STEP 1709 H= 427:15 +CPU= 41.623
  23:30:19 STEP 1710 H= 427:30 +CPU= 41.691
  23:31:01 STEP 1711 H= 427:45 +CPU= 41.497
  23:31:43 STEP 1712 H= 428:00 +CPU= 42.067
  23:32:25 STEP 1713 H= 428:15 +CPU= 41.660
  23:33:06 STEP 1714 H= 428:30 +CPU= 41.667
  23:33:48 STEP 1715 H= 428:45 +CPU= 41.627
  23:34:40 STEP 1716 H= 429:00 +CPU= 52.376
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001716
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001716
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001716
  23:35:22 STEP 1717 H= 429:15 +CPU= 41.390
  23:36:04 STEP 1718 H= 429:30 +CPU= 41.849
  23:36:46 STEP 1719 H= 429:45 +CPU= 41.795
  23:37:28 STEP 1720 H= 430:00 +CPU= 42.355
  23:38:10 STEP 1721 H= 430:15 +CPU= 41.900
  23:38:52 STEP 1722 H= 430:30 +CPU= 41.772
  23:39:34 STEP 1723 H= 430:45 +CPU= 41.724
  23:40:16 STEP 1724 H= 431:00 +CPU= 42.345
  23:40:58 STEP 1725 H= 431:15 +CPU= 41.855
  23:41:40 STEP 1726 H= 431:30 +CPU= 41.710
  23:42:22 STEP 1727 H= 431:45 +CPU= 41.798
  23:43:20 STEP 1728 H= 432:00 +CPU= 57.946
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001728
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001728
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001728
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMGGh7zg+001632
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMSHh7zg+001632
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMUAh7zg+001632
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMGGh7zg+001644
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMSHh7zg+001644
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMUAh7zg+001644
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMGGh7zg+001656
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMSHh7zg+001656
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMUAh7zg+001656
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMGGh7zg+001668
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMSHh7zg+001668
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMUAh7zg+001668
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMGGh7zg+001680
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMSHh7zg+001680
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMUAh7zg+001680
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMGGh7zg+001692
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMSHh7zg+001692
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMUAh7zg+001692
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMGGh7zg+001704
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMSHh7zg+001704
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMUAh7zg+001704
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMGGh7zg+001716
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMSHh7zg+001716
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMUAh7zg+001716
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a15q_2016092300_20_1018_12289829_0_r1746830500_17.zip
Uploading the intermediate file: upload_file_17.zip
  23:44:02 STEP 1729 H= 432:15 +CPU= 41.708
  23:44:43 STEP 1730 H= 432:30 +CPU= 41.846
  23:45:26 STEP 1731 H= 432:45 +CPU= 42.034
  23:46:08 STEP 1732 H= 433:00 +CPU= 42.502
  23:46:50 STEP 1733 H= 433:15 +CPU= 41.861
  23:47:32 STEP 1734 H= 433:30 +CPU= 41.896
  23:48:14 STEP 1735 H= 433:45 +CPU= 41.810
  23:48:56 STEP 1736 H= 434:00 +CPU= 42.533
  23:49:38 STEP 1737 H= 434:15 +CPU= 41.885
  23:50:20 STEP 1738 H= 434:30 +CPU= 41.830
  23:51:02 STEP 1739 H= 434:45 +CPU= 41.810
  23:51:55 STEP 1740 H= 435:00 +CPU= 52.605
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001740
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001740
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001740
  23:52:37 STEP 1741 H= 435:15 +CPU= 41.750
  23:53:18 STEP 1742 H= 435:30 +CPU= 41.736
  23:54:00 STEP 1743 H= 435:45 +CPU= 41.710
  23:54:43 STEP 1744 H= 436:00 +CPU= 42.324
  23:55:24 STEP 1745 H= 436:15 +CPU= 41.605
  23:56:06 STEP 1746 H= 436:30 +CPU= 41.635
  23:56:48 STEP 1747 H= 436:45 +CPU= 41.655
  23:57:30 STEP 1748 H= 437:00 +CPU= 42.294
  23:58:12 STEP 1749 H= 437:15 +CPU= 41.709
  23:58:54 STEP 1750 H= 437:30 +CPU= 41.731
  23:59:35 STEP 1751 H= 437:45 +CPU= 41.647
  00:00:28 STEP 1752 H= 438:00 +CPU= 52.827
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001752
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001752
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001752
  00:01:10 STEP 1753 H= 438:15 +CPU= 41.783
  00:01:52 STEP 1754 H= 438:30 +CPU= 41.732
  00:02:34 STEP 1755 H= 438:45 +CPU= 41.749
  00:03:16 STEP 1756 H= 439:00 +CPU= 42.364
  00:03:58 STEP 1757 H= 439:15 +CPU= 41.810
  00:04:40 STEP 1758 H= 439:30 +CPU= 41.759
  00:05:22 STEP 1759 H= 439:45 +CPU= 41.762
  00:06:04 STEP 1760 H= 440:00 +CPU= 42.409
  00:06:46 STEP 1761 H= 440:15 +CPU= 41.824
  00:07:28 STEP 1762 H= 440:30 +CPU= 41.714
  00:08:10 STEP 1763 H= 440:45 +CPU= 41.802
  00:09:02 STEP 1764 H= 441:00 +CPU= 52.635
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001764
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001764
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001764
  00:09:44 STEP 1765 H= 441:15 +CPU= 41.856
  00:10:26 STEP 1766 H= 441:30 +CPU= 41.923
  00:11:08 STEP 1767 H= 441:45 +CPU= 41.980
  00:11:51 STEP 1768 H= 442:00 +CPU= 42.601
  00:12:33 STEP 1769 H= 442:15 +CPU= 41.862
  00:13:15 STEP 1770 H= 442:30 +CPU= 41.940
  00:13:57 STEP 1771 H= 442:45 +CPU= 41.928
  00:14:39 STEP 1772 H= 443:00 +CPU= 42.540
  00:15:21 STEP 1773 H= 443:15 +CPU= 41.883
  00:16:03 STEP 1774 H= 443:30 +CPU= 41.882
  00:16:45 STEP 1775 H= 443:45 +CPU= 41.787
  00:17:43 STEP 1776 H= 444:00 +CPU= 57.792
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001776
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001776
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001776
  00:18:25 STEP 1777 H= 444:15 +CPU= 41.850
  00:19:07 STEP 1778 H= 444:30 +CPU= 41.769
  00:19:49 STEP 1779 H= 444:45 +CPU= 41.893
  00:20:31 STEP 1780 H= 445:00 +CPU= 42.417
  00:21:13 STEP 1781 H= 445:15 +CPU= 41.878
  00:21:55 STEP 1782 H= 445:30 +CPU= 41.859
  00:22:37 STEP 1783 H= 445:45 +CPU= 41.889
  00:23:20 STEP 1784 H= 446:00 +CPU= 42.528
  00:24:02 STEP 1785 H= 446:15 +CPU= 41.960
  00:24:44 STEP 1786 H= 446:30 +CPU= 41.904
  00:25:26 STEP 1787 H= 446:45 +CPU= 41.919
  00:26:18 STEP 1788 H= 447:00 +CPU= 52.728
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001788
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001788
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001788
  00:27:00 STEP 1789 H= 447:15 +CPU= 41.699
  00:27:42 STEP 1790 H= 447:30 +CPU= 41.846
  00:28:24 STEP 1791 H= 447:45 +CPU= 41.879
  00:29:07 STEP 1792 H= 448:00 +CPU= 42.433
  00:29:48 STEP 1793 H= 448:15 +CPU= 41.866
  00:30:30 STEP 1794 H= 448:30 +CPU= 41.847
  00:31:12 STEP 1795 H= 448:45 +CPU= 41.806
  00:31:55 STEP 1796 H= 449:00 +CPU= 42.592
  00:32:37 STEP 1797 H= 449:15 +CPU= 41.973
  00:33:19 STEP 1798 H= 449:30 +CPU= 41.940
  00:34:01 STEP 1799 H= 449:45 +CPU= 41.968
  00:34:54 STEP 1800 H= 450:00 +CPU= 52.582
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001800
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001800
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001800
  00:35:36 STEP 1801 H= 450:15 +CPU= 41.918
  00:36:18 STEP 1802 H= 450:30 +CPU= 41.871
  00:36:59 STEP 1803 H= 450:45 +CPU= 41.894
  00:37:42 STEP 1804 H= 451:00 +CPU= 42.474
  00:38:24 STEP 1805 H= 451:15 +CPU= 41.911
  00:39:06 STEP 1806 H= 451:30 +CPU= 41.969
  00:39:48 STEP 1807 H= 451:45 +CPU= 41.822
  00:40:31 STEP 1808 H= 452:00 +CPU= 42.593
  00:41:13 STEP 1809 H= 452:15 +CPU= 41.920
  00:41:54 STEP 1810 H= 452:30 +CPU= 41.877
  00:42:36 STEP 1811 H= 452:45 +CPU= 41.850
  00:43:29 STEP 1812 H= 453:00 +CPU= 52.487
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001812
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001812
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001812
  00:44:11 STEP 1813 H= 453:15 +CPU= 41.923
  00:44:53 STEP 1814 H= 453:30 +CPU= 41.838
  00:45:35 STEP 1815 H= 453:45 +CPU= 41.909
  00:46:17 STEP 1816 H= 454:00 +CPU= 42.609
  00:47:00 STEP 1817 H= 454:15 +CPU= 42.002
  00:47:42 STEP 1818 H= 454:30 +CPU= 41.899
  00:48:23 STEP 1819 H= 454:45 +CPU= 41.886
  00:49:06 STEP 1820 H= 455:00 +CPU= 42.505
  00:49:48 STEP 1821 H= 455:15 +CPU= 41.783
  00:50:30 STEP 1822 H= 455:30 +CPU= 41.868
  00:51:12 STEP 1823 H= 455:45 +CPU= 41.828
  00:52:10 STEP 1824 H= 456:00 +CPU= 57.867
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001824
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001824
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001824
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMGGh7zg+001728
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMSHh7zg+001728
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMUAh7zg+001728
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMGGh7zg+001740
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMSHh7zg+001740
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMUAh7zg+001740
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMGGh7zg+001752
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMSHh7zg+001752
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMUAh7zg+001752
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMGGh7zg+001764
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMSHh7zg+001764
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMUAh7zg+001764
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMGGh7zg+001776
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMSHh7zg+001776
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMUAh7zg+001776
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMGGh7zg+001788
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMSHh7zg+001788
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMUAh7zg+001788
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMGGh7zg+001800
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMSHh7zg+001800
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMUAh7zg+001800
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMGGh7zg+001812
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMSHh7zg+001812
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMUAh7zg+001812
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a15q_2016092300_20_1018_12289829_0_r1746830500_18.zip
Uploading the intermediate file: upload_file_18.zip
  00:52:52 STEP 1825 H= 456:15 +CPU= 41.811
  00:53:34 STEP 1826 H= 456:30 +CPU= 41.941
  00:54:15 STEP 1827 H= 456:45 +CPU= 41.807
  00:54:58 STEP 1828 H= 457:00 +CPU= 42.531
  00:55:40 STEP 1829 H= 457:15 +CPU= 41.962
  00:56:21 STEP 1830 H= 457:30 +CPU= 41.087
  00:57:02 STEP 1831 H= 457:45 +CPU= 41.126
  00:57:44 STEP 1832 H= 458:00 +CPU= 41.679
  00:58:25 STEP 1833 H= 458:15 +CPU= 41.089
  00:59:06 STEP 1834 H= 458:30 +CPU= 41.030
  00:59:47 STEP 1835 H= 458:45 +CPU= 41.046
  01:00:39 STEP 1836 H= 459:00 +CPU= 51.416
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001836
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001836
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001836
  01:01:20 STEP 1837 H= 459:15 +CPU= 41.004
  01:02:01 STEP 1838 H= 459:30 +CPU= 40.990
  01:02:42 STEP 1839 H= 459:45 +CPU= 40.915
  01:03:24 STEP 1840 H= 460:00 +CPU= 41.471
  01:04:05 STEP 1841 H= 460:15 +CPU= 40.916
  01:04:46 STEP 1842 H= 460:30 +CPU= 40.933
  01:05:27 STEP 1843 H= 460:45 +CPU= 41.003
  01:06:08 STEP 1844 H= 461:00 +CPU= 41.554
  01:06:49 STEP 1845 H= 461:15 +CPU= 40.924
  01:07:30 STEP 1846 H= 461:30 +CPU= 40.963
  01:08:11 STEP 1847 H= 461:45 +CPU= 40.987
  01:09:03 STEP 1848 H= 462:00 +CPU= 51.592
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001848
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001848
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001848
  01:09:44 STEP 1849 H= 462:15 +CPU= 40.795
  01:10:25 STEP 1850 H= 462:30 +CPU= 40.900
  01:11:06 STEP 1851 H= 462:45 +CPU= 40.926
  01:11:47 STEP 1852 H= 463:00 +CPU= 41.584
  01:12:28 STEP 1853 H= 463:15 +CPU= 40.847
  01:13:09 STEP 1854 H= 463:30 +CPU= 40.888
  01:13:50 STEP 1855 H= 463:45 +CPU= 40.856
  01:14:32 STEP 1856 H= 464:00 +CPU= 41.508
  01:15:12 STEP 1857 H= 464:15 +CPU= 40.794
  01:15:53 STEP 1858 H= 464:30 +CPU= 40.857
  01:16:34 STEP 1859 H= 464:45 +CPU= 40.944
  01:17:26 STEP 1860 H= 465:00 +CPU= 51.587
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001860
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001860
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001860
  01:18:07 STEP 1861 H= 465:15 +CPU= 40.953
  01:18:48 STEP 1862 H= 465:30 +CPU= 40.942
  01:19:29 STEP 1863 H= 465:45 +CPU= 40.907
  01:20:11 STEP 1864 H= 466:00 +CPU= 41.543
  01:20:52 STEP 1865 H= 466:15 +CPU= 41.008
  01:21:33 STEP 1866 H= 466:30 +CPU= 40.928
  01:22:14 STEP 1867 H= 466:45 +CPU= 40.958
  01:22:55 STEP 1868 H= 467:00 +CPU= 41.490
  01:23:36 STEP 1869 H= 467:15 +CPU= 40.894
  01:24:17 STEP 1870 H= 467:30 +CPU= 41.006
  01:24:58 STEP 1871 H= 467:45 +CPU= 40.889
  01:25:55 STEP 1872 H= 468:00 +CPU= 56.946
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001872
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001872
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001872
  01:26:36 STEP 1873 H= 468:15 +CPU= 40.634
  01:27:17 STEP 1874 H= 468:30 +CPU= 40.886
  01:27:58 STEP 1875 H= 468:45 +CPU= 40.858
  01:28:39 STEP 1876 H= 469:00 +CPU= 41.404
  01:29:20 STEP 1877 H= 469:15 +CPU= 40.929
  01:30:01 STEP 1878 H= 469:30 +CPU= 40.940
  01:30:42 STEP 1879 H= 469:45 +CPU= 40.987
  01:31:24 STEP 1880 H= 470:00 +CPU= 41.536
  01:32:05 STEP 1881 H= 470:15 +CPU= 40.908
  01:32:46 STEP 1882 H= 470:30 +CPU= 40.870
  01:33:27 STEP 1883 H= 470:45 +CPU= 40.867
  01:34:18 STEP 1884 H= 471:00 +CPU= 51.536
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001884
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001884
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001884
  01:34:59 STEP 1885 H= 471:15 +CPU= 40.740
  01:35:40 STEP 1886 H= 471:30 +CPU= 40.869
  01:36:21 STEP 1887 H= 471:45 +CPU= 40.821
  01:37:02 STEP 1888 H= 472:00 +CPU= 41.383
  01:37:43 STEP 1889 H= 472:15 +CPU= 40.749
  01:38:24 STEP 1890 H= 472:30 +CPU= 40.734
  01:39:05 STEP 1891 H= 472:45 +CPU= 40.799
  01:39:46 STEP 1892 H= 473:00 +CPU= 41.436
  01:40:27 STEP 1893 H= 473:15 +CPU= 40.756
  01:41:08 STEP 1894 H= 473:30 +CPU= 40.784
  01:41:49 STEP 1895 H= 473:45 +CPU= 40.777
  01:42:40 STEP 1896 H= 474:00 +CPU= 51.360
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001896
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001896
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001896
  01:43:21 STEP 1897 H= 474:15 +CPU= 40.712
  01:44:02 STEP 1898 H= 474:30 +CPU= 40.672
  01:44:42 STEP 1899 H= 474:45 +CPU= 40.734
  01:45:24 STEP 1900 H= 475:00 +CPU= 41.391
  01:46:05 STEP 1901 H= 475:15 +CPU= 40.822
  01:46:46 STEP 1902 H= 475:30 +CPU= 40.694
  01:47:26 STEP 1903 H= 475:45 +CPU= 40.746
  01:48:08 STEP 1904 H= 476:00 +CPU= 41.279
  01:48:49 STEP 1905 H= 476:15 +CPU= 40.741
  01:49:29 STEP 1906 H= 476:30 +CPU= 40.604
  01:50:10 STEP 1907 H= 476:45 +CPU= 40.697
  01:51:02 STEP 1908 H= 477:00 +CPU= 51.515
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001908
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001908
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001908
  01:51:42 STEP 1909 H= 477:15 +CPU= 40.708
  01:52:23 STEP 1910 H= 477:30 +CPU= 40.775
  01:53:04 STEP 1911 H= 477:45 +CPU= 40.830
  01:53:45 STEP 1912 H= 478:00 +CPU= 41.295
  01:54:26 STEP 1913 H= 478:15 +CPU= 40.621
  01:55:07 STEP 1914 H= 478:30 +CPU= 40.705
  01:55:48 STEP 1915 H= 478:45 +CPU= 40.697
  01:56:29 STEP 1916 H= 479:00 +CPU= 41.329
  01:57:10 STEP 1917 H= 479:15 +CPU= 40.677
  01:57:50 STEP 1918 H= 479:30 +CPU= 40.700
  01:58:31 STEP 1919 H= 479:45 +CPU= 40.722
  01:59:24 STEP 1920 H= 480:00 +CPU= 52.464
..The child process terminated with status: 0
>>> Printing last 70 lines from file: NODE.001_01

 
DDH------------------------------------------------------------------------
I                                                                         I
I TIME =  480.000H DDH * Diagnostic Var.,     AREA                GLOBAL  I
I                                                                         I
I-------------------------------------------------------------------------I
I   VARIABLE   I      Initial value         I        Current value        I
I-------------------------------------------------------------------------I
I  OMEGA (P/S) I     -0.2867389985E-21      I       0.2593187624E-05      I
ICloud fractionI      0.0000000000E+00      I       0.6656294917E-01      I
I  Relat. Hum. I      0.3068177894E+00      I       0.3999662837E+00      I
I  PASS 01     I      0.0000000000E+00      I       0.1055175524E+04      I
DDH------------------------------------------------------------------------
  
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
 NSTEP =  1920 SCAN2M_HPOS  P
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ICMGGh7zg+001920 MODE=a
 IO-STREAM CLOSED - ICMGGh7zg+001920
 MPI-TASK:   1 -    301953156 BYTES IN      7 RECORDS TRANSFERRED IN    0.0004 SECONDS754882.9 Mbytes/s, TOTAL TIME=    0.0160( 2.5%)
 01:59:24 STEP 1920 H= 480:00 +CPU= 52.464
 END CNT3
 NSTEP =  1920 CNT0   000000000  37.373  37.373  37.421   0.999               0               0               0               0               0
IO-STREAM STATISTICS, TOTAL NO OF REC/BYTES READ           46/  1069125708 WRITTEN         8580/************
 -TASK-OPENED-OPEN-RECS IN -KBYTE IN    -RECS OUT-KBYTE OUT-WALL   -WALL IN-WALL
  OU-TOT IN -TOT OUT
   1    153    0       46      1044068     8580 ************    24.6     1.4    23.3     1.7    46.7
===-=== START OF TIMING STATISTICS ===-===
 
STATS FOR ALL TASKS
 NUM ROUTINE                                     CALLS  MEAN(ms)   MAX(ms)   FRAC(%)  UNBAL(%)
   0 CNT0     - COMPLETE EXECUTION                   1 ********* *********    100.00      0.00
   1 CNT4     - FORWARD INTEGRATION                  1 ********* *********     99.98      0.00
   8 SCAN2M - GRID-POINT DYNAMICS                  313    6447.7    6447.7     17.42      0.00
   9 SPCM     - SPECTRAL COMP.                     288    1864.0    1864.0      4.63      0.00
  10 SCAN2M - PHYSICS                              289   18248.2   18248.2     45.52      0.00
  11 IOPACK   - OUTPUT P.P. RESULTS                 24    1335.5    1335.5      0.28      0.00
  12 SPNORM   - SPECTRAL NORM COMP.                 13      75.1      75.1      0.01      0.00
  17 GRIDFPOS IN CNT4                               24      38.7      38.7      0.01      0.00
  27 RESTART FILES - WRITING                         6    5342.0    5342.0      0.28      0.00
  28 RESTART FILES - READING                         1    2196.0    2196.0      0.02      0.00
  29 SU4FPOS IN CNT4                                24       0.1       0.1      0.00      0.00
  30 DYNFPOS IN CNT4                                24    8691.1    8691.1      1.80      0.00
  31 POSDDH IN STEPO                                 2       1.0       1.0      0.00      0.00
  37 CPGLAG   - SL COMPUTATIONS                    289    8372.1    8372.1     20.88      0.00
  39 SU0YOMB                                         1     167.6     167.6      0.00      0.00
  51 SCAN2M   - SL COMM. PART 1                    289      22.1      22.1      0.06      0.00
  54 SPCM     - M TO S/S TO M TRANSP.              288     363.6     363.6      0.90      0.00
  55 SPCIMPF  - S TO M/M TO S TRANSP.              288      64.5      64.5      0.16      0.00
  56 SPNORM   - SPECTRAL NORM COMM.                 13       0.3       0.3      0.00      0.00
 102 LTINV_CTL   - INVERSE LEGENDRE TRANSFORM      628     593.6     593.6      3.22      0.00
 103 LTDIR_CTL   - DIRECT LEGENDRE TRANSFORM       600     336.5     336.5      1.74      0.00
 106 FTDIR_CTL   - DIRECT FOURIER TRANSFORM        600     141.4     141.4      0.73      0.00
 107 FTINV_CTL   - INVERSE FOURIER TRANSFORM       628     234.5     234.5      1.27      0.00
 140 SULEG       - COMP. OF LEGENDRE POL.            1      27.3      27.3      0.00      0.00
 152 LTINV_CTL   - M TO L TRANSPOSITION            628      61.7      61.7      0.33      0.00
 153 LTDIR_CTL   - L TO M TRANSPOSITION            600      35.2      35.2      0.18      0.00
 157 FTINV_CTL   - L TO G TRANSPOSITION            628      93.1      93.1      0.50      0.00
 158 FTDIR_CTL   - G TO L TRANSPOSITION            600      59.4      59.4      0.31      0.00
 400 GSTATS                                      34412       0.0       0.0      0.00      0.00
 401 GSTATS HOOK                                 31964       0.0       0.0      0.00      0.00
TOTAL MEASURED IMBALANCE =       0.0 SECONDS,  0.0 PERCENT
TOTAL WALLCLOCK TIME    11585.1 CPU TIME   11567.1 VECTOR TIME    11567.1


===-=== END   OF TIMING STATISTICS ===-===
FORECAST DAYS PER DAY      149.2
  *** END CNT0 *** 
------------------------------------------------
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGh7zg+001920
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHh7zg+001920
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAh7zg+001920
Adding to the zip: /var/lib/boinc-client/slots/6/NODE.001_01
Adding to the zip: /var/lib/boinc-client/slots/6/ifs.stat
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMSHh7zg+001908
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMSHh7zg+001860
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMUAh7zg+001872
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMSHh7zg+001884
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMSHh7zg+001872
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMSHh7zg+001896
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMGGh7zg+001896
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMGGh7zg+001860
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMGGh7zg+001872
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMSHh7zg+001920
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMUAh7zg+001824
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMUAh7zg+001884
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMUAh7zg+001896
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMSHh7zg+001848
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMUAh7zg+001836
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMGGh7zg+001884
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMUAh7zg+001908
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMGGh7zg+001848
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMGGh7zg+001824
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMSHh7zg+001836
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMGGh7zg+001920
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMUAh7zg+001920
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMUAh7zg+001860
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMUAh7zg+001848
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMGGh7zg+001908
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMSHh7zg+001824
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12289829/ICMGGh7zg+001836
Zipping up the final file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a15q_2016092300_20_1018_12289829_0_r1746830500_19.zip
Uploading the final file: upload_file_19.zip
Uploading trickle at timestep: 1727100
02:02:56 (3335): called boinc_finish(0)

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
15 Jun 2024 09:51:05 1541000 22443417 oifs_43r3_bl_a15q_2016092300_20_1018_12289829_0 1,727,100 53,545 0.0310
14 Jun 2024 21:31:38 1541000 22443417 oifs_43r3_bl_a15q_2016092300_20_1018_12289829_0 1,382,400 38,679 0.0280


©2024 cpdn.org