climateprediction.net home page
Task 22446116

Task 22446116

Name oifs_43r3_bl_a387_2016092300_20_1018_12292510_0
Workunit 12292510
Created 13 Jun 2024, 2:05:06 UTC
Sent 13 Jun 2024, 16:36:32 UTC
Report deadline 12 Aug 2024, 16:36:32 UTC
Received 18 Jun 2024, 10:49:25 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 0 (0x00000000)
Computer ID 1489681
Run time 11 hours 59 min 18 sec
CPU time 7 hours 45 min 31 sec
Validate state Invalid
Credit 2,635.55
Device peak FLOPS 8.68 GFLOPS
Application version OpenIFS 43r3 Baroclinic Lifecycle v1.13
x86_64-pc-linux-gnu
Peak working set size 5,370.12 MB
Peak swap size 5,981.13 MB
Peak disk usage 1,906.07 MB
Stderr
<core_client_version>7.20.2</core_client_version>
<![CDATA[
<stderr_txt>
process
Resuming the child process
  10:01:57 STEP 1659 H= 414:45 +CPU= 20.134
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:02:20 STEP 1660 H= 415:00 +CPU= 21.001
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:02:42 STEP 1661 H= 415:15 +CPU= 20.171
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:03:04 STEP 1662 H= 415:30 +CPU= 20.254
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:03:26 STEP 1663 H= 415:45 +CPU= 20.104
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:03:52 STEP 1664 H= 416:00 +CPU= 21.287
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:04:14 STEP 1665 H= 416:15 +CPU= 20.529
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:04:36 STEP 1666 H= 416:30 +CPU= 20.197
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:04:59 STEP 1667 H= 416:45 +CPU= 20.507
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:05:27 STEP 1668 H= 417:00 +CPU= 25.907
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /mnt/nvme/scratch1/boinc/slots/1/ICMGGh7zg+001668
Moving to projects directory: /mnt/nvme/scratch1/boinc/slots/1/ICMSHh7zg+001668
Moving to projects directory: /mnt/nvme/scratch1/boinc/slots/1/ICMUAh7zg+001668
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:05:51 STEP 1669 H= 417:15 +CPU= 20.151
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:06:13 STEP 1670 H= 417:30 +CPU= 20.395
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:06:36 STEP 1671 H= 417:45 +CPU= 20.541
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:06:59 STEP 1672 H= 418:00 +CPU= 21.078
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:07:22 STEP 1673 H= 418:15 +CPU= 20.283
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:07:44 STEP 1674 H= 418:30 +CPU= 20.116
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:08:07 STEP 1675 H= 418:45 +CPU= 20.003
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:08:31 STEP 1676 H= 419:00 +CPU= 20.675
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:08:53 STEP 1677 H= 419:15 +CPU= 19.931
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:09:15 STEP 1678 H= 419:30 +CPU= 19.940
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:09:37 STEP 1679 H= 419:45 +CPU= 19.965
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:10:10 STEP 1680 H= 420:00 +CPU= 28.542
Moving to projects directory: /mnt/nvme/scratch1/boinc/slots/1/ICMGGh7zg+001680
Moving to projects directory: /mnt/nvme/scratch1/boinc/slots/1/ICMSHh7zg+001680
Moving to projects directory: /mnt/nvme/scratch1/boinc/slots/1/ICMUAh7zg+001680
Adding to the zip: /mnt/nvme/scratch1/boinc/projects/climateprediction.net/oifs_43r3_bl_12292510/ICMGGh7zg+001584
Adding to the zip: /mnt/nvme/scratch1/boinc/projects/climateprediction.net/oifs_43r3_bl_12292510/ICMSHh7zg+001584
Adding to the zip: /mnt/nvme/scratch1/boinc/projects/climateprediction.net/oifs_43r3_bl_12292510/ICMUAh7zg+001584
Adding to the zip: /mnt/nvme/scratch1/boinc/projects/climateprediction.net/oifs_43r3_bl_12292510/ICMGGh7zg+001596
Adding to the zip: /mnt/nvme/scratch1/boinc/projects/climateprediction.net/oifs_43r3_bl_12292510/ICMSHh7zg+001596
Adding to the zip: /mnt/nvme/scratch1/boinc/projects/climateprediction.net/oifs_43r3_bl_12292510/ICMUAh7zg+001596
Adding to the zip: /mnt/nvme/scratch1/boinc/projects/climateprediction.net/oifs_43r3_bl_12292510/ICMGGh7zg+001608
Adding to the zip: /mnt/nvme/scratch1/boinc/projects/climateprediction.net/oifs_43r3_bl_12292510/ICMSHh7zg+001608
Adding to the zip: /mnt/nvme/scratch1/boinc/projects/climateprediction.net/oifs_43r3_bl_12292510/ICMUAh7zg+001608
Adding to the zip: /mnt/nvme/scratch1/boinc/projects/climateprediction.net/oifs_43r3_bl_12292510/ICMGGh7zg+001620
Adding to the zip: /mnt/nvme/scratch1/boinc/projects/climateprediction.net/oifs_43r3_bl_12292510/ICMSHh7zg+001620
Adding to the zip: /mnt/nvme/scratch1/boinc/projects/climateprediction.net/oifs_43r3_bl_12292510/ICMUAh7zg+001620
Adding to the zip: /mnt/nvme/scratch1/boinc/projects/climateprediction.net/oifs_43r3_bl_12292510/ICMGGh7zg+001632
Adding to the zip: /mnt/nvme/scratch1/boinc/projects/climateprediction.net/oifs_43r3_bl_12292510/ICMSHh7zg+001632
Adding to the zip: /mnt/nvme/scratch1/boinc/projects/climateprediction.net/oifs_43r3_bl_12292510/ICMUAh7zg+001632
Adding to the zip: /mnt/nvme/scratch1/boinc/projects/climateprediction.net/oifs_43r3_bl_12292510/ICMGGh7zg+001644
Adding to the zip: /mnt/nvme/scratch1/boinc/projects/climateprediction.net/oifs_43r3_bl_12292510/ICMSHh7zg+001644
Adding to the zip: /mnt/nvme/scratch1/boinc/projects/climateprediction.net/oifs_43r3_bl_12292510/ICMUAh7zg+001644
Adding to the zip: /mnt/nvme/scratch1/boinc/projects/climateprediction.net/oifs_43r3_bl_12292510/ICMGGh7zg+001656
Adding to the zip: /mnt/nvme/scratch1/boinc/projects/climateprediction.net/oifs_43r3_bl_12292510/ICMSHh7zg+001656
Adding to the zip: /mnt/nvme/scratch1/boinc/projects/climateprediction.net/oifs_43r3_bl_12292510/ICMUAh7zg+001656
Adding to the zip: /mnt/nvme/scratch1/boinc/projects/climateprediction.net/oifs_43r3_bl_12292510/ICMGGh7zg+001668
Adding to the zip: /mnt/nvme/scratch1/boinc/projects/climateprediction.net/oifs_43r3_bl_12292510/ICMSHh7zg+001668
Adding to the zip: /mnt/nvme/scratch1/boinc/projects/climateprediction.net/oifs_43r3_bl_12292510/ICMUAh7zg+001668
Zipping up the intermediate file: /mnt/nvme/scratch1/boinc/projects/climateprediction.net/oifs_43r3_bl_a387_2016092300_20_1018_12292510_0_r205782200_11.zip
Uploading the intermediate file: upload_file_11.zip
  10:10:33 STEP 1681 H= 420:15 +CPU= 20.916
  10:10:53 STEP 1682 H= 420:30 +CPU= 20.034
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:11:15 STEP 1683 H= 420:45 +CPU= 20.404
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:11:38 STEP 1684 H= 421:00 +CPU= 20.791
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:12:00 STEP 1685 H= 421:15 +CPU= 20.164
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:12:22 STEP 1686 H= 421:30 +CPU= 20.141
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:12:44 STEP 1687 H= 421:45 +CPU= 20.177
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:13:10 STEP 1688 H= 422:00 +CPU= 20.893
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:13:32 STEP 1689 H= 422:15 +CPU= 20.123
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:13:54 STEP 1690 H= 422:30 +CPU= 20.083
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:14:16 STEP 1691 H= 422:45 +CPU= 20.127
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:14:44 STEP 1692 H= 423:00 +CPU= 25.645
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /mnt/nvme/scratch1/boinc/slots/1/ICMGGh7zg+001692
Moving to projects directory: /mnt/nvme/scratch1/boinc/slots/1/ICMSHh7zg+001692
Moving to projects directory: /mnt/nvme/scratch1/boinc/slots/1/ICMUAh7zg+001692
  10:15:06 STEP 1693 H= 423:15 +CPU= 20.006
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:15:30 STEP 1694 H= 423:30 +CPU= 20.060
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:15:52 STEP 1695 H= 423:45 +CPU= 20.137
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:16:15 STEP 1696 H= 424:00 +CPU= 21.078
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:16:37 STEP 1697 H= 424:15 +CPU= 20.117
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:16:59 STEP 1698 H= 424:30 +CPU= 20.207
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:17:21 STEP 1699 H= 424:45 +CPU= 20.069
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:17:44 STEP 1700 H= 425:00 +CPU= 20.809
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:18:06 STEP 1701 H= 425:15 +CPU= 19.992
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:18:30 STEP 1702 H= 425:30 +CPU= 19.979
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:18:53 STEP 1703 H= 425:45 +CPU= 20.127
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:19:20 STEP 1704 H= 426:00 +CPU= 25.896
Moving to projects directory: /mnt/nvme/scratch1/boinc/slots/1/ICMGGh7zg+001704
Moving to projects directory: /mnt/nvme/scratch1/boinc/slots/1/ICMSHh7zg+001704
Moving to projects directory: /mnt/nvme/scratch1/boinc/slots/1/ICMUAh7zg+001704
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:19:43 STEP 1705 H= 426:15 +CPU= 20.169
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:20:05 STEP 1706 H= 426:30 +CPU= 20.050
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:20:29 STEP 1707 H= 426:45 +CPU= 20.321
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:20:52 STEP 1708 H= 427:00 +CPU= 21.295
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:21:15 STEP 1709 H= 427:15 +CPU= 20.269
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:21:37 STEP 1710 H= 427:30 +CPU= 20.079
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:21:59 STEP 1711 H= 427:45 +CPU= 19.980
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:22:22 STEP 1712 H= 428:00 +CPU= 20.676
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:22:44 STEP 1713 H= 428:15 +CPU= 19.982
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:23:06 STEP 1714 H= 428:30 +CPU= 20.079
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:23:30 STEP 1715 H= 428:45 +CPU= 19.943
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:23:58 STEP 1716 H= 429:00 +CPU= 25.758
Moving to projects directory: /mnt/nvme/scratch1/boinc/slots/1/ICMGGh7zg+001716
Moving to projects directory: /mnt/nvme/scratch1/boinc/slots/1/ICMSHh7zg+001716
Moving to projects directory: /mnt/nvme/scratch1/boinc/slots/1/ICMUAh7zg+001716
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:24:20 STEP 1717 H= 429:15 +CPU= 20.074
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:24:42 STEP 1718 H= 429:30 +CPU= 19.894
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:25:04 STEP 1719 H= 429:45 +CPU= 19.839
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:25:28 STEP 1720 H= 430:00 +CPU= 20.661
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:25:50 STEP 1721 H= 430:15 +CPU= 19.948
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:26:12 STEP 1722 H= 430:30 +CPU= 19.879
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:26:34 STEP 1723 H= 430:45 +CPU= 19.808
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:26:57 STEP 1724 H= 431:00 +CPU= 20.956
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:27:20 STEP 1725 H= 431:15 +CPU= 20.233
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:27:42 STEP 1726 H= 431:30 +CPU= 20.108
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:28:04 STEP 1727 H= 431:45 +CPU= 20.024
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:28:36 STEP 1728 H= 432:00 +CPU= 28.458
Moving to projects directory: /mnt/nvme/scratch1/boinc/slots/1/ICMGGh7zg+001728
Moving to projects directory: /mnt/nvme/scratch1/boinc/slots/1/ICMSHh7zg+001728
Moving to projects directory: /mnt/nvme/scratch1/boinc/slots/1/ICMUAh7zg+001728
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:28:59 STEP 1729 H= 432:15 +CPU= 21.049
  10:29:20 STEP 1730 H= 432:30 +CPU= 20.373
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:29:42 STEP 1731 H= 432:45 +CPU= 20.329
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:30:05 STEP 1732 H= 433:00 +CPU= 21.090
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:30:30 STEP 1733 H= 433:15 +CPU= 20.402
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:30:52 STEP 1734 H= 433:30 +CPU= 20.295
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:31:14 STEP 1735 H= 433:45 +CPU= 20.251
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:31:37 STEP 1736 H= 434:00 +CPU= 20.999
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:32:00 STEP 1737 H= 434:15 +CPU= 20.197
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:32:22 STEP 1738 H= 434:30 +CPU= 20.237
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:32:44 STEP 1739 H= 434:45 +CPU= 20.315
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:33:15 STEP 1740 H= 435:00 +CPU= 26.733
Moving to projects directory: /mnt/nvme/scratch1/boinc/slots/1/ICMGGh7zg+001740
Moving to projects directory: /mnt/nvme/scratch1/boinc/slots/1/ICMSHh7zg+001740
Moving to projects directory: /mnt/nvme/scratch1/boinc/slots/1/ICMUAh7zg+001740
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:33:38 STEP 1741 H= 435:15 +CPU= 20.987
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:34:01 STEP 1742 H= 435:30 +CPU= 21.035
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:34:25 STEP 1743 H= 435:45 +CPU= 21.301
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:34:50 STEP 1744 H= 436:00 +CPU= 21.893
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:35:14 STEP 1745 H= 436:15 +CPU= 21.155
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:35:37 STEP 1746 H= 436:30 +CPU= 21.231
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:36:00 STEP 1747 H= 436:45 +CPU= 20.910
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:36:23 STEP 1748 H= 437:00 +CPU= 20.961
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:36:45 STEP 1749 H= 437:15 +CPU= 20.463
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:37:10 STEP 1750 H= 437:30 +CPU= 20.391
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:37:32 STEP 1751 H= 437:45 +CPU= 20.502
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:38:01 STEP 1752 H= 438:00 +CPU= 26.576
Moving to projects directory: /mnt/nvme/scratch1/boinc/slots/1/ICMGGh7zg+001752
Moving to projects directory: /mnt/nvme/scratch1/boinc/slots/1/ICMSHh7zg+001752
Moving to projects directory: /mnt/nvme/scratch1/boinc/slots/1/ICMUAh7zg+001752
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:38:24 STEP 1753 H= 438:15 +CPU= 20.669
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:38:46 STEP 1754 H= 438:30 +CPU= 20.656
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:39:11 STEP 1755 H= 438:45 +CPU= 20.387
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:39:34 STEP 1756 H= 439:00 +CPU= 21.131
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:39:57 STEP 1757 H= 439:15 +CPU= 20.764
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:40:19 STEP 1758 H= 439:30 +CPU= 20.331
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:40:41 STEP 1759 H= 439:45 +CPU= 20.335
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:41:05 STEP 1760 H= 440:00 +CPU= 21.083
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:41:29 STEP 1761 H= 440:15 +CPU= 20.438
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:41:52 STEP 1762 H= 440:30 +CPU= 20.622
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:42:14 STEP 1763 H= 440:45 +CPU= 20.616
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:42:43 STEP 1764 H= 441:00 +CPU= 26.607
Moving to projects directory: /mnt/nvme/scratch1/boinc/slots/1/ICMGGh7zg+001764
Moving to projects directory: /mnt/nvme/scratch1/boinc/slots/1/ICMSHh7zg+001764
Moving to projects directory: /mnt/nvme/scratch1/boinc/slots/1/ICMUAh7zg+001764
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:43:09 STEP 1765 H= 441:15 +CPU= 21.493
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:43:32 STEP 1766 H= 441:30 +CPU= 21.177
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:43:56 STEP 1767 H= 441:45 +CPU= 21.034
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:44:20 STEP 1768 H= 442:00 +CPU= 21.917
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:44:43 STEP 1769 H= 442:15 +CPU= 20.987
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:45:06 STEP 1770 H= 442:30 +CPU= 21.224
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:45:31 STEP 1771 H= 442:45 +CPU= 21.243
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:45:55 STEP 1772 H= 443:00 +CPU= 22.042
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:46:19 STEP 1773 H= 443:15 +CPU= 21.302
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:46:42 STEP 1774 H= 443:30 +CPU= 20.985
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:47:04 STEP 1775 H= 443:45 +CPU= 20.319
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:47:37 STEP 1776 H= 444:00 +CPU= 28.991
Moving to projects directory: /mnt/nvme/scratch1/boinc/slots/1/ICMGGh7zg+001776
Moving to projects directory: /mnt/nvme/scratch1/boinc/slots/1/ICMSHh7zg+001776
Moving to projects directory: /mnt/nvme/scratch1/boinc/slots/1/ICMUAh7zg+001776
Adding to the zip: /mnt/nvme/scratch1/boinc/projects/climateprediction.net/oifs_43r3_bl_12292510/ICMGGh7zg+001680
Adding to the zip: /mnt/nvme/scratch1/boinc/projects/climateprediction.net/oifs_43r3_bl_12292510/ICMSHh7zg+001680
Adding to the zip: /mnt/nvme/scratch1/boinc/projects/climateprediction.net/oifs_43r3_bl_12292510/ICMUAh7zg+001680
Adding to the zip: /mnt/nvme/scratch1/boinc/projects/climateprediction.net/oifs_43r3_bl_12292510/ICMGGh7zg+001692
Adding to the zip: /mnt/nvme/scratch1/boinc/projects/climateprediction.net/oifs_43r3_bl_12292510/ICMSHh7zg+001692
Adding to the zip: /mnt/nvme/scratch1/boinc/projects/climateprediction.net/oifs_43r3_bl_12292510/ICMUAh7zg+001692
Adding to the zip: /mnt/nvme/scratch1/boinc/projects/climateprediction.net/oifs_43r3_bl_12292510/ICMGGh7zg+001704
Adding to the zip: /mnt/nvme/scratch1/boinc/projects/climateprediction.net/oifs_43r3_bl_12292510/ICMSHh7zg+001704
Adding to the zip: /mnt/nvme/scratch1/boinc/projects/climateprediction.net/oifs_43r3_bl_12292510/ICMUAh7zg+001704
Adding to the zip: /mnt/nvme/scratch1/boinc/projects/climateprediction.net/oifs_43r3_bl_12292510/ICMGGh7zg+001716
Adding to the zip: /mnt/nvme/scratch1/boinc/projects/climateprediction.net/oifs_43r3_bl_12292510/ICMSHh7zg+001716
Adding to the zip: /mnt/nvme/scratch1/boinc/projects/climateprediction.net/oifs_43r3_bl_12292510/ICMUAh7zg+001716
Adding to the zip: /mnt/nvme/scratch1/boinc/projects/climateprediction.net/oifs_43r3_bl_12292510/ICMGGh7zg+001728
Adding to the zip: /mnt/nvme/scratch1/boinc/projects/climateprediction.net/oifs_43r3_bl_12292510/ICMSHh7zg+001728
Adding to the zip: /mnt/nvme/scratch1/boinc/projects/climateprediction.net/oifs_43r3_bl_12292510/ICMUAh7zg+001728
Adding to the zip: /mnt/nvme/scratch1/boinc/projects/climateprediction.net/oifs_43r3_bl_12292510/ICMGGh7zg+001740
Adding to the zip: /mnt/nvme/scratch1/boinc/projects/climateprediction.net/oifs_43r3_bl_12292510/ICMSHh7zg+001740
Adding to the zip: /mnt/nvme/scratch1/boinc/projects/climateprediction.net/oifs_43r3_bl_12292510/ICMUAh7zg+001740
Adding to the zip: /mnt/nvme/scratch1/boinc/projects/climateprediction.net/oifs_43r3_bl_12292510/ICMGGh7zg+001752
Adding to the zip: /mnt/nvme/scratch1/boinc/projects/climateprediction.net/oifs_43r3_bl_12292510/ICMSHh7zg+001752
Adding to the zip: /mnt/nvme/scratch1/boinc/projects/climateprediction.net/oifs_43r3_bl_12292510/ICMUAh7zg+001752
Adding to the zip: /mnt/nvme/scratch1/boinc/projects/climateprediction.net/oifs_43r3_bl_12292510/ICMGGh7zg+001764
Adding to the zip: /mnt/nvme/scratch1/boinc/projects/climateprediction.net/oifs_43r3_bl_12292510/ICMSHh7zg+001764
Adding to the zip: /mnt/nvme/scratch1/boinc/projects/climateprediction.net/oifs_43r3_bl_12292510/ICMUAh7zg+001764
Zipping up the intermediate file: /mnt/nvme/scratch1/boinc/projects/climateprediction.net/oifs_43r3_bl_a387_2016092300_20_1018_12292510_0_r205782200_12.zip
  10:47:59 STEP 1777 H= 444:15 +CPU= 20.766
Uploading the intermediate file: upload_file_12.zip
  10:48:20 STEP 1778 H= 444:30 +CPU= 20.655
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:48:43 STEP 1779 H= 444:45 +CPU= 20.701
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:49:09 STEP 1780 H= 445:00 +CPU= 21.399
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:49:32 STEP 1781 H= 445:15 +CPU= 20.795
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:49:55 STEP 1782 H= 445:30 +CPU= 20.701
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:50:17 STEP 1783 H= 445:45 +CPU= 20.706
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:50:41 STEP 1784 H= 446:00 +CPU= 21.489
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:51:04 STEP 1785 H= 446:15 +CPU= 20.726
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:51:28 STEP 1786 H= 446:30 +CPU= 20.618
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:51:51 STEP 1787 H= 446:45 +CPU= 20.659
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:52:21 STEP 1788 H= 447:00 +CPU= 27.537
Moving to projects directory: /mnt/nvme/scratch1/boinc/slots/1/ICMGGh7zg+001788
Moving to projects directory: /mnt/nvme/scratch1/boinc/slots/1/ICMSHh7zg+001788
Moving to projects directory: /mnt/nvme/scratch1/boinc/slots/1/ICMUAh7zg+001788
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:52:44 STEP 1789 H= 447:15 +CPU= 20.793
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:53:08 STEP 1790 H= 447:30 +CPU= 20.689
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:53:31 STEP 1791 H= 447:45 +CPU= 21.087
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:53:55 STEP 1792 H= 448:00 +CPU= 21.980
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:54:19 STEP 1793 H= 448:15 +CPU= 21.397
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:54:42 STEP 1794 H= 448:30 +CPU= 21.383
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:55:06 STEP 1795 H= 448:45 +CPU= 21.366
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:55:32 STEP 1796 H= 449:00 +CPU= 22.212
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:55:55 STEP 1797 H= 449:15 +CPU= 21.352
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:56:18 STEP 1798 H= 449:30 +CPU= 20.893
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:56:41 STEP 1799 H= 449:45 +CPU= 20.698
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:57:12 STEP 1800 H= 450:00 +CPU= 27.016
Moving to projects directory: /mnt/nvme/scratch1/boinc/slots/1/ICMGGh7zg+001800
Moving to projects directory: /mnt/nvme/scratch1/boinc/slots/1/ICMSHh7zg+001800
Moving to projects directory: /mnt/nvme/scratch1/boinc/slots/1/ICMUAh7zg+001800
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:57:35 STEP 1801 H= 450:15 +CPU= 21.149
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:57:58 STEP 1802 H= 450:30 +CPU= 21.017
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:58:21 STEP 1803 H= 450:45 +CPU= 20.624
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:58:44 STEP 1804 H= 451:00 +CPU= 21.163
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:59:09 STEP 1805 H= 451:15 +CPU= 20.527
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:59:31 STEP 1806 H= 451:30 +CPU= 20.565
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:59:54 STEP 1807 H= 451:45 +CPU= 20.457
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:00:17 STEP 1808 H= 452:00 +CPU= 21.215
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:00:40 STEP 1809 H= 452:15 +CPU= 20.630
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:01:02 STEP 1810 H= 452:30 +CPU= 20.456
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:01:25 STEP 1811 H= 452:45 +CPU= 20.510
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:01:55 STEP 1812 H= 453:00 +CPU= 26.091
Moving to projects directory: /mnt/nvme/scratch1/boinc/slots/1/ICMGGh7zg+001812
Moving to projects directory: /mnt/nvme/scratch1/boinc/slots/1/ICMSHh7zg+001812
Moving to projects directory: /mnt/nvme/scratch1/boinc/slots/1/ICMUAh7zg+001812
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:02:17 STEP 1813 H= 453:15 +CPU= 20.438
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:02:40 STEP 1814 H= 453:30 +CPU= 20.396
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:03:02 STEP 1815 H= 453:45 +CPU= 20.798
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:03:28 STEP 1816 H= 454:00 +CPU= 21.204
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:03:50 STEP 1817 H= 454:15 +CPU= 20.475
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:04:13 STEP 1818 H= 454:30 +CPU= 20.358
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:04:35 STEP 1819 H= 454:45 +CPU= 20.388
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:04:58 STEP 1820 H= 455:00 +CPU= 21.089
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:05:20 STEP 1821 H= 455:15 +CPU= 20.353
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:05:43 STEP 1822 H= 455:30 +CPU= 20.387
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:06:05 STEP 1823 H= 455:45 +CPU= 20.331
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:06:39 STEP 1824 H= 456:00 +CPU= 29.422
Moving to projects directory: /mnt/nvme/scratch1/boinc/slots/1/ICMGGh7zg+001824
Moving to projects directory: /mnt/nvme/scratch1/boinc/slots/1/ICMSHh7zg+001824
Moving to projects directory: /mnt/nvme/scratch1/boinc/slots/1/ICMUAh7zg+001824
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:07:02 STEP 1825 H= 456:15 +CPU= 21.687
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:07:28 STEP 1826 H= 456:30 +CPU= 21.373
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:07:51 STEP 1827 H= 456:45 +CPU= 21.478
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:08:16 STEP 1828 H= 457:00 +CPU= 22.311
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:08:40 STEP 1829 H= 457:15 +CPU= 21.811
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:09:03 STEP 1830 H= 457:30 +CPU= 21.666
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:09:29 STEP 1831 H= 457:45 +CPU= 21.395
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:09:52 STEP 1832 H= 458:00 +CPU= 21.694
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:10:15 STEP 1833 H= 458:15 +CPU= 20.796
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:10:38 STEP 1834 H= 458:30 +CPU= 21.257
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:11:02 STEP 1835 H= 458:45 +CPU= 21.513
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:11:33 STEP 1836 H= 459:00 +CPU= 27.332
Moving to projects directory: /mnt/nvme/scratch1/boinc/slots/1/ICMGGh7zg+001836
Moving to projects directory: /mnt/nvme/scratch1/boinc/slots/1/ICMSHh7zg+001836
Moving to projects directory: /mnt/nvme/scratch1/boinc/slots/1/ICMUAh7zg+001836
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:11:57 STEP 1837 H= 459:15 +CPU= 21.290
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:12:20 STEP 1838 H= 459:30 +CPU= 21.591
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:12:43 STEP 1839 H= 459:45 +CPU= 21.263
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:13:10 STEP 1840 H= 460:00 +CPU= 22.157
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:13:33 STEP 1841 H= 460:15 +CPU= 21.329
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:13:56 STEP 1842 H= 460:30 +CPU= 21.376
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:14:20 STEP 1843 H= 460:45 +CPU= 21.428
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:14:44 STEP 1844 H= 461:00 +CPU= 21.561
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:15:08 STEP 1845 H= 461:15 +CPU= 20.554
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:15:31 STEP 1846 H= 461:30 +CPU= 21.149
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:15:55 STEP 1847 H= 461:45 +CPU= 21.375
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:16:24 STEP 1848 H= 462:00 +CPU= 27.585
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /mnt/nvme/scratch1/boinc/slots/1/ICMGGh7zg+001848
Moving to projects directory: /mnt/nvme/scratch1/boinc/slots/1/ICMSHh7zg+001848
Moving to projects directory: /mnt/nvme/scratch1/boinc/slots/1/ICMUAh7zg+001848
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:16:49 STEP 1849 H= 462:15 +CPU= 20.959
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:17:12 STEP 1850 H= 462:30 +CPU= 20.568
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:17:35 STEP 1851 H= 462:45 +CPU= 20.671
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:17:58 STEP 1852 H= 463:00 +CPU= 21.625
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:18:22 STEP 1853 H= 463:15 +CPU= 21.182
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:18:45 STEP 1854 H= 463:30 +CPU= 21.286
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:19:10 STEP 1855 H= 463:45 +CPU= 21.232
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:19:34 STEP 1856 H= 464:00 +CPU= 21.979
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:19:58 STEP 1857 H= 464:15 +CPU= 21.282
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:20:21 STEP 1858 H= 464:30 +CPU= 21.290
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:20:49 STEP 1859 H= 464:45 +CPU= 21.310
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:21:19 STEP 1860 H= 465:00 +CPU= 27.334
Moving to projects directory: /mnt/nvme/scratch1/boinc/slots/1/ICMGGh7zg+001860
Moving to projects directory: /mnt/nvme/scratch1/boinc/slots/1/ICMSHh7zg+001860
Moving to projects directory: /mnt/nvme/scratch1/boinc/slots/1/ICMUAh7zg+001860
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:21:42 STEP 1861 H= 465:15 +CPU= 20.749
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:22:05 STEP 1862 H= 465:30 +CPU= 20.920
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:22:30 STEP 1863 H= 465:45 +CPU= 21.351
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:22:54 STEP 1864 H= 466:00 +CPU= 22.070
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:23:18 STEP 1865 H= 466:15 +CPU= 21.376
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:23:41 STEP 1866 H= 466:30 +CPU= 21.225
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:24:05 STEP 1867 H= 466:45 +CPU= 21.589
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:24:31 STEP 1868 H= 467:00 +CPU= 22.282
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:24:54 STEP 1869 H= 467:15 +CPU= 21.242
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:25:17 STEP 1870 H= 467:30 +CPU= 20.927
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:25:40 STEP 1871 H= 467:45 +CPU= 20.614
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:26:13 STEP 1872 H= 468:00 +CPU= 29.251
Moving to projects directory: /mnt/nvme/scratch1/boinc/slots/1/ICMGGh7zg+001872
Moving to projects directory: /mnt/nvme/scratch1/boinc/slots/1/ICMSHh7zg+001872
Moving to projects directory: /mnt/nvme/scratch1/boinc/slots/1/ICMUAh7zg+001872
Adding to the zip: /mnt/nvme/scratch1/boinc/projects/climateprediction.net/oifs_43r3_bl_12292510/ICMGGh7zg+001776
Adding to the zip: /mnt/nvme/scratch1/boinc/projects/climateprediction.net/oifs_43r3_bl_12292510/ICMSHh7zg+001776
Adding to the zip: /mnt/nvme/scratch1/boinc/projects/climateprediction.net/oifs_43r3_bl_12292510/ICMUAh7zg+001776
Adding to the zip: /mnt/nvme/scratch1/boinc/projects/climateprediction.net/oifs_43r3_bl_12292510/ICMGGh7zg+001788
Adding to the zip: /mnt/nvme/scratch1/boinc/projects/climateprediction.net/oifs_43r3_bl_12292510/ICMSHh7zg+001788
Adding to the zip: /mnt/nvme/scratch1/boinc/projects/climateprediction.net/oifs_43r3_bl_12292510/ICMUAh7zg+001788
Adding to the zip: /mnt/nvme/scratch1/boinc/projects/climateprediction.net/oifs_43r3_bl_12292510/ICMGGh7zg+001800
Adding to the zip: /mnt/nvme/scratch1/boinc/projects/climateprediction.net/oifs_43r3_bl_12292510/ICMSHh7zg+001800
Adding to the zip: /mnt/nvme/scratch1/boinc/projects/climateprediction.net/oifs_43r3_bl_12292510/ICMUAh7zg+001800
Adding to the zip: /mnt/nvme/scratch1/boinc/projects/climateprediction.net/oifs_43r3_bl_12292510/ICMGGh7zg+001812
Adding to the zip: /mnt/nvme/scratch1/boinc/projects/climateprediction.net/oifs_43r3_bl_12292510/ICMSHh7zg+001812
Adding to the zip: /mnt/nvme/scratch1/boinc/projects/climateprediction.net/oifs_43r3_bl_12292510/ICMUAh7zg+001812
Adding to the zip: /mnt/nvme/scratch1/boinc/projects/climateprediction.net/oifs_43r3_bl_12292510/ICMGGh7zg+001824
Adding to the zip: /mnt/nvme/scratch1/boinc/projects/climateprediction.net/oifs_43r3_bl_12292510/ICMSHh7zg+001824
Adding to the zip: /mnt/nvme/scratch1/boinc/projects/climateprediction.net/oifs_43r3_bl_12292510/ICMUAh7zg+001824
Adding to the zip: /mnt/nvme/scratch1/boinc/projects/climateprediction.net/oifs_43r3_bl_12292510/ICMGGh7zg+001836
Adding to the zip: /mnt/nvme/scratch1/boinc/projects/climateprediction.net/oifs_43r3_bl_12292510/ICMSHh7zg+001836
Adding to the zip: /mnt/nvme/scratch1/boinc/projects/climateprediction.net/oifs_43r3_bl_12292510/ICMUAh7zg+001836
Adding to the zip: /mnt/nvme/scratch1/boinc/projects/climateprediction.net/oifs_43r3_bl_12292510/ICMGGh7zg+001848
Adding to the zip: /mnt/nvme/scratch1/boinc/projects/climateprediction.net/oifs_43r3_bl_12292510/ICMSHh7zg+001848
Adding to the zip: /mnt/nvme/scratch1/boinc/projects/climateprediction.net/oifs_43r3_bl_12292510/ICMUAh7zg+001848
Adding to the zip: /mnt/nvme/scratch1/boinc/projects/climateprediction.net/oifs_43r3_bl_12292510/ICMGGh7zg+001860
Adding to the zip: /mnt/nvme/scratch1/boinc/projects/climateprediction.net/oifs_43r3_bl_12292510/ICMSHh7zg+001860
Adding to the zip: /mnt/nvme/scratch1/boinc/projects/climateprediction.net/oifs_43r3_bl_12292510/ICMUAh7zg+001860
Zipping up the intermediate file: /mnt/nvme/scratch1/boinc/projects/climateprediction.net/oifs_43r3_bl_a387_2016092300_20_1018_12292510_0_r205782200_13.zip
Uploading the intermediate file: upload_file_13.zip
  11:26:37 STEP 1873 H= 468:15 +CPU= 21.379
  11:26:58 STEP 1874 H= 468:30 +CPU= 20.845
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:27:21 STEP 1875 H= 468:45 +CPU= 20.508
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:27:44 STEP 1876 H= 469:00 +CPU= 21.242
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:28:09 STEP 1877 H= 469:15 +CPU= 21.064
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:28:32 STEP 1878 H= 469:30 +CPU= 20.990
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:28:54 STEP 1879 H= 469:45 +CPU= 20.386
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:29:18 STEP 1880 H= 470:00 +CPU= 21.182
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:29:41 STEP 1881 H= 470:15 +CPU= 21.010
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:30:04 STEP 1882 H= 470:30 +CPU= 21.303
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:30:29 STEP 1883 H= 470:45 +CPU= 21.367
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:30:59 STEP 1884 H= 471:00 +CPU= 27.449
Moving to projects directory: /mnt/nvme/scratch1/boinc/slots/1/ICMGGh7zg+001884
Moving to projects directory: /mnt/nvme/scratch1/boinc/slots/1/ICMSHh7zg+001884
Moving to projects directory: /mnt/nvme/scratch1/boinc/slots/1/ICMUAh7zg+001884
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:31:21 STEP 1885 H= 471:15 +CPU= 20.450
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:31:44 STEP 1886 H= 471:30 +CPU= 20.342
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:32:08 STEP 1887 H= 471:45 +CPU= 20.310
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:32:32 STEP 1888 H= 472:00 +CPU= 21.441
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:32:55 STEP 1889 H= 472:15 +CPU= 20.774
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:33:17 STEP 1890 H= 472:30 +CPU= 20.750
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:33:40 STEP 1891 H= 472:45 +CPU= 20.599
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:34:04 STEP 1892 H= 473:00 +CPU= 21.457
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:34:28 STEP 1893 H= 473:15 +CPU= 20.647
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:34:51 STEP 1894 H= 473:30 +CPU= 20.523
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:35:13 STEP 1895 H= 473:45 +CPU= 20.486
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:35:41 STEP 1896 H= 474:00 +CPU= 26.044
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /mnt/nvme/scratch1/boinc/slots/1/ICMGGh7zg+001896
Moving to projects directory: /mnt/nvme/scratch1/boinc/slots/1/ICMSHh7zg+001896
Moving to projects directory: /mnt/nvme/scratch1/boinc/slots/1/ICMUAh7zg+001896
  11:36:04 STEP 1897 H= 474:15 +CPU= 20.294
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:36:28 STEP 1898 H= 474:30 +CPU= 20.335
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:36:50 STEP 1899 H= 474:45 +CPU= 20.308
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:37:13 STEP 1900 H= 475:00 +CPU= 21.032
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:37:36 STEP 1901 H= 475:15 +CPU= 20.468
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:37:59 STEP 1902 H= 475:30 +CPU= 20.545
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:38:21 STEP 1903 H= 475:45 +CPU= 20.567
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:38:46 STEP 1904 H= 476:00 +CPU= 21.237
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:39:09 STEP 1905 H= 476:15 +CPU= 20.622
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:39:32 STEP 1906 H= 476:30 +CPU= 20.582
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:39:54 STEP 1907 H= 476:45 +CPU= 20.536
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:40:22 STEP 1908 H= 477:00 +CPU= 26.088
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /mnt/nvme/scratch1/boinc/slots/1/ICMGGh7zg+001908
Moving to projects directory: /mnt/nvme/scratch1/boinc/slots/1/ICMSHh7zg+001908
Moving to projects directory: /mnt/nvme/scratch1/boinc/slots/1/ICMUAh7zg+001908
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:40:47 STEP 1909 H= 477:15 +CPU= 20.493
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:41:09 STEP 1910 H= 477:30 +CPU= 20.476
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:41:32 STEP 1911 H= 477:45 +CPU= 20.633
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:41:56 STEP 1912 H= 478:00 +CPU= 21.421
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:42:18 STEP 1913 H= 478:15 +CPU= 20.739
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:42:41 STEP 1914 H= 478:30 +CPU= 20.489
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:43:03 STEP 1915 H= 478:45 +CPU= 20.419
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:43:28 STEP 1916 H= 479:00 +CPU= 21.083
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:43:51 STEP 1917 H= 479:15 +CPU= 20.555
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:44:14 STEP 1918 H= 479:30 +CPU= 20.701
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:44:37 STEP 1919 H= 479:45 +CPU= 20.781
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:45:09 STEP 1920 H= 480:00 +CPU= 28.839
..The child process terminated with status: 0
>>> Printing last 70 lines from file: NODE.001_01

 
DDH------------------------------------------------------------------------
I                                                                         I
I TIME =  480.000H DDH * Diagnostic Var.,     AREA                GLOBAL  I
I                                                                         I
I-------------------------------------------------------------------------I
I   VARIABLE   I      Initial value         I        Current value        I
I-------------------------------------------------------------------------I
I  OMEGA (P/S) I     -0.1467150861E-20      I       0.4419872360E-05      I
ICloud fractionI      0.0000000000E+00      I       0.5787121638E-01      I
I  Relat. Hum. I      0.3967781272E+00      I       0.4503585404E+00      I
I  PASS 01     I      0.0000000000E+00      I       0.9423782793E+03      I
DDH------------------------------------------------------------------------
  
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
 NSTEP =  1920 SCAN2M_HPOS  P
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ICMGGh7zg+001920 MODE=a
 IO-STREAM CLOSED - ICMGGh7zg+001920
 MPI-TASK:   1 -    321155204 BYTES IN      7 RECORDS TRANSFERRED IN    0.0001 SECONDS******** Mbytes/s, TOTAL TIME=    0.0079( 1.3%)
 11:45:09 STEP 1920 H= 480:00 +CPU= 28.839
 END CNT3
 NSTEP =  1920 CNT0   000000000  20.676  20.676  22.692   0.911               0               0               0               0               0
IO-STREAM STATISTICS, TOTAL NO OF REC/BYTES READ           46/  1069125708 WRITTEN        17160/************
 -TASK-OPENED-OPEN-RECS IN -KBYTE IN    -RECS OUT-KBYTE OUT-WALL   -WALL IN-WALL
  OU-TOT IN -TOT OUT
   1    303    0       46      1044068    17160 ************    23.4     1.3    22.1     1.6    47.0
===-=== START OF TIMING STATISTICS ===-===
 
STATS FOR ALL TASKS
 NUM ROUTINE                                     CALLS  MEAN(ms)   MAX(ms)   FRAC(%)  UNBAL(%)
   0 CNT0     - COMPLETE EXECUTION                   1 ********* *********    100.00      0.00
   1 CNT4     - FORWARD INTEGRATION                  1 ********* *********     99.98      0.00
   8 SCAN2M - GRID-POINT DYNAMICS                  625    3615.8    3615.8     16.73      0.00
   9 SPCM     - SPECTRAL COMP.                     576     409.4     409.4      1.75      0.00
  10 SCAN2M - PHYSICS                              577   11722.5   11722.5     50.08      0.00
  11 IOPACK   - OUTPUT P.P. RESULTS                 48     558.1     558.1      0.20      0.00
  12 SPNORM   - SPECTRAL NORM COMP.                 25      22.4      22.4      0.00      0.00
  17 GRIDFPOS IN CNT4                               48      18.9      18.9      0.01      0.00
  27 RESTART FILES - WRITING                        12    2232.4    2232.4      0.20      0.00
  28 RESTART FILES - READING                         1    2080.4    2080.4      0.02      0.00
  29 SU4FPOS IN CNT4                                48       0.2       0.2      0.00      0.00
  30 DYNFPOS IN CNT4                                48    5422.7    5422.7      1.93      0.00
  31 POSDDH IN STEPO                                 3       3.6       3.6      0.00      0.00
  37 CPGLAG   - SL COMPUTATIONS                    577    5240.0    5240.0     22.39      0.00
  39 SU0YOMB                                         1     295.5     295.5      0.00      0.00
  51 SCAN2M   - SL COMM. PART 1                    577      11.1      11.1      0.05      0.00
  54 SPCM     - M TO S/S TO M TRANSP.              576      78.3      78.3      0.33      0.00
  55 SPCIMPF  - S TO M/M TO S TRANSP.              576      30.1      30.1      0.13      0.00
  56 SPNORM   - SPECTRAL NORM COMM.                 25       0.2       0.2      0.00      0.00
 102 LTINV_CTL   - INVERSE LEGENDRE TRANSFORM     1252     279.5     279.5      2.59      0.00
 103 LTDIR_CTL   - DIRECT LEGENDRE TRANSFORM      1200     183.8     183.8      1.63      0.00
 106 FTDIR_CTL   - DIRECT FOURIER TRANSFORM       1200      56.7      56.7      0.50      0.00
 107 FTINV_CTL   - INVERSE FOURIER TRANSFORM      1252      91.5      91.5      0.85      0.00
 140 SULEG       - COMP. OF LEGENDRE POL.            1      37.9      37.9      0.00      0.00
 152 LTINV_CTL   - M TO L TRANSPOSITION           1252      40.8      40.8      0.38      0.00
 153 LTDIR_CTL   - L TO M TRANSPOSITION           1200      16.3      16.3      0.15      0.00
 157 FTINV_CTL   - L TO G TRANSPOSITION           1252      30.6      30.6      0.28      0.00
 158 FTDIR_CTL   - G TO L TRANSPOSITION           1200      23.7      23.7      0.21      0.00
 400 GSTATS                                      67705       0.0       0.0      0.00      0.00
 401 GSTATS HOOK                                 62941       0.0       0.0      0.00      0.00
TOTAL MEASURED IMBALANCE =       0.0 SECONDS,  0.0 PERCENT
TOTAL WALLCLOCK TIME    13505.9 CPU TIME   12141.6 VECTOR TIME    12141.6


===-=== END   OF TIMING STATISTICS ===-===
FORECAST DAYS PER DAY      128.0
  *** END CNT0 *** 
------------------------------------------------
Moving to projects directory: /mnt/nvme/scratch1/boinc/slots/1/ICMGGh7zg+001920
Moving to projects directory: /mnt/nvme/scratch1/boinc/slots/1/ICMSHh7zg+001920
Moving to projects directory: /mnt/nvme/scratch1/boinc/slots/1/ICMUAh7zg+001920
Adding to the zip: /mnt/nvme/scratch1/boinc/slots/1/NODE.001_01
Adding to the zip: /mnt/nvme/scratch1/boinc/slots/1/ifs.stat
Adding to the zip: /mnt/nvme/scratch1/boinc/projects/climateprediction.net/oifs_43r3_bl_12292510/ICMGGh7zg+001872
Adding to the zip: /mnt/nvme/scratch1/boinc/projects/climateprediction.net/oifs_43r3_bl_12292510/ICMSHh7zg+001872
Adding to the zip: /mnt/nvme/scratch1/boinc/projects/climateprediction.net/oifs_43r3_bl_12292510/ICMUAh7zg+001872
Adding to the zip: /mnt/nvme/scratch1/boinc/projects/climateprediction.net/oifs_43r3_bl_12292510/ICMGGh7zg+001884
Adding to the zip: /mnt/nvme/scratch1/boinc/projects/climateprediction.net/oifs_43r3_bl_12292510/ICMSHh7zg+001884
Adding to the zip: /mnt/nvme/scratch1/boinc/projects/climateprediction.net/oifs_43r3_bl_12292510/ICMUAh7zg+001884
Adding to the zip: /mnt/nvme/scratch1/boinc/projects/climateprediction.net/oifs_43r3_bl_12292510/ICMGGh7zg+001896
Adding to the zip: /mnt/nvme/scratch1/boinc/projects/climateprediction.net/oifs_43r3_bl_12292510/ICMSHh7zg+001896
Adding to the zip: /mnt/nvme/scratch1/boinc/projects/climateprediction.net/oifs_43r3_bl_12292510/ICMUAh7zg+001896
Adding to the zip: /mnt/nvme/scratch1/boinc/projects/climateprediction.net/oifs_43r3_bl_12292510/ICMGGh7zg+001908
Adding to the zip: /mnt/nvme/scratch1/boinc/projects/climateprediction.net/oifs_43r3_bl_12292510/ICMSHh7zg+001908
Adding to the zip: /mnt/nvme/scratch1/boinc/projects/climateprediction.net/oifs_43r3_bl_12292510/ICMUAh7zg+001908
Adding to the zip: /mnt/nvme/scratch1/boinc/projects/climateprediction.net/oifs_43r3_bl_12292510/ICMGGh7zg+001920
Adding to the zip: /mnt/nvme/scratch1/boinc/projects/climateprediction.net/oifs_43r3_bl_12292510/ICMSHh7zg+001920
Adding to the zip: /mnt/nvme/scratch1/boinc/projects/climateprediction.net/oifs_43r3_bl_12292510/ICMUAh7zg+001920
Zipping up the final file: /mnt/nvme/scratch1/boinc/projects/climateprediction.net/oifs_43r3_bl_a387_2016092300_20_1018_12292510_0_r205782200_14.zip
Uploading the final file: upload_file_14.zip
Uploading trickle at timestep: 1727100
11:48:37 (1525005): called boinc_finish(0)

</stderr_txt>
<message>
upload failure: <file_xfer_error>
  <file_name>oifs_43r3_bl_a387_2016092300_20_1018_12292510_0_r205782200_15.zip</file_name>
  <error_code>-161 (not found)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>oifs_43r3_bl_a387_2016092300_20_1018_12292510_0_r205782200_16.zip</file_name>
  <error_code>-161 (not found)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>oifs_43r3_bl_a387_2016092300_20_1018_12292510_0_r205782200_17.zip</file_name>
  <error_code>-161 (not found)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>oifs_43r3_bl_a387_2016092300_20_1018_12292510_0_r205782200_18.zip</file_name>
  <error_code>-161 (not found)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>oifs_43r3_bl_a387_2016092300_20_1018_12292510_0_r205782200_19.zip</file_name>
  <error_code>-161 (not found)</error_code>
</file_xfer_error>
</message>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
18 Jun 2024 09:47:06 1489681 22446116 oifs_43r3_bl_a387_2016092300_20_1018_12292510_0 1,727,100 27,931 0.0162


©2024 climateprediction.net