climateprediction.net home page
Task 22446130

Task 22446130

Name oifs_43r3_bl_a38l_2016092300_20_1018_12292524_0
Workunit 12292524
Created 13 Jun 2024, 2:06:16 UTC
Sent 14 Jun 2024, 2:36:13 UTC
Report deadline 13 Aug 2024, 2:36:13 UTC
Received 15 Jun 2024, 17:46:49 UTC
Server state Over
Outcome Success
Client state Done
Exit status 0 (0x00000000)
Computer ID 1537687
Run time 15 hours 44 min 54 sec
CPU time 15 hours 44 min 54 sec
Validate state Valid
Credit 2,635.55
Device peak FLOPS 7.20 GFLOPS
Application version OpenIFS 43r3 Baroclinic Lifecycle v1.13
x86_64-pc-linux-gnu
Peak working set size 5,014.96 MB
Peak swap size 5,981.13 MB
Peak disk usage 1,389.29 MB
Stderr
<core_client_version>7.18.1</core_client_version>
<![CDATA[
<stderr_txt>
ar/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMGGh7zg+001440
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMSHh7zg+001440
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMUAh7zg+001440
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a38l_2016092300_20_1018_12292524_0_r1750201042_14.zip
Uploading the intermediate file: upload_file_14.zip
  11:43:04 STEP 1442 H= 360:30 +CPU= 26.045
  11:43:31 STEP 1443 H= 360:45 +CPU= 26.480
  11:43:58 STEP 1444 H= 361:00 +CPU= 27.341
  11:44:26 STEP 1445 H= 361:15 +CPU= 27.269
  11:44:54 STEP 1446 H= 361:30 +CPU= 27.997
  11:45:23 STEP 1447 H= 361:45 +CPU= 28.476
  11:45:54 STEP 1448 H= 362:00 +CPU= 30.229
  11:46:24 STEP 1449 H= 362:15 +CPU= 29.408
  11:46:53 STEP 1450 H= 362:30 +CPU= 28.619
  11:47:22 STEP 1451 H= 362:45 +CPU= 28.843
  11:47:57 STEP 1452 H= 363:00 +CPU= 35.005
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMGGh7zg+001452
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMSHh7zg+001452
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMUAh7zg+001452
  11:48:26 STEP 1453 H= 363:15 +CPU= 28.557
  11:48:55 STEP 1454 H= 363:30 +CPU= 28.560
  11:49:24 STEP 1455 H= 363:45 +CPU= 28.793
  11:49:53 STEP 1456 H= 364:00 +CPU= 29.451
  11:50:23 STEP 1457 H= 364:15 +CPU= 29.252
  11:50:52 STEP 1458 H= 364:30 +CPU= 28.592
  11:51:21 STEP 1459 H= 364:45 +CPU= 28.760
  11:51:51 STEP 1460 H= 365:00 +CPU= 28.951
  11:52:20 STEP 1461 H= 365:15 +CPU= 28.828
  11:52:49 STEP 1462 H= 365:30 +CPU= 28.968
  11:53:19 STEP 1463 H= 365:45 +CPU= 29.174
  11:53:54 STEP 1464 H= 366:00 +CPU= 35.118
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMGGh7zg+001464
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMSHh7zg+001464
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMUAh7zg+001464
  11:54:24 STEP 1465 H= 366:15 +CPU= 28.504
  11:54:53 STEP 1466 H= 366:30 +CPU= 28.980
  11:55:22 STEP 1467 H= 366:45 +CPU= 28.815
  11:55:53 STEP 1468 H= 367:00 +CPU= 29.901
  11:56:22 STEP 1469 H= 367:15 +CPU= 29.208
  11:56:52 STEP 1470 H= 367:30 +CPU= 28.701
  11:57:20 STEP 1471 H= 367:45 +CPU= 28.630
  11:57:51 STEP 1472 H= 368:00 +CPU= 29.641
  11:58:20 STEP 1473 H= 368:15 +CPU= 28.941
  11:58:48 STEP 1474 H= 368:30 +CPU= 28.707
  11:59:18 STEP 1475 H= 368:45 +CPU= 28.815
  11:59:54 STEP 1476 H= 369:00 +CPU= 36.157
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMGGh7zg+001476
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMSHh7zg+001476
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMUAh7zg+001476
  12:00:23 STEP 1477 H= 369:15 +CPU= 28.920
  12:00:52 STEP 1478 H= 369:30 +CPU= 28.734
  12:01:22 STEP 1479 H= 369:45 +CPU= 28.636
  12:01:52 STEP 1480 H= 370:00 +CPU= 30.110
  12:02:22 STEP 1481 H= 370:15 +CPU= 29.058
  12:02:51 STEP 1482 H= 370:30 +CPU= 28.968
  12:03:21 STEP 1483 H= 370:45 +CPU= 29.274
  12:03:53 STEP 1484 H= 371:00 +CPU= 29.285
  12:04:23 STEP 1485 H= 371:15 +CPU= 29.094
  12:04:52 STEP 1486 H= 371:30 +CPU= 29.057
  12:05:22 STEP 1487 H= 371:45 +CPU= 28.667
  12:06:03 STEP 1488 H= 372:00 +CPU= 39.708
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMGGh7zg+001488
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMSHh7zg+001488
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMUAh7zg+001488
  12:06:32 STEP 1489 H= 372:15 +CPU= 28.883
  12:07:01 STEP 1490 H= 372:30 +CPU= 28.184
  12:07:30 STEP 1491 H= 372:45 +CPU= 28.751
  12:08:01 STEP 1492 H= 373:00 +CPU= 29.496
  12:08:29 STEP 1493 H= 373:15 +CPU= 28.328
  12:08:58 STEP 1494 H= 373:30 +CPU= 28.208
  12:09:27 STEP 1495 H= 373:45 +CPU= 28.000
  12:09:57 STEP 1496 H= 374:00 +CPU= 29.710
  12:10:26 STEP 1497 H= 374:15 +CPU= 28.777
  12:10:54 STEP 1498 H= 374:30 +CPU= 28.575
  12:11:23 STEP 1499 H= 374:45 +CPU= 28.707
  12:11:59 STEP 1500 H= 375:00 +CPU= 35.891
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMGGh7zg+001500
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMSHh7zg+001500
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMUAh7zg+001500
  12:12:28 STEP 1501 H= 375:15 +CPU= 28.565
  12:12:57 STEP 1502 H= 375:30 +CPU= 29.188
  12:13:26 STEP 1503 H= 375:45 +CPU= 28.967
  12:13:56 STEP 1504 H= 376:00 +CPU= 29.313
  12:14:25 STEP 1505 H= 376:15 +CPU= 29.158
  12:14:55 STEP 1506 H= 376:30 +CPU= 29.340
  12:15:24 STEP 1507 H= 376:45 +CPU= 28.773
  12:15:53 STEP 1508 H= 377:00 +CPU= 28.901
  12:16:22 STEP 1509 H= 377:15 +CPU= 28.419
  12:16:50 STEP 1510 H= 377:30 +CPU= 28.584
  12:17:20 STEP 1511 H= 377:45 +CPU= 28.765
  12:17:56 STEP 1512 H= 378:00 +CPU= 35.139
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMGGh7zg+001512
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMSHh7zg+001512
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMUAh7zg+001512
  12:18:25 STEP 1513 H= 378:15 +CPU= 28.620
  12:18:55 STEP 1514 H= 378:30 +CPU= 29.250
  12:19:24 STEP 1515 H= 378:45 +CPU= 28.553
  12:19:54 STEP 1516 H= 379:00 +CPU= 28.764
  12:20:22 STEP 1517 H= 379:15 +CPU= 28.680
  12:20:51 STEP 1518 H= 379:30 +CPU= 28.679
  12:21:21 STEP 1519 H= 379:45 +CPU= 28.440
  12:21:50 STEP 1520 H= 380:00 +CPU= 28.791
  12:22:19 STEP 1521 H= 380:15 +CPU= 28.544
  12:22:49 STEP 1522 H= 380:30 +CPU= 28.720
  12:23:18 STEP 1523 H= 380:45 +CPU= 28.821
  12:23:54 STEP 1524 H= 381:00 +CPU= 35.574
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMGGh7zg+001524
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMSHh7zg+001524
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMUAh7zg+001524
  12:24:23 STEP 1525 H= 381:15 +CPU= 28.819
  12:24:52 STEP 1526 H= 381:30 +CPU= 28.867
  12:25:21 STEP 1527 H= 381:45 +CPU= 28.610
  12:25:50 STEP 1528 H= 382:00 +CPU= 29.123
  12:26:19 STEP 1529 H= 382:15 +CPU= 28.498
  12:26:48 STEP 1530 H= 382:30 +CPU= 29.045
  12:27:17 STEP 1531 H= 382:45 +CPU= 28.317
  12:27:46 STEP 1532 H= 383:00 +CPU= 28.704
  12:28:16 STEP 1533 H= 383:15 +CPU= 29.137
  12:28:45 STEP 1534 H= 383:30 +CPU= 28.525
  12:29:13 STEP 1535 H= 383:45 +CPU= 28.446
  12:29:53 STEP 1536 H= 384:00 +CPU= 38.984
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMGGh7zg+001536
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMSHh7zg+001536
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMUAh7zg+001536
  12:30:22 STEP 1537 H= 384:15 +CPU= 29.261
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMGGh7zg+001452
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMSHh7zg+001452
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMUAh7zg+001452
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMGGh7zg+001464
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMSHh7zg+001464
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMUAh7zg+001464
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMGGh7zg+001476
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMSHh7zg+001476
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMUAh7zg+001476
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMGGh7zg+001488
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMSHh7zg+001488
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMUAh7zg+001488
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMGGh7zg+001500
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMSHh7zg+001500
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMUAh7zg+001500
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMGGh7zg+001512
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMSHh7zg+001512
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMUAh7zg+001512
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMGGh7zg+001524
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMSHh7zg+001524
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMUAh7zg+001524
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMGGh7zg+001536
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMSHh7zg+001536
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMUAh7zg+001536
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a38l_2016092300_20_1018_12292524_0_r1750201042_15.zip
Uploading the intermediate file: upload_file_15.zip
  12:30:51 STEP 1538 H= 384:30 +CPU= 27.677
  12:31:20 STEP 1539 H= 384:45 +CPU= 28.755
  12:31:49 STEP 1540 H= 385:00 +CPU= 29.174
  12:32:19 STEP 1541 H= 385:15 +CPU= 28.927
  12:32:48 STEP 1542 H= 385:30 +CPU= 28.773
  12:33:17 STEP 1543 H= 385:45 +CPU= 28.589
  12:33:47 STEP 1544 H= 386:00 +CPU= 29.920
  12:34:16 STEP 1545 H= 386:15 +CPU= 28.800
  12:34:44 STEP 1546 H= 386:30 +CPU= 28.214
  12:35:12 STEP 1547 H= 386:45 +CPU= 28.040
  12:35:47 STEP 1548 H= 387:00 +CPU= 34.099
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMGGh7zg+001548
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMSHh7zg+001548
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMUAh7zg+001548
  12:36:14 STEP 1549 H= 387:15 +CPU= 27.634
  12:36:41 STEP 1550 H= 387:30 +CPU= 26.795
  12:37:06 STEP 1551 H= 387:45 +CPU= 24.965
  12:37:32 STEP 1552 H= 388:00 +CPU= 25.503
  12:37:58 STEP 1553 H= 388:15 +CPU= 26.250
  12:38:25 STEP 1554 H= 388:30 +CPU= 26.751
  12:38:52 STEP 1555 H= 388:45 +CPU= 27.396
  12:39:22 STEP 1556 H= 389:00 +CPU= 29.581
  12:39:51 STEP 1557 H= 389:15 +CPU= 28.674
  12:40:20 STEP 1558 H= 389:30 +CPU= 28.932
  12:40:51 STEP 1559 H= 389:45 +CPU= 28.798
  12:41:28 STEP 1560 H= 390:00 +CPU= 36.389
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMGGh7zg+001560
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMSHh7zg+001560
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMUAh7zg+001560
  12:41:58 STEP 1561 H= 390:15 +CPU= 28.853
  12:42:27 STEP 1562 H= 390:30 +CPU= 28.994
  12:42:56 STEP 1563 H= 390:45 +CPU= 28.823
  12:43:26 STEP 1564 H= 391:00 +CPU= 29.952
  12:43:55 STEP 1565 H= 391:15 +CPU= 28.883
  12:44:24 STEP 1566 H= 391:30 +CPU= 28.375
  12:44:52 STEP 1567 H= 391:45 +CPU= 28.513
  12:45:23 STEP 1568 H= 392:00 +CPU= 30.090
  12:45:52 STEP 1569 H= 392:15 +CPU= 28.637
  12:46:20 STEP 1570 H= 392:30 +CPU= 28.247
  12:46:49 STEP 1571 H= 392:45 +CPU= 28.747
  12:47:25 STEP 1572 H= 393:00 +CPU= 35.909
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMGGh7zg+001572
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMSHh7zg+001572
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMUAh7zg+001572
  12:47:54 STEP 1573 H= 393:15 +CPU= 28.665
  12:48:23 STEP 1574 H= 393:30 +CPU= 28.801
  12:48:52 STEP 1575 H= 393:45 +CPU= 28.903
  12:49:22 STEP 1576 H= 394:00 +CPU= 29.673
  12:49:51 STEP 1577 H= 394:15 +CPU= 29.143
  12:50:21 STEP 1578 H= 394:30 +CPU= 29.430
  12:50:50 STEP 1579 H= 394:45 +CPU= 27.799
  12:51:20 STEP 1580 H= 395:00 +CPU= 29.851
  12:51:49 STEP 1581 H= 395:15 +CPU= 28.840
  12:52:19 STEP 1582 H= 395:30 +CPU= 29.049
  12:52:49 STEP 1583 H= 395:45 +CPU= 28.388
  12:53:29 STEP 1584 H= 396:00 +CPU= 39.642
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMGGh7zg+001584
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMSHh7zg+001584
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMUAh7zg+001584
  12:53:57 STEP 1585 H= 396:15 +CPU= 27.973
  12:54:23 STEP 1586 H= 396:30 +CPU= 26.058
  12:54:49 STEP 1587 H= 396:45 +CPU= 25.389
  12:55:14 STEP 1588 H= 397:00 +CPU= 25.553
  12:55:41 STEP 1589 H= 397:15 +CPU= 26.214
  12:56:08 STEP 1590 H= 397:30 +CPU= 27.300
  12:56:34 STEP 1591 H= 397:45 +CPU= 25.558
  12:57:00 STEP 1592 H= 398:00 +CPU= 26.376
  12:57:26 STEP 1593 H= 398:15 +CPU= 26.181
  12:57:55 STEP 1594 H= 398:30 +CPU= 28.350
  12:58:24 STEP 1595 H= 398:45 +CPU= 28.948
  12:59:00 STEP 1596 H= 399:00 +CPU= 36.170
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMGGh7zg+001596
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMSHh7zg+001596
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMUAh7zg+001596
  12:59:30 STEP 1597 H= 399:15 +CPU= 29.094
  12:59:58 STEP 1598 H= 399:30 +CPU= 28.497
  13:00:29 STEP 1599 H= 399:45 +CPU= 29.544
  13:00:59 STEP 1600 H= 400:00 +CPU= 29.706
  13:01:28 STEP 1601 H= 400:15 +CPU= 29.178
  13:01:58 STEP 1602 H= 400:30 +CPU= 29.244
  13:02:27 STEP 1603 H= 400:45 +CPU= 28.780
  13:02:57 STEP 1604 H= 401:00 +CPU= 30.171
  13:03:26 STEP 1605 H= 401:15 +CPU= 28.919
  13:03:55 STEP 1606 H= 401:30 +CPU= 28.639
  13:04:24 STEP 1607 H= 401:45 +CPU= 28.971
  13:05:02 STEP 1608 H= 402:00 +CPU= 36.838
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMGGh7zg+001608
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMSHh7zg+001608
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMUAh7zg+001608
  13:05:31 STEP 1609 H= 402:15 +CPU= 28.569
  13:06:00 STEP 1610 H= 402:30 +CPU= 28.880
  13:06:29 STEP 1611 H= 402:45 +CPU= 28.030
  13:06:58 STEP 1612 H= 403:00 +CPU= 29.616
  13:07:28 STEP 1613 H= 403:15 +CPU= 29.446
  13:07:58 STEP 1614 H= 403:30 +CPU= 29.145
  13:08:27 STEP 1615 H= 403:45 +CPU= 29.279
  13:08:57 STEP 1616 H= 404:00 +CPU= 29.321
  13:09:27 STEP 1617 H= 404:15 +CPU= 29.211
  13:09:56 STEP 1618 H= 404:30 +CPU= 28.872
  13:10:25 STEP 1619 H= 404:45 +CPU= 29.081
  13:11:03 STEP 1620 H= 405:00 +CPU= 37.218
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMGGh7zg+001620
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMSHh7zg+001620
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMUAh7zg+001620
  13:11:32 STEP 1621 H= 405:15 +CPU= 28.728
  13:12:01 STEP 1622 H= 405:30 +CPU= 28.891
  13:12:30 STEP 1623 H= 405:45 +CPU= 28.773
  13:12:56 STEP 1624 H= 406:00 +CPU= 25.949
  13:13:20 STEP 1625 H= 406:15 +CPU= 23.724
  13:13:40 STEP 1626 H= 406:30 +CPU= 20.419
  13:14:05 STEP 1627 H= 406:45 +CPU= 25.013
  13:14:30 STEP 1628 H= 407:00 +CPU= 24.701
  13:14:56 STEP 1629 H= 407:15 +CPU= 25.787
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_bl_1.13_x86_64-pc-linux-gnu
(argv1) start_date: 2016092300
(argv2) exptid: h7zg
(argv3) unique_member_id: a38l
(argv4) batchid: 1018
(argv5) wuid: 12292524
(argv6) fclen: 20
(argv7) app_name: oifs_43r3_bl
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/16
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_bl
version: 1.13
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/16/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/16/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_58e1878e287563f6f45c91fb2e0ddc1f to: /var/lib/boinc-client/slots/16/oifs_43r3_bl_a38l_2016092300_20_1018_12292524.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/16/oifs_43r3_bl_a38l_2016092300_20_1018_12292524.zip
ic_ancil_file: ic_ancil_12292524
ifsdata_file: ifsdata_12292524
climate_data_file: clim_data_12292524
horiz_resolution: 159
vert_resolution: 91
grid_type: l_2
upload_interval: 96
utstep: 900
nfrres: restart dump frequency (steps) 48
Copying IC ancils from: ../../projects/climateprediction.net/jf_f7ecfd27aa1e68bbe7e8d19800e25c73 to: /var/lib/boinc-client/slots/16/ic_ancil_12292524.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/16/ic_ancil_12292524.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_4b5a82f397796db128c974f644faa565 to: /var/lib/boinc-client/slots/16/ifsdata/ifsdata_12292524.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/16/ifsdata/ifsdata_12292524.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/16/159l_2/clim_data_12292524.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/16/159l_2/clim_data_12292524.zip
Checking for progress XML file: /var/lib/boinc-client/slots/16/progress_file_12292524.xml
Opened progress file ok : /var/lib/boinc-client/slots/16/progress_file_12292524.xml
-- Model is restarting --
Adjusting last_iter, 1630, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/16/progress_file_12292524.xml
last_cpu_time: 46673
upload_file_number: 16
last_iter: 1585
last_upload: 1383300
model_completed: 0
total_length_of_simulation: 1728000
result_base_name: oifs_43r3_bl_a38l_2016092300_20_1018_12292524_0_r1750201042
The child process has been launched with process id: 909972
Executing the command: /var/lib/boinc-client/slots/16/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:box.zenlime.org:1:1:909972:909972] [20240615:135236:1718459556:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2091] fp = 0x314f440
[EC_DRHOOK:box.zenlime.org:1:1:909972:909972] [20240615:135236:1718459556:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:box.zenlime.org:1:1:909972:909972] [20240615:135236:1718459556:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:box.zenlime.org:1:1:909972:909972] [20240615:135236:1718459556:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:box.zenlime.org:1:1:909972:909972] [20240615:135236:1718459556:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:box.zenlime.org:1:1:909972:909972] [20240615:135236:1718459556:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:box.zenlime.org:1:1:909972:909972] [20240615:135236:1718459556:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:box.zenlime.org:1:1:909972:909972] [20240615:135236:1718459556:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:box.zenlime.org:1:1:909972:909972] [20240615:135236:1718459556:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:box.zenlime.org:1:1:909972:909972] [20240615:135236:1718459556:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:box.zenlime.org:1:1:909972:909972] [20240615:135236:1718459556:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:box.zenlime.org:1:1:909972:909972] [20240615:135236:1718459556:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:box.zenlime.org:1:1:909972:909972] [20240615:135236:1718459556:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:box.zenlime.org:1:1:909972:909972] [20240615:135236:1718459556:0.000] [ignore_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:box.zenlime.org:1:1:909972:909972] [20240615:135236:1718459556:0.000] [restore_default_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:box.zenlime.org:1:1:909972:909972] [20240615:135236:1718459556:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:box.zenlime.org:1:1:909972:909972] [20240615:135236:1718459556:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:box.zenlime.org:1:1:909972:909972] [20240615:135236:1718459556:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:box.zenlime.org:1:1:909972:909972] [20240615:135236:1718459556:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x820cf0 (old at (nil))
[EC_DRHOOK:box.zenlime.org:1:1:909972:909972] [20240615:135236:1718459556:0.000] [trapfpe_treatment@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:box.zenlime.org:1:1:909972:909972] [20240615:135236:1718459556:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:box.zenlime.org:1:1:909972:909972] [20240615:135236:1718459556:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:box.zenlime.org:1:1:909972:909972] [20240615:135236:1718459556:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x820cf0 (old at (nil))
[EC_DRHOOK:box.zenlime.org:1:1:909972:909972] [20240615:135236:1718459556:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:box.zenlime.org:1:1:909972:909972] [20240615:135236:1718459556:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:box.zenlime.org:1:1:909972:909972] [20240615:135236:1718459556:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:box.zenlime.org:1:1:909972:909972] [20240615:135236:1718459556:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x820cf0 (old at (nil))
[EC_DRHOOK:box.zenlime.org:1:1:909972:909972] [20240615:135236:1718459556:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x820cf0 (old at (nil))
[EC_DRHOOK:box.zenlime.org:1:1:909972:909972] [20240615:135236:1718459556:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:box.zenlime.org:1:1:909972:909972] [20240615:135236:1718459556:0.000] [catch_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  13:53:10 STEP 1584 H= 396:00 +CPU= 30.308
  13:53:40 STEP 1585 H= 396:15 +CPU= 29.502
  13:54:09 STEP 1586 H= 396:30 +CPU= 28.652
  13:54:38 STEP 1587 H= 396:45 +CPU= 28.772
  13:55:09 STEP 1588 H= 397:00 +CPU= 29.489
  13:55:37 STEP 1589 H= 397:15 +CPU= 28.522
  13:56:06 STEP 1590 H= 397:30 +CPU= 28.898
  13:56:36 STEP 1591 H= 397:45 +CPU= 29.166
  13:57:06 STEP 1592 H= 398:00 +CPU= 29.215
  13:57:35 STEP 1593 H= 398:15 +CPU= 29.000
  13:58:04 STEP 1594 H= 398:30 +CPU= 28.813
  13:58:33 STEP 1595 H= 398:45 +CPU= 29.375
  13:59:09 STEP 1596 H= 399:00 +CPU= 35.465
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMGGh7zg+001596
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMSHh7zg+001596
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMUAh7zg+001596
  13:59:39 STEP 1597 H= 399:15 +CPU= 29.263
  14:00:09 STEP 1598 H= 399:30 +CPU= 26.575
  14:00:39 STEP 1599 H= 399:45 +CPU= 27.215
  14:01:08 STEP 1600 H= 400:00 +CPU= 28.907
  14:01:37 STEP 1601 H= 400:15 +CPU= 27.816
  14:02:06 STEP 1602 H= 400:30 +CPU= 28.713
  14:02:35 STEP 1603 H= 400:45 +CPU= 29.006
  14:03:05 STEP 1604 H= 401:00 +CPU= 28.810
  14:03:33 STEP 1605 H= 401:15 +CPU= 28.783
  14:04:03 STEP 1606 H= 401:30 +CPU= 28.311
  14:04:32 STEP 1607 H= 401:45 +CPU= 28.930
  14:05:08 STEP 1608 H= 402:00 +CPU= 35.660
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMGGh7zg+001608
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMSHh7zg+001608
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMUAh7zg+001608
  14:05:39 STEP 1609 H= 402:15 +CPU= 29.515
  14:06:08 STEP 1610 H= 402:30 +CPU= 29.522
  14:06:38 STEP 1611 H= 402:45 +CPU= 29.168
  14:07:08 STEP 1612 H= 403:00 +CPU= 29.589
  14:07:38 STEP 1613 H= 403:15 +CPU= 29.207
  14:08:08 STEP 1614 H= 403:30 +CPU= 29.202
  14:08:38 STEP 1615 H= 403:45 +CPU= 29.217
  14:09:08 STEP 1616 H= 404:00 +CPU= 30.067
  14:09:38 STEP 1617 H= 404:15 +CPU= 29.339
  14:10:07 STEP 1618 H= 404:30 +CPU= 29.220
  14:10:37 STEP 1619 H= 404:45 +CPU= 29.971
  14:11:13 STEP 1620 H= 405:00 +CPU= 35.284
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMGGh7zg+001620
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMSHh7zg+001620
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMUAh7zg+001620
  14:11:43 STEP 1621 H= 405:15 +CPU= 28.769
  14:12:12 STEP 1622 H= 405:30 +CPU= 29.458
  14:12:42 STEP 1623 H= 405:45 +CPU= 29.265
  14:13:12 STEP 1624 H= 406:00 +CPU= 30.213
  14:13:42 STEP 1625 H= 406:15 +CPU= 29.142
  14:14:11 STEP 1626 H= 406:30 +CPU= 29.637
  14:14:41 STEP 1627 H= 406:45 +CPU= 29.373
  14:15:10 STEP 1628 H= 407:00 +CPU= 29.540
  14:15:39 STEP 1629 H= 407:15 +CPU= 28.623
  14:16:08 STEP 1630 H= 407:30 +CPU= 29.290
  14:16:37 STEP 1631 H= 407:45 +CPU= 28.681
  14:17:17 STEP 1632 H= 408:00 +CPU= 39.595
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMGGh7zg+001632
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMSHh7zg+001632
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMUAh7zg+001632
  14:17:47 STEP 1633 H= 408:15 +CPU= 28.574
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMGGh7zg+001548
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMSHh7zg+001548
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMUAh7zg+001548
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMGGh7zg+001560
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMSHh7zg+001560
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMUAh7zg+001560
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMGGh7zg+001572
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMSHh7zg+001572
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMUAh7zg+001572
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMGGh7zg+001584
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMSHh7zg+001584
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMUAh7zg+001584
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMGGh7zg+001596
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMSHh7zg+001596
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMUAh7zg+001596
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMGGh7zg+001608
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMSHh7zg+001608
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMUAh7zg+001608
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMGGh7zg+001620
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMSHh7zg+001620
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMUAh7zg+001620
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMGGh7zg+001632
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMSHh7zg+001632
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMUAh7zg+001632
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a38l_2016092300_20_1018_12292524_0_r1750201042_16.zip
Uploading the intermediate file: upload_file_16.zip
  14:18:16 STEP 1634 H= 408:30 +CPU= 28.271
  14:18:45 STEP 1635 H= 408:45 +CPU= 28.598
  14:19:15 STEP 1636 H= 409:00 +CPU= 29.842
  14:19:45 STEP 1637 H= 409:15 +CPU= 29.276
  14:20:14 STEP 1638 H= 409:30 +CPU= 28.763
  14:20:44 STEP 1639 H= 409:45 +CPU= 29.357
  14:21:14 STEP 1640 H= 410:00 +CPU= 29.656
  14:21:43 STEP 1641 H= 410:15 +CPU= 28.473
  14:22:13 STEP 1642 H= 410:30 +CPU= 29.469
  14:22:43 STEP 1643 H= 410:45 +CPU= 28.930
  14:23:20 STEP 1644 H= 411:00 +CPU= 35.913
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMGGh7zg+001644
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMSHh7zg+001644
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMUAh7zg+001644
  14:23:50 STEP 1645 H= 411:15 +CPU= 28.823
  14:24:19 STEP 1646 H= 411:30 +CPU= 28.651
  14:24:49 STEP 1647 H= 411:45 +CPU= 28.776
  14:25:19 STEP 1648 H= 412:00 +CPU= 29.763
  14:25:49 STEP 1649 H= 412:15 +CPU= 29.136
  14:26:18 STEP 1650 H= 412:30 +CPU= 29.218
  14:26:48 STEP 1651 H= 412:45 +CPU= 29.406
  14:27:18 STEP 1652 H= 413:00 +CPU= 29.901
  14:27:48 STEP 1653 H= 413:15 +CPU= 29.308
  14:28:18 STEP 1654 H= 413:30 +CPU= 29.082
  14:28:48 STEP 1655 H= 413:45 +CPU= 28.977
  14:29:25 STEP 1656 H= 414:00 +CPU= 36.399
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMGGh7zg+001656
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMSHh7zg+001656
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMUAh7zg+001656
  14:29:54 STEP 1657 H= 414:15 +CPU= 29.260
  14:30:25 STEP 1658 H= 414:30 +CPU= 29.074
  14:30:53 STEP 1659 H= 414:45 +CPU= 27.916
  14:31:23 STEP 1660 H= 415:00 +CPU= 29.890
  14:31:52 STEP 1661 H= 415:15 +CPU= 28.541
  14:32:20 STEP 1662 H= 415:30 +CPU= 28.739
  14:32:51 STEP 1663 H= 415:45 +CPU= 29.551
  14:33:21 STEP 1664 H= 416:00 +CPU= 30.286
  14:33:52 STEP 1665 H= 416:15 +CPU= 29.783
  14:34:22 STEP 1666 H= 416:30 +CPU= 29.395
  14:34:51 STEP 1667 H= 416:45 +CPU= 28.859
  14:35:27 STEP 1668 H= 417:00 +CPU= 35.266
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMGGh7zg+001668
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMSHh7zg+001668
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMUAh7zg+001668
  14:35:57 STEP 1669 H= 417:15 +CPU= 29.132
  14:36:25 STEP 1670 H= 417:30 +CPU= 28.087
  14:36:54 STEP 1671 H= 417:45 +CPU= 28.612
  14:37:24 STEP 1672 H= 418:00 +CPU= 29.818
  14:37:54 STEP 1673 H= 418:15 +CPU= 29.093
  14:38:23 STEP 1674 H= 418:30 +CPU= 28.790
  14:38:53 STEP 1675 H= 418:45 +CPU= 29.234
  14:39:22 STEP 1676 H= 419:00 +CPU= 29.440
  14:39:52 STEP 1677 H= 419:15 +CPU= 29.126
  14:40:21 STEP 1678 H= 419:30 +CPU= 28.171
  14:40:51 STEP 1679 H= 419:45 +CPU= 29.060
  14:41:30 STEP 1680 H= 420:00 +CPU= 39.092
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMGGh7zg+001680
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMSHh7zg+001680
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMUAh7zg+001680
  14:42:00 STEP 1681 H= 420:15 +CPU= 29.397
  14:42:29 STEP 1682 H= 420:30 +CPU= 28.765
  14:42:59 STEP 1683 H= 420:45 +CPU= 28.986
  14:43:29 STEP 1684 H= 421:00 +CPU= 29.888
  14:43:58 STEP 1685 H= 421:15 +CPU= 28.923
  14:44:28 STEP 1686 H= 421:30 +CPU= 28.951
  14:44:56 STEP 1687 H= 421:45 +CPU= 28.314
  14:45:26 STEP 1688 H= 422:00 +CPU= 29.753
  14:45:57 STEP 1689 H= 422:15 +CPU= 28.688
  14:46:26 STEP 1690 H= 422:30 +CPU= 28.988
  14:46:55 STEP 1691 H= 422:45 +CPU= 29.117
  14:47:31 STEP 1692 H= 423:00 +CPU= 33.123
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMGGh7zg+001692
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMSHh7zg+001692
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMUAh7zg+001692
  14:48:01 STEP 1693 H= 423:15 +CPU= 27.736
  14:48:31 STEP 1694 H= 423:30 +CPU= 27.784
  14:49:00 STEP 1695 H= 423:45 +CPU= 28.570
  14:49:29 STEP 1696 H= 424:00 +CPU= 28.815
  14:49:59 STEP 1697 H= 424:15 +CPU= 29.313
  14:50:28 STEP 1698 H= 424:30 +CPU= 28.730
  14:50:57 STEP 1699 H= 424:45 +CPU= 28.621
  14:51:27 STEP 1700 H= 425:00 +CPU= 29.479
  14:51:56 STEP 1701 H= 425:15 +CPU= 28.881
  14:52:25 STEP 1702 H= 425:30 +CPU= 28.593
  14:52:54 STEP 1703 H= 425:45 +CPU= 28.671
  14:53:30 STEP 1704 H= 426:00 +CPU= 34.888
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMGGh7zg+001704
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMSHh7zg+001704
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMUAh7zg+001704
  14:54:00 STEP 1705 H= 426:15 +CPU= 28.813
  14:54:29 STEP 1706 H= 426:30 +CPU= 28.847
  14:54:58 STEP 1707 H= 426:45 +CPU= 28.767
  14:55:28 STEP 1708 H= 427:00 +CPU= 29.466
  14:55:58 STEP 1709 H= 427:15 +CPU= 29.536
  14:56:27 STEP 1710 H= 427:30 +CPU= 28.896
  14:56:56 STEP 1711 H= 427:45 +CPU= 28.401
  14:57:25 STEP 1712 H= 428:00 +CPU= 29.654
  14:57:55 STEP 1713 H= 428:15 +CPU= 29.275
  14:58:24 STEP 1714 H= 428:30 +CPU= 28.733
  14:58:53 STEP 1715 H= 428:45 +CPU= 29.130
  14:59:30 STEP 1716 H= 429:00 +CPU= 35.188
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMGGh7zg+001716
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMSHh7zg+001716
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMUAh7zg+001716
  15:00:00 STEP 1717 H= 429:15 +CPU= 29.316
  15:00:29 STEP 1718 H= 429:30 +CPU= 29.067
  15:00:58 STEP 1719 H= 429:45 +CPU= 28.848
  15:01:28 STEP 1720 H= 430:00 +CPU= 29.257
  15:01:58 STEP 1721 H= 430:15 +CPU= 28.548
  15:02:27 STEP 1722 H= 430:30 +CPU= 28.897
  15:02:57 STEP 1723 H= 430:45 +CPU= 28.912
  15:03:26 STEP 1724 H= 431:00 +CPU= 29.627
  15:03:56 STEP 1725 H= 431:15 +CPU= 28.925
  15:04:25 STEP 1726 H= 431:30 +CPU= 28.655
  15:04:54 STEP 1727 H= 431:45 +CPU= 28.946
  15:05:34 STEP 1728 H= 432:00 +CPU= 39.508
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMGGh7zg+001728
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMSHh7zg+001728
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMUAh7zg+001728
  15:06:04 STEP 1729 H= 432:15 +CPU= 29.005
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMGGh7zg+001644
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMSHh7zg+001644
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMUAh7zg+001644
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMGGh7zg+001656
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMSHh7zg+001656
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMUAh7zg+001656
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMGGh7zg+001668
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMSHh7zg+001668
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMUAh7zg+001668
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMGGh7zg+001680
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMSHh7zg+001680
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMUAh7zg+001680
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMGGh7zg+001692
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMSHh7zg+001692
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMUAh7zg+001692
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMGGh7zg+001704
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMSHh7zg+001704
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMUAh7zg+001704
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMGGh7zg+001716
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMSHh7zg+001716
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMUAh7zg+001716
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMGGh7zg+001728
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMSHh7zg+001728
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMUAh7zg+001728
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a38l_2016092300_20_1018_12292524_0_r1750201042_17.zip
Uploading the intermediate file: upload_file_17.zip
  15:06:33 STEP 1730 H= 432:30 +CPU= 28.086
  15:07:04 STEP 1731 H= 432:45 +CPU= 29.512
  15:07:35 STEP 1732 H= 433:00 +CPU= 29.963
  15:08:04 STEP 1733 H= 433:15 +CPU= 29.400
  15:08:34 STEP 1734 H= 433:30 +CPU= 28.943
  15:09:04 STEP 1735 H= 433:45 +CPU= 29.176
  15:09:35 STEP 1736 H= 434:00 +CPU= 30.211
  15:10:05 STEP 1737 H= 434:15 +CPU= 29.322
  15:10:34 STEP 1738 H= 434:30 +CPU= 29.427
  15:11:05 STEP 1739 H= 434:45 +CPU= 29.900
  15:11:41 STEP 1740 H= 435:00 +CPU= 35.660
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMGGh7zg+001740
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMSHh7zg+001740
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMUAh7zg+001740
  15:12:10 STEP 1741 H= 435:15 +CPU= 28.311
  15:12:40 STEP 1742 H= 435:30 +CPU= 29.145
  15:13:11 STEP 1743 H= 435:45 +CPU= 28.939
  15:13:41 STEP 1744 H= 436:00 +CPU= 29.684
  15:14:12 STEP 1745 H= 436:15 +CPU= 29.310
  15:14:41 STEP 1746 H= 436:30 +CPU= 29.105
  15:15:11 STEP 1747 H= 436:45 +CPU= 29.571
  15:15:41 STEP 1748 H= 437:00 +CPU= 29.701
  15:16:10 STEP 1749 H= 437:15 +CPU= 28.882
  15:16:39 STEP 1750 H= 437:30 +CPU= 28.752
  15:17:09 STEP 1751 H= 437:45 +CPU= 29.560
  15:17:45 STEP 1752 H= 438:00 +CPU= 36.154
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMGGh7zg+001752
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMSHh7zg+001752
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMUAh7zg+001752
  15:18:15 STEP 1753 H= 438:15 +CPU= 29.034
  15:18:44 STEP 1754 H= 438:30 +CPU= 28.318
  15:19:13 STEP 1755 H= 438:45 +CPU= 28.790
  15:19:42 STEP 1756 H= 439:00 +CPU= 29.396
  15:20:11 STEP 1757 H= 439:15 +CPU= 29.114
  15:20:42 STEP 1758 H= 439:30 +CPU= 28.861
  15:21:11 STEP 1759 H= 439:45 +CPU= 29.420
  15:21:40 STEP 1760 H= 440:00 +CPU= 29.319
  15:22:10 STEP 1761 H= 440:15 +CPU= 28.756
  15:22:39 STEP 1762 H= 440:30 +CPU= 28.972
  15:23:08 STEP 1763 H= 440:45 +CPU= 29.047
  15:23:45 STEP 1764 H= 441:00 +CPU= 35.498
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMGGh7zg+001764
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMSHh7zg+001764
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMUAh7zg+001764
  15:24:13 STEP 1765 H= 441:15 +CPU= 28.562
  15:24:43 STEP 1766 H= 441:30 +CPU= 29.300
  15:25:12 STEP 1767 H= 441:45 +CPU= 29.496
  15:25:43 STEP 1768 H= 442:00 +CPU= 29.965
  15:26:12 STEP 1769 H= 442:15 +CPU= 28.789
  15:26:41 STEP 1770 H= 442:30 +CPU= 28.619
  15:27:10 STEP 1771 H= 442:45 +CPU= 28.549
  15:27:40 STEP 1772 H= 443:00 +CPU= 29.878
  15:28:10 STEP 1773 H= 443:15 +CPU= 29.045
  15:28:39 STEP 1774 H= 443:30 +CPU= 29.275
  15:29:08 STEP 1775 H= 443:45 +CPU= 29.220
  15:29:47 STEP 1776 H= 444:00 +CPU= 38.391
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMGGh7zg+001776
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMSHh7zg+001776
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMUAh7zg+001776
  15:30:16 STEP 1777 H= 444:15 +CPU= 29.007
  15:30:46 STEP 1778 H= 444:30 +CPU= 29.377
  15:31:16 STEP 1779 H= 444:45 +CPU= 28.973
  15:31:45 STEP 1780 H= 445:00 +CPU= 29.586
  15:32:15 STEP 1781 H= 445:15 +CPU= 29.356
  15:32:44 STEP 1782 H= 445:30 +CPU= 28.894
  15:33:13 STEP 1783 H= 445:45 +CPU= 29.044
  15:33:42 STEP 1784 H= 446:00 +CPU= 29.328
  15:34:12 STEP 1785 H= 446:15 +CPU= 29.000
  15:34:41 STEP 1786 H= 446:30 +CPU= 28.251
  15:35:10 STEP 1787 H= 446:45 +CPU= 27.790
  15:35:47 STEP 1788 H= 447:00 +CPU= 35.582
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMGGh7zg+001788
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMSHh7zg+001788
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMUAh7zg+001788
  15:36:16 STEP 1789 H= 447:15 +CPU= 27.404
  15:36:46 STEP 1790 H= 447:30 +CPU= 28.535
  15:37:15 STEP 1791 H= 447:45 +CPU= 28.444
  15:37:44 STEP 1792 H= 448:00 +CPU= 29.037
  15:38:13 STEP 1793 H= 448:15 +CPU= 28.491
  15:38:42 STEP 1794 H= 448:30 +CPU= 29.012
  15:39:11 STEP 1795 H= 448:45 +CPU= 28.749
  15:39:41 STEP 1796 H= 449:00 +CPU= 29.880
  15:40:10 STEP 1797 H= 449:15 +CPU= 29.068
  15:40:39 STEP 1798 H= 449:30 +CPU= 28.698
  15:41:08 STEP 1799 H= 449:45 +CPU= 29.255
  15:41:44 STEP 1800 H= 450:00 +CPU= 35.721
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMGGh7zg+001800
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMSHh7zg+001800
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMUAh7zg+001800
  15:42:13 STEP 1801 H= 450:15 +CPU= 28.745
  15:42:42 STEP 1802 H= 450:30 +CPU= 29.230
  15:43:12 STEP 1803 H= 450:45 +CPU= 29.064
  15:43:42 STEP 1804 H= 451:00 +CPU= 29.338
  15:44:11 STEP 1805 H= 451:15 +CPU= 28.925
  15:44:41 STEP 1806 H= 451:30 +CPU= 28.967
  15:45:10 STEP 1807 H= 451:45 +CPU= 29.017
  15:45:39 STEP 1808 H= 452:00 +CPU= 29.204
  15:46:08 STEP 1809 H= 452:15 +CPU= 28.708
  15:46:38 STEP 1810 H= 452:30 +CPU= 29.233
  15:47:07 STEP 1811 H= 452:45 +CPU= 28.331
  15:47:43 STEP 1812 H= 453:00 +CPU= 35.381
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMGGh7zg+001812
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMSHh7zg+001812
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMUAh7zg+001812
  15:48:12 STEP 1813 H= 453:15 +CPU= 28.753
  15:48:41 STEP 1814 H= 453:30 +CPU= 29.149
  15:49:10 STEP 1815 H= 453:45 +CPU= 28.747
  15:49:40 STEP 1816 H= 454:00 +CPU= 29.784
  15:50:10 STEP 1817 H= 454:15 +CPU= 29.204
  15:50:40 STEP 1818 H= 454:30 +CPU= 28.735
  15:51:08 STEP 1819 H= 454:45 +CPU= 28.841
  15:51:38 STEP 1820 H= 455:00 +CPU= 29.414
  15:52:06 STEP 1821 H= 455:15 +CPU= 28.481
  15:52:37 STEP 1822 H= 455:30 +CPU= 29.294
  15:53:06 STEP 1823 H= 455:45 +CPU= 29.326
  15:53:46 STEP 1824 H= 456:00 +CPU= 38.907
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMGGh7zg+001824
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMSHh7zg+001824
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMUAh7zg+001824
  15:54:15 STEP 1825 H= 456:15 +CPU= 29.323
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMGGh7zg+001740
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMSHh7zg+001740
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMUAh7zg+001740
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMGGh7zg+001752
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMSHh7zg+001752
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMUAh7zg+001752
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMGGh7zg+001764
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMSHh7zg+001764
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMUAh7zg+001764
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMGGh7zg+001776
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMSHh7zg+001776
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMUAh7zg+001776
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMGGh7zg+001788
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMSHh7zg+001788
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMUAh7zg+001788
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMGGh7zg+001800
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMSHh7zg+001800
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMUAh7zg+001800
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMGGh7zg+001812
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMSHh7zg+001812
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMUAh7zg+001812
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMGGh7zg+001824
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMSHh7zg+001824
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMUAh7zg+001824
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a38l_2016092300_20_1018_12292524_0_r1750201042_18.zip
Uploading the intermediate file: upload_file_18.zip
  15:54:45 STEP 1826 H= 456:30 +CPU= 28.539
  15:55:14 STEP 1827 H= 456:45 +CPU= 29.150
  15:55:45 STEP 1828 H= 457:00 +CPU= 29.533
  15:56:15 STEP 1829 H= 457:15 +CPU= 29.764
  15:56:45 STEP 1830 H= 457:30 +CPU= 29.482
  15:57:15 STEP 1831 H= 457:45 +CPU= 29.670
  15:57:45 STEP 1832 H= 458:00 +CPU= 29.910
  15:58:15 STEP 1833 H= 458:15 +CPU= 29.095
  15:58:44 STEP 1834 H= 458:30 +CPU= 28.958
  15:59:14 STEP 1835 H= 458:45 +CPU= 29.421
  15:59:51 STEP 1836 H= 459:00 +CPU= 35.384
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMGGh7zg+001836
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMSHh7zg+001836
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMUAh7zg+001836
  16:00:21 STEP 1837 H= 459:15 +CPU= 28.781
  16:00:52 STEP 1838 H= 459:30 +CPU= 29.317
  16:01:21 STEP 1839 H= 459:45 +CPU= 28.859
  16:01:52 STEP 1840 H= 460:00 +CPU= 30.003
  16:02:22 STEP 1841 H= 460:15 +CPU= 29.326
  16:02:52 STEP 1842 H= 460:30 +CPU= 29.311
  16:03:22 STEP 1843 H= 460:45 +CPU= 29.402
  16:03:53 STEP 1844 H= 461:00 +CPU= 29.721
  16:04:23 STEP 1845 H= 461:15 +CPU= 29.260
  16:04:53 STEP 1846 H= 461:30 +CPU= 29.099
  16:05:23 STEP 1847 H= 461:45 +CPU= 29.570
  16:06:00 STEP 1848 H= 462:00 +CPU= 35.982
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMGGh7zg+001848
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMSHh7zg+001848
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMUAh7zg+001848
  16:06:30 STEP 1849 H= 462:15 +CPU= 28.458
  16:06:59 STEP 1850 H= 462:30 +CPU= 29.156
  16:07:29 STEP 1851 H= 462:45 +CPU= 29.175
  16:07:59 STEP 1852 H= 463:00 +CPU= 30.003
  16:08:29 STEP 1853 H= 463:15 +CPU= 29.334
  16:08:59 STEP 1854 H= 463:30 +CPU= 29.737
  16:09:28 STEP 1855 H= 463:45 +CPU= 29.274
  16:09:58 STEP 1856 H= 464:00 +CPU= 29.879
  16:10:29 STEP 1857 H= 464:15 +CPU= 29.135
  16:10:57 STEP 1858 H= 464:30 +CPU= 28.803
  16:11:26 STEP 1859 H= 464:45 +CPU= 28.664
  16:12:02 STEP 1860 H= 465:00 +CPU= 35.827
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMGGh7zg+001860
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMSHh7zg+001860
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMUAh7zg+001860
  16:12:31 STEP 1861 H= 465:15 +CPU= 28.753
  16:13:00 STEP 1862 H= 465:30 +CPU= 29.251
  16:13:30 STEP 1863 H= 465:45 +CPU= 29.105
  16:14:00 STEP 1864 H= 466:00 +CPU= 29.783
  16:14:29 STEP 1865 H= 466:15 +CPU= 28.785
  16:14:59 STEP 1866 H= 466:30 +CPU= 29.283
  16:15:29 STEP 1867 H= 466:45 +CPU= 29.195
  16:15:59 STEP 1868 H= 467:00 +CPU= 29.903
  16:16:28 STEP 1869 H= 467:15 +CPU= 28.723
  16:16:57 STEP 1870 H= 467:30 +CPU= 29.029
  16:17:26 STEP 1871 H= 467:45 +CPU= 29.030
  16:18:05 STEP 1872 H= 468:00 +CPU= 38.336
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMGGh7zg+001872
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMSHh7zg+001872
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMUAh7zg+001872
  16:18:34 STEP 1873 H= 468:15 +CPU= 29.089
  16:19:04 STEP 1874 H= 468:30 +CPU= 29.208
  16:19:35 STEP 1875 H= 468:45 +CPU= 29.531
  16:20:05 STEP 1876 H= 469:00 +CPU= 29.391
  16:20:35 STEP 1877 H= 469:15 +CPU= 29.037
  16:21:04 STEP 1878 H= 469:30 +CPU= 29.169
  16:21:34 STEP 1879 H= 469:45 +CPU= 29.122
  16:22:04 STEP 1880 H= 470:00 +CPU= 29.280
  16:22:34 STEP 1881 H= 470:15 +CPU= 28.512
  16:23:03 STEP 1882 H= 470:30 +CPU= 29.281
  16:23:33 STEP 1883 H= 470:45 +CPU= 28.847
  16:24:10 STEP 1884 H= 471:00 +CPU= 34.942
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMGGh7zg+001884
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMSHh7zg+001884
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMUAh7zg+001884
  16:24:40 STEP 1885 H= 471:15 +CPU= 29.608
  16:25:10 STEP 1886 H= 471:30 +CPU= 28.980
  16:25:41 STEP 1887 H= 471:45 +CPU= 29.247
  16:26:11 STEP 1888 H= 472:00 +CPU= 30.016
  16:26:42 STEP 1889 H= 472:15 +CPU= 29.055
  16:27:12 STEP 1890 H= 472:30 +CPU= 29.423
  16:27:42 STEP 1891 H= 472:45 +CPU= 29.542
  16:28:13 STEP 1892 H= 473:00 +CPU= 30.580
  16:28:42 STEP 1893 H= 473:15 +CPU= 29.667
  16:29:12 STEP 1894 H= 473:30 +CPU= 29.538
  16:29:42 STEP 1895 H= 473:45 +CPU= 28.976
  16:30:19 STEP 1896 H= 474:00 +CPU= 36.335
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMGGh7zg+001896
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMSHh7zg+001896
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMUAh7zg+001896
  16:30:49 STEP 1897 H= 474:15 +CPU= 29.337
  16:31:19 STEP 1898 H= 474:30 +CPU= 29.514
  16:31:49 STEP 1899 H= 474:45 +CPU= 29.277
  16:32:20 STEP 1900 H= 475:00 +CPU= 30.599
  16:32:50 STEP 1901 H= 475:15 +CPU= 29.948
  16:33:20 STEP 1902 H= 475:30 +CPU= 29.680
  16:33:50 STEP 1903 H= 475:45 +CPU= 29.574
  16:34:20 STEP 1904 H= 476:00 +CPU= 29.573
  16:34:51 STEP 1905 H= 476:15 +CPU= 28.874
  16:35:21 STEP 1906 H= 476:30 +CPU= 29.596
  16:35:50 STEP 1907 H= 476:45 +CPU= 28.997
  16:36:26 STEP 1908 H= 477:00 +CPU= 36.123
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMGGh7zg+001908
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMSHh7zg+001908
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMUAh7zg+001908
  16:36:56 STEP 1909 H= 477:15 +CPU= 29.588
  16:37:25 STEP 1910 H= 477:30 +CPU= 29.075
  16:37:55 STEP 1911 H= 477:45 +CPU= 29.417
  16:38:26 STEP 1912 H= 478:00 +CPU= 30.320
  16:38:56 STEP 1913 H= 478:15 +CPU= 29.718
  16:39:25 STEP 1914 H= 478:30 +CPU= 29.096
  16:39:55 STEP 1915 H= 478:45 +CPU= 29.185
  16:40:25 STEP 1916 H= 479:00 +CPU= 30.026
  16:40:54 STEP 1917 H= 479:15 +CPU= 29.195
  16:41:24 STEP 1918 H= 479:30 +CPU= 29.378
  16:41:53 STEP 1919 H= 479:45 +CPU= 29.049
  16:42:30 STEP 1920 H= 480:00 +CPU= 36.930
..The child process terminated with status: 0
>>> Printing last 70 lines from file: NODE.001_01

 
DDH------------------------------------------------------------------------
I                                                                         I
I TIME =  480.000H DDH * Diagnostic Var.,     AREA                GLOBAL  I
I                                                                         I
I-------------------------------------------------------------------------I
I   VARIABLE   I      Initial value         I        Current value        I
I-------------------------------------------------------------------------I
I  OMEGA (P/S) I     -0.1378862392E-20      I       0.8994429589E-05      I
ICloud fractionI      0.0000000000E+00      I       0.4558038303E-01      I
I  Relat. Hum. I      0.3627105020E+00      I       0.4098254424E+00      I
I  PASS 01     I      0.0000000000E+00      I       0.8267778578E+03      I
DDH------------------------------------------------------------------------
  
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
 NSTEP =  1920 SCAN2M_HPOS  P
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ICMGGh7zg+001920 MODE=a
 IO-STREAM CLOSED - ICMGGh7zg+001920
 MPI-TASK:   1 -    300334132 BYTES IN      7 RECORDS TRANSFERRED IN    0.0004 SECONDS750835.3 Mbytes/s, TOTAL TIME=    0.0100( 4.0%)
 16:42:30 STEP 1920 H= 480:00 +CPU= 36.930
 END CNT3
 NSTEP =  1920 CNT0   000000000  27.846  27.846  28.027   0.994               0               0               0               0               0
IO-STREAM STATISTICS, TOTAL NO OF REC/BYTES READ           46/  1069125708 WRITTEN        10010/************
 -TASK-OPENED-OPEN-RECS IN -KBYTE IN    -RECS OUT-KBYTE OUT-WALL   -WALL IN-WALL
  OU-TOT IN -TOT OUT
   1    178    0       46      1044068    10010 ************    17.7     1.6    16.1     2.3    34.1
===-=== START OF TIMING STATISTICS ===-===
 
STATS FOR ALL TASKS
 NUM ROUTINE                                     CALLS  MEAN(ms)   MAX(ms)   FRAC(%)  UNBAL(%)
   0 CNT0     - COMPLETE EXECUTION                   1 ********* *********    100.00      0.00
   1 CNT4     - FORWARD INTEGRATION                  1 ********* *********     99.96      0.00
   8 SCAN2M - GRID-POINT DYNAMICS                  365    5642.7    5642.7     20.20      0.00
   9 SPCM     - SPECTRAL COMP.                     336     962.6     962.6      3.17      0.00
  10 SCAN2M - PHYSICS                              337   13105.5   13105.5     43.32      0.00
  11 IOPACK   - OUTPUT P.P. RESULTS                 28     832.9     832.9      0.23      0.00
  12 SPNORM   - SPECTRAL NORM COMP.                 15      78.8      78.8      0.01      0.00
  17 GRIDFPOS IN CNT4                               28      30.8      30.8      0.01      0.00
  27 RESTART FILES - WRITING                         7    3330.5    3330.5      0.23      0.00
  28 RESTART FILES - READING                         1    3214.3    3214.3      0.03      0.00
  29 SU4FPOS IN CNT4                                28       0.1       0.1      0.00      0.00
  30 DYNFPOS IN CNT4                                28    5779.6    5779.6      1.59      0.00
  31 POSDDH IN STEPO                                 2       1.4       1.4      0.00      0.00
  37 CPGLAG   - SL COMPUTATIONS                    337    6386.9    6386.9     21.11      0.00
  39 SU0YOMB                                         1     336.1     336.1      0.00      0.00
  51 SCAN2M   - SL COMM. PART 1                    337      32.2      32.2      0.11      0.00
  54 SPCM     - M TO S/S TO M TRANSP.              336     583.8     583.8      1.92      0.00
  55 SPCIMPF  - S TO M/M TO S TRANSP.              336      72.8      72.8      0.24      0.00
  56 SPNORM   - SPECTRAL NORM COMM.                 15       0.3       0.3      0.00      0.00
 102 LTINV_CTL   - INVERSE LEGENDRE TRANSFORM      732     414.8     414.8      2.98      0.00
 103 LTDIR_CTL   - DIRECT LEGENDRE TRANSFORM       700     240.2     240.2      1.65      0.00
 106 FTDIR_CTL   - DIRECT FOURIER TRANSFORM        700      80.0      80.0      0.55      0.00
 107 FTINV_CTL   - INVERSE FOURIER TRANSFORM       732     116.2     116.2      0.83      0.00
 140 SULEG       - COMP. OF LEGENDRE POL.            1      33.5      33.5      0.00      0.00
 152 LTINV_CTL   - M TO L TRANSPOSITION            732      65.1      65.1      0.47      0.00
 153 LTDIR_CTL   - L TO M TRANSPOSITION            700      36.8      36.8      0.25      0.00
 157 FTINV_CTL   - L TO G TRANSPOSITION            732     139.9     139.9      1.00      0.00
 158 FTDIR_CTL   - G TO L TRANSPOSITION            700      62.3      62.3      0.43      0.00
 400 GSTATS                                      39960       0.0       0.0      0.00      0.00
 401 GSTATS HOOK                                 37126       0.0       0.0      0.00      0.00
TOTAL MEASURED IMBALANCE =       0.0 SECONDS,  0.0 PERCENT
TOTAL WALLCLOCK TIME    10194.7 CPU TIME   10021.2 VECTOR TIME    10021.2


===-=== END   OF TIMING STATISTICS ===-===
FORECAST DAYS PER DAY      169.6
  *** END CNT0 *** 
------------------------------------------------
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMGGh7zg+001920
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMSHh7zg+001920
Moving to projects directory: /var/lib/boinc-client/slots/16/ICMUAh7zg+001920
Adding to the zip: /var/lib/boinc-client/slots/16/NODE.001_01
Adding to the zip: /var/lib/boinc-client/slots/16/ifs.stat
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMGGh7zg+000864
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMGGh7zg+001884
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMSHh7zg+001920
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMSHh7zg+001884
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMUAh7zg+001884
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMUAh7zg+001872
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMGGh7zg+001896
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMUAh7zg+001896
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMUAh7zg+001908
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMUAh7zg+001860
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMUAh7zg+000864
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMGGh7zg+001908
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMSHh7zg+000864
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMSHh7zg+001908
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMUAh7zg+001836
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMSHh7zg+001848
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMGGh7zg+001848
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMGGh7zg+001920
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMGGh7zg+001872
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMSHh7zg+001860
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMSHh7zg+001872
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMSHh7zg+001896
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMUAh7zg+001920
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMGGh7zg+001836
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMGGh7zg+001860
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMSHh7zg+001836
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12292524/ICMUAh7zg+001848
Zipping up the final file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a38l_2016092300_20_1018_12292524_0_r1750201042_19.zip
Uploading the final file: upload_file_19.zip
Uploading trickle at timestep: 1727100
16:45:59 (909969): called boinc_finish(0)

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
15 Jun 2024 16:44:18 1537687 22446130 oifs_43r3_bl_a38l_2016092300_20_1018_12292524_0 1,727,100 56,694 0.0328


©2024 climateprediction.net