climateprediction.net home page
Task 22449669

Task 22449669

Name oifs_43r3_bl_a3ht_2016092300_20_1018_12292856_1
Workunit 12292856
Created 17 Jun 2024, 15:57:14 UTC
Sent 17 Jun 2024, 15:59:04 UTC
Report deadline 16 Aug 2024, 15:59:04 UTC
Received 18 Jun 2024, 23:14:12 UTC
Server state Over
Outcome Success
Client state Done
Exit status 0 (0x00000000)
Computer ID 1496046
Run time 14 hours 10 min 46 sec
CPU time 14 hours 10 min 46 sec
Validate state Valid
Credit 2,635.55
Device peak FLOPS 5.56 GFLOPS
Application version OpenIFS 43r3 Baroclinic Lifecycle v1.13
x86_64-pc-linux-gnu
Peak working set size 5,022.86 MB
Peak swap size 5,981.07 MB
Peak disk usage 1,676.35 MB
Stderr
<core_client_version>7.20.2</core_client_version>
<![CDATA[
<stderr_txt>
5.528
  13:50:37 STEP 1432 H= 358:00 +CPU= 26.098
  13:51:02 STEP 1433 H= 358:15 +CPU= 25.239
  13:51:28 STEP 1434 H= 358:30 +CPU= 25.465
  13:51:53 STEP 1435 H= 358:45 +CPU= 25.565
  13:52:20 STEP 1436 H= 359:00 +CPU= 26.646
  13:52:47 STEP 1437 H= 359:15 +CPU= 26.264
  13:53:12 STEP 1438 H= 359:30 +CPU= 25.316
  13:53:37 STEP 1439 H= 359:45 +CPU= 24.455
  13:54:12 STEP 1440 H= 360:00 +CPU= 34.862
Moving to projects directory: /boinc_data/boinc/slots/14/ICMGGh7zg+001440
Moving to projects directory: /boinc_data/boinc/slots/14/ICMSHh7zg+001440
Moving to projects directory: /boinc_data/boinc/slots/14/ICMUAh7zg+001440
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMGGh7zg+001344
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMSHh7zg+001344
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMUAh7zg+001344
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMGGh7zg+001356
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMSHh7zg+001356
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMUAh7zg+001356
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMGGh7zg+001368
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMSHh7zg+001368
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMUAh7zg+001368
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMGGh7zg+001380
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMSHh7zg+001380
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMUAh7zg+001380
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMGGh7zg+001392
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMSHh7zg+001392
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMUAh7zg+001392
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMGGh7zg+001404
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMSHh7zg+001404
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMUAh7zg+001404
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMGGh7zg+001416
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMSHh7zg+001416
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMUAh7zg+001416
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMGGh7zg+001428
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMSHh7zg+001428
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMUAh7zg+001428
Zipping up the intermediate file: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_a3ht_2016092300_20_1018_12292856_1_r1411055572_14.zip
Uploading the intermediate file: upload_file_14.zip
  13:54:36 STEP 1441 H= 360:15 +CPU= 24.707
  13:55:03 STEP 1442 H= 360:30 +CPU= 26.172
  13:55:29 STEP 1443 H= 360:45 +CPU= 25.828
  13:55:55 STEP 1444 H= 361:00 +CPU= 26.367
  13:56:20 STEP 1445 H= 361:15 +CPU= 24.624
  13:56:46 STEP 1446 H= 361:30 +CPU= 25.778
  13:57:12 STEP 1447 H= 361:45 +CPU= 25.873
  13:57:38 STEP 1448 H= 362:00 +CPU= 25.847
  13:58:03 STEP 1449 H= 362:15 +CPU= 25.500
  13:58:29 STEP 1450 H= 362:30 +CPU= 25.350
  13:58:54 STEP 1451 H= 362:45 +CPU= 25.170
  13:59:25 STEP 1452 H= 363:00 +CPU= 31.309
Moving to projects directory: /boinc_data/boinc/slots/14/ICMGGh7zg+001452
Moving to projects directory: /boinc_data/boinc/slots/14/ICMSHh7zg+001452
Moving to projects directory: /boinc_data/boinc/slots/14/ICMUAh7zg+001452
  13:59:51 STEP 1453 H= 363:15 +CPU= 24.983
  14:00:17 STEP 1454 H= 363:30 +CPU= 26.000
  14:00:43 STEP 1455 H= 363:45 +CPU= 25.937
  14:01:10 STEP 1456 H= 364:00 +CPU= 26.671
  14:01:35 STEP 1457 H= 364:15 +CPU= 25.095
  14:02:01 STEP 1458 H= 364:30 +CPU= 25.678
  14:02:26 STEP 1459 H= 364:45 +CPU= 24.923
  14:02:51 STEP 1460 H= 365:00 +CPU= 24.938
  14:03:16 STEP 1461 H= 365:15 +CPU= 24.916
  14:03:40 STEP 1462 H= 365:30 +CPU= 24.518
  14:04:06 STEP 1463 H= 365:45 +CPU= 25.568
  14:04:38 STEP 1464 H= 366:00 +CPU= 31.878
Moving to projects directory: /boinc_data/boinc/slots/14/ICMGGh7zg+001464
Moving to projects directory: /boinc_data/boinc/slots/14/ICMSHh7zg+001464
Moving to projects directory: /boinc_data/boinc/slots/14/ICMUAh7zg+001464
  14:05:04 STEP 1465 H= 366:15 +CPU= 26.172
  14:05:30 STEP 1466 H= 366:30 +CPU= 25.907
  14:05:57 STEP 1467 H= 366:45 +CPU= 26.307
  14:06:22 STEP 1468 H= 367:00 +CPU= 25.275
  14:06:46 STEP 1469 H= 367:15 +CPU= 24.135
  14:07:11 STEP 1470 H= 367:30 +CPU= 24.596
  14:07:36 STEP 1471 H= 367:45 +CPU= 25.197
  14:08:03 STEP 1472 H= 368:00 +CPU= 25.942
  14:08:27 STEP 1473 H= 368:15 +CPU= 24.528
  14:08:52 STEP 1474 H= 368:30 +CPU= 24.403
  14:09:18 STEP 1475 H= 368:45 +CPU= 25.818
  14:09:49 STEP 1476 H= 369:00 +CPU= 31.499
Moving to projects directory: /boinc_data/boinc/slots/14/ICMGGh7zg+001476
Moving to projects directory: /boinc_data/boinc/slots/14/ICMSHh7zg+001476
Moving to projects directory: /boinc_data/boinc/slots/14/ICMUAh7zg+001476
  14:10:15 STEP 1477 H= 369:15 +CPU= 25.475
  14:10:40 STEP 1478 H= 369:30 +CPU= 25.362
  14:11:06 STEP 1479 H= 369:45 +CPU= 25.519
  14:11:33 STEP 1480 H= 370:00 +CPU= 26.981
  14:11:59 STEP 1481 H= 370:15 +CPU= 25.832
  14:12:25 STEP 1482 H= 370:30 +CPU= 25.391
  14:12:50 STEP 1483 H= 370:45 +CPU= 25.260
  14:13:16 STEP 1484 H= 371:00 +CPU= 26.180
  14:13:42 STEP 1485 H= 371:15 +CPU= 25.383
  14:14:07 STEP 1486 H= 371:30 +CPU= 24.780
  14:14:32 STEP 1487 H= 371:45 +CPU= 25.538
  14:15:08 STEP 1488 H= 372:00 +CPU= 35.920
Moving to projects directory: /boinc_data/boinc/slots/14/ICMGGh7zg+001488
Moving to projects directory: /boinc_data/boinc/slots/14/ICMSHh7zg+001488
Moving to projects directory: /boinc_data/boinc/slots/14/ICMUAh7zg+001488
  14:15:33 STEP 1489 H= 372:15 +CPU= 24.962
  14:15:58 STEP 1490 H= 372:30 +CPU= 24.498
  14:16:23 STEP 1491 H= 372:45 +CPU= 24.659
  14:16:50 STEP 1492 H= 373:00 +CPU= 26.791
  14:17:15 STEP 1493 H= 373:15 +CPU= 25.239
  14:17:40 STEP 1494 H= 373:30 +CPU= 25.054
  14:18:06 STEP 1495 H= 373:45 +CPU= 25.435
  14:18:32 STEP 1496 H= 374:00 +CPU= 26.065
  14:18:56 STEP 1497 H= 374:15 +CPU= 24.028
  14:19:20 STEP 1498 H= 374:30 +CPU= 24.102
  14:19:44 STEP 1499 H= 374:45 +CPU= 23.783
  14:20:15 STEP 1500 H= 375:00 +CPU= 30.146
Moving to projects directory: /boinc_data/boinc/slots/14/ICMGGh7zg+001500
Moving to projects directory: /boinc_data/boinc/slots/14/ICMSHh7zg+001500
Moving to projects directory: /boinc_data/boinc/slots/14/ICMUAh7zg+001500
  14:20:39 STEP 1501 H= 375:15 +CPU= 23.971
  14:21:03 STEP 1502 H= 375:30 +CPU= 24.498
  14:21:30 STEP 1503 H= 375:45 +CPU= 26.682
  14:21:57 STEP 1504 H= 376:00 +CPU= 26.448
  14:22:22 STEP 1505 H= 376:15 +CPU= 25.030
  14:22:47 STEP 1506 H= 376:30 +CPU= 25.161
  14:23:12 STEP 1507 H= 376:45 +CPU= 24.875
  14:23:38 STEP 1508 H= 377:00 +CPU= 25.355
  14:24:04 STEP 1509 H= 377:15 +CPU= 25.992
  14:24:29 STEP 1510 H= 377:30 +CPU= 25.024
  14:24:55 STEP 1511 H= 377:45 +CPU= 26.103
  14:25:28 STEP 1512 H= 378:00 +CPU= 32.752
Moving to projects directory: /boinc_data/boinc/slots/14/ICMGGh7zg+001512
Moving to projects directory: /boinc_data/boinc/slots/14/ICMSHh7zg+001512
Moving to projects directory: /boinc_data/boinc/slots/14/ICMUAh7zg+001512
  14:25:53 STEP 1513 H= 378:15 +CPU= 25.425
  14:26:18 STEP 1514 H= 378:30 +CPU= 24.720
  14:26:43 STEP 1515 H= 378:45 +CPU= 24.737
  14:27:10 STEP 1516 H= 379:00 +CPU= 26.684
  14:27:36 STEP 1517 H= 379:15 +CPU= 25.397
  14:28:02 STEP 1518 H= 379:30 +CPU= 26.030
  14:28:27 STEP 1519 H= 379:45 +CPU= 25.382
  14:28:54 STEP 1520 H= 380:00 +CPU= 26.970
  14:29:21 STEP 1521 H= 380:15 +CPU= 26.293
  14:29:46 STEP 1522 H= 380:30 +CPU= 25.433
  14:30:11 STEP 1523 H= 380:45 +CPU= 24.763
  14:30:44 STEP 1524 H= 381:00 +CPU= 32.884
Moving to projects directory: /boinc_data/boinc/slots/14/ICMGGh7zg+001524
Moving to projects directory: /boinc_data/boinc/slots/14/ICMSHh7zg+001524
Moving to projects directory: /boinc_data/boinc/slots/14/ICMUAh7zg+001524
  14:31:09 STEP 1525 H= 381:15 +CPU= 24.782
  14:31:35 STEP 1526 H= 381:30 +CPU= 26.059
  14:32:01 STEP 1527 H= 381:45 +CPU= 25.808
  14:32:29 STEP 1528 H= 382:00 +CPU= 27.215
  14:32:54 STEP 1529 H= 382:15 +CPU= 25.008
  14:33:20 STEP 1530 H= 382:30 +CPU= 25.785
  14:33:46 STEP 1531 H= 382:45 +CPU= 25.826
  14:34:11 STEP 1532 H= 383:00 +CPU= 25.147
  14:34:37 STEP 1533 H= 383:15 +CPU= 25.875
  14:35:03 STEP 1534 H= 383:30 +CPU= 26.123
  14:35:29 STEP 1535 H= 383:45 +CPU= 25.597
  14:36:04 STEP 1536 H= 384:00 +CPU= 35.142
Moving to projects directory: /boinc_data/boinc/slots/14/ICMGGh7zg+001536
Moving to projects directory: /boinc_data/boinc/slots/14/ICMSHh7zg+001536
Moving to projects directory: /boinc_data/boinc/slots/14/ICMUAh7zg+001536
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMGGh7zg+001440
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMSHh7zg+001440
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMUAh7zg+001440
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMGGh7zg+001452
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMSHh7zg+001452
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMUAh7zg+001452
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMGGh7zg+001464
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMSHh7zg+001464
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMUAh7zg+001464
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMGGh7zg+001476
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMSHh7zg+001476
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMUAh7zg+001476
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMGGh7zg+001488
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMSHh7zg+001488
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMUAh7zg+001488
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMGGh7zg+001500
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMSHh7zg+001500
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMUAh7zg+001500
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMGGh7zg+001512
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMSHh7zg+001512
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMUAh7zg+001512
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMGGh7zg+001524
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMSHh7zg+001524
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMUAh7zg+001524
Zipping up the intermediate file: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_a3ht_2016092300_20_1018_12292856_1_r1411055572_15.zip
Uploading the intermediate file: upload_file_15.zip
  14:36:30 STEP 1537 H= 384:15 +CPU= 25.580
  14:36:56 STEP 1538 H= 384:30 +CPU= 25.589
  14:37:21 STEP 1539 H= 384:45 +CPU= 25.681
  14:37:48 STEP 1540 H= 385:00 +CPU= 26.528
  14:38:15 STEP 1541 H= 385:15 +CPU= 26.871
  14:38:40 STEP 1542 H= 385:30 +CPU= 25.334
  14:39:06 STEP 1543 H= 385:45 +CPU= 25.181
  14:39:32 STEP 1544 H= 386:00 +CPU= 26.291
  14:39:58 STEP 1545 H= 386:15 +CPU= 25.681
  14:40:23 STEP 1546 H= 386:30 +CPU= 24.550
  14:40:48 STEP 1547 H= 386:45 +CPU= 25.235
  14:41:18 STEP 1548 H= 387:00 +CPU= 29.718
Moving to projects directory: /boinc_data/boinc/slots/14/ICMGGh7zg+001548
Moving to projects directory: /boinc_data/boinc/slots/14/ICMSHh7zg+001548
Moving to projects directory: /boinc_data/boinc/slots/14/ICMUAh7zg+001548
  14:41:43 STEP 1549 H= 387:15 +CPU= 25.337
  14:42:09 STEP 1550 H= 387:30 +CPU= 25.831
  14:42:35 STEP 1551 H= 387:45 +CPU= 25.194
  14:42:59 STEP 1552 H= 388:00 +CPU= 24.647
  14:43:24 STEP 1553 H= 388:15 +CPU= 24.615
  14:43:50 STEP 1554 H= 388:30 +CPU= 25.827
  14:44:16 STEP 1555 H= 388:45 +CPU= 25.820
  14:44:43 STEP 1556 H= 389:00 +CPU= 26.913
  14:45:09 STEP 1557 H= 389:15 +CPU= 25.941
  14:45:34 STEP 1558 H= 389:30 +CPU= 25.102
  14:45:59 STEP 1559 H= 389:45 +CPU= 24.714
  14:46:32 STEP 1560 H= 390:00 +CPU= 32.823
Moving to projects directory: /boinc_data/boinc/slots/14/ICMGGh7zg+001560
Moving to projects directory: /boinc_data/boinc/slots/14/ICMSHh7zg+001560
Moving to projects directory: /boinc_data/boinc/slots/14/ICMUAh7zg+001560
  14:46:57 STEP 1561 H= 390:15 +CPU= 25.163
  14:47:23 STEP 1562 H= 390:30 +CPU= 25.758
  14:47:49 STEP 1563 H= 390:45 +CPU= 25.411
  14:48:15 STEP 1564 H= 391:00 +CPU= 26.411
  14:48:41 STEP 1565 H= 391:15 +CPU= 25.269
  14:49:06 STEP 1566 H= 391:30 +CPU= 25.798
  14:49:32 STEP 1567 H= 391:45 +CPU= 25.671
  14:49:58 STEP 1568 H= 392:00 +CPU= 25.714
  14:50:23 STEP 1569 H= 392:15 +CPU= 24.792
  14:50:48 STEP 1570 H= 392:30 +CPU= 24.913
  14:51:13 STEP 1571 H= 392:45 +CPU= 24.900
  14:51:44 STEP 1572 H= 393:00 +CPU= 31.107
Moving to projects directory: /boinc_data/boinc/slots/14/ICMGGh7zg+001572
Moving to projects directory: /boinc_data/boinc/slots/14/ICMSHh7zg+001572
Moving to projects directory: /boinc_data/boinc/slots/14/ICMUAh7zg+001572
  14:52:10 STEP 1573 H= 393:15 +CPU= 25.594
  14:52:36 STEP 1574 H= 393:30 +CPU= 25.906
  14:53:02 STEP 1575 H= 393:45 +CPU= 25.676
  14:53:28 STEP 1576 H= 394:00 +CPU= 25.837
  14:53:54 STEP 1577 H= 394:15 +CPU= 25.967
  14:54:20 STEP 1578 H= 394:30 +CPU= 25.201
  14:54:46 STEP 1579 H= 394:45 +CPU= 25.969
  14:55:11 STEP 1580 H= 395:00 +CPU= 25.262
  14:55:37 STEP 1581 H= 395:15 +CPU= 25.666
  14:56:03 STEP 1582 H= 395:30 +CPU= 25.853
  14:56:28 STEP 1583 H= 395:45 +CPU= 25.346
  14:57:04 STEP 1584 H= 396:00 +CPU= 35.988
Moving to projects directory: /boinc_data/boinc/slots/14/ICMGGh7zg+001584
Moving to projects directory: /boinc_data/boinc/slots/14/ICMSHh7zg+001584
Moving to projects directory: /boinc_data/boinc/slots/14/ICMUAh7zg+001584
  14:57:30 STEP 1585 H= 396:15 +CPU= 25.939
  14:57:55 STEP 1586 H= 396:30 +CPU= 24.963
  14:58:21 STEP 1587 H= 396:45 +CPU= 25.657
  14:58:47 STEP 1588 H= 397:00 +CPU= 26.068
  14:59:12 STEP 1589 H= 397:15 +CPU= 24.512
  14:59:38 STEP 1590 H= 397:30 +CPU= 25.469
  15:00:02 STEP 1591 H= 397:45 +CPU= 24.542
  15:00:29 STEP 1592 H= 398:00 +CPU= 26.576
  15:00:55 STEP 1593 H= 398:15 +CPU= 26.092
  15:01:20 STEP 1594 H= 398:30 +CPU= 24.984
  15:01:45 STEP 1595 H= 398:45 +CPU= 24.647
  15:02:17 STEP 1596 H= 399:00 +CPU= 31.762
Moving to projects directory: /boinc_data/boinc/slots/14/ICMGGh7zg+001596
Moving to projects directory: /boinc_data/boinc/slots/14/ICMSHh7zg+001596
Moving to projects directory: /boinc_data/boinc/slots/14/ICMUAh7zg+001596
  15:02:43 STEP 1597 H= 399:15 +CPU= 25.953
  15:03:09 STEP 1598 H= 399:30 +CPU= 25.953
  15:03:34 STEP 1599 H= 399:45 +CPU= 24.834
  15:04:00 STEP 1600 H= 400:00 +CPU= 26.466
  15:04:25 STEP 1601 H= 400:15 +CPU= 24.325
  15:04:50 STEP 1602 H= 400:30 +CPU= 24.586
  15:05:16 STEP 1603 H= 400:45 +CPU= 26.033
  15:05:43 STEP 1604 H= 401:00 +CPU= 27.331
  15:06:09 STEP 1605 H= 401:15 +CPU= 25.914
  15:06:34 STEP 1606 H= 401:30 +CPU= 24.793
  15:06:59 STEP 1607 H= 401:45 +CPU= 24.195
  15:07:30 STEP 1608 H= 402:00 +CPU= 31.565
Moving to projects directory: /boinc_data/boinc/slots/14/ICMGGh7zg+001608
Moving to projects directory: /boinc_data/boinc/slots/14/ICMSHh7zg+001608
Moving to projects directory: /boinc_data/boinc/slots/14/ICMUAh7zg+001608
  15:07:56 STEP 1609 H= 402:15 +CPU= 25.803
  15:08:22 STEP 1610 H= 402:30 +CPU= 25.462
  15:08:48 STEP 1611 H= 402:45 +CPU= 26.165
  15:09:15 STEP 1612 H= 403:00 +CPU= 26.756
  15:09:41 STEP 1613 H= 403:15 +CPU= 26.082
  15:10:06 STEP 1614 H= 403:30 +CPU= 24.708
  15:10:31 STEP 1615 H= 403:45 +CPU= 24.844
  15:10:57 STEP 1616 H= 404:00 +CPU= 25.982
  15:11:23 STEP 1617 H= 404:15 +CPU= 25.584
  15:11:48 STEP 1618 H= 404:30 +CPU= 25.166
  15:12:14 STEP 1619 H= 404:45 +CPU= 26.040
  15:12:48 STEP 1620 H= 405:00 +CPU= 33.057
Moving to projects directory: /boinc_data/boinc/slots/14/ICMGGh7zg+001620
Moving to projects directory: /boinc_data/boinc/slots/14/ICMSHh7zg+001620
Moving to projects directory: /boinc_data/boinc/slots/14/ICMUAh7zg+001620
  15:13:14 STEP 1621 H= 405:15 +CPU= 25.513
  15:13:40 STEP 1622 H= 405:30 +CPU= 25.612
  15:14:06 STEP 1623 H= 405:45 +CPU= 25.909
  15:14:33 STEP 1624 H= 406:00 +CPU= 27.315
  15:15:00 STEP 1625 H= 406:15 +CPU= 26.022
  15:15:27 STEP 1626 H= 406:30 +CPU= 26.601
  15:15:53 STEP 1627 H= 406:45 +CPU= 25.887
  15:16:19 STEP 1628 H= 407:00 +CPU= 26.091
  15:16:45 STEP 1629 H= 407:15 +CPU= 25.102
  15:17:10 STEP 1630 H= 407:30 +CPU= 25.668
  15:17:37 STEP 1631 H= 407:45 +CPU= 26.748
  15:18:15 STEP 1632 H= 408:00 +CPU= 37.224
Moving to projects directory: /boinc_data/boinc/slots/14/ICMGGh7zg+001632
Moving to projects directory: /boinc_data/boinc/slots/14/ICMSHh7zg+001632
Moving to projects directory: /boinc_data/boinc/slots/14/ICMUAh7zg+001632
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMGGh7zg+001536
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMSHh7zg+001536
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMUAh7zg+001536
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMGGh7zg+001548
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMSHh7zg+001548
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMUAh7zg+001548
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMGGh7zg+001560
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMSHh7zg+001560
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMUAh7zg+001560
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMGGh7zg+001572
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMSHh7zg+001572
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMUAh7zg+001572
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMGGh7zg+001584
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMSHh7zg+001584
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMUAh7zg+001584
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMGGh7zg+001596
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMSHh7zg+001596
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMUAh7zg+001596
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMGGh7zg+001608
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMSHh7zg+001608
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMUAh7zg+001608
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMGGh7zg+001620
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMSHh7zg+001620
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMUAh7zg+001620
Zipping up the intermediate file: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_a3ht_2016092300_20_1018_12292856_1_r1411055572_16.zip
Uploading the intermediate file: upload_file_16.zip
  15:18:42 STEP 1633 H= 408:15 +CPU= 25.994
  15:19:07 STEP 1634 H= 408:30 +CPU= 24.643
  15:19:32 STEP 1635 H= 408:45 +CPU= 24.844
  15:19:58 STEP 1636 H= 409:00 +CPU= 26.201
  15:20:24 STEP 1637 H= 409:15 +CPU= 25.883
  15:20:49 STEP 1638 H= 409:30 +CPU= 24.781
  15:21:15 STEP 1639 H= 409:45 +CPU= 25.801
  15:21:43 STEP 1640 H= 410:00 +CPU= 27.461
  15:22:10 STEP 1641 H= 410:15 +CPU= 26.461
  15:22:36 STEP 1642 H= 410:30 +CPU= 25.823
  15:23:02 STEP 1643 H= 410:45 +CPU= 25.432
  15:23:36 STEP 1644 H= 411:00 +CPU= 33.299
Moving to projects directory: /boinc_data/boinc/slots/14/ICMGGh7zg+001644
Moving to projects directory: /boinc_data/boinc/slots/14/ICMSHh7zg+001644
Moving to projects directory: /boinc_data/boinc/slots/14/ICMUAh7zg+001644
  15:24:02 STEP 1645 H= 411:15 +CPU= 25.490
  15:24:28 STEP 1646 H= 411:30 +CPU= 25.860
  15:24:53 STEP 1647 H= 411:45 +CPU= 25.332
  15:25:19 STEP 1648 H= 412:00 +CPU= 25.813
  15:25:44 STEP 1649 H= 412:15 +CPU= 25.058
  15:26:09 STEP 1650 H= 412:30 +CPU= 24.758
  15:26:35 STEP 1651 H= 412:45 +CPU= 25.690
  15:27:02 STEP 1652 H= 413:00 +CPU= 26.804
  15:27:30 STEP 1653 H= 413:15 +CPU= 26.694
  15:27:56 STEP 1654 H= 413:30 +CPU= 26.026
  15:28:23 STEP 1655 H= 413:45 +CPU= 26.565
  15:28:56 STEP 1656 H= 414:00 +CPU= 32.868
Moving to projects directory: /boinc_data/boinc/slots/14/ICMGGh7zg+001656
Moving to projects directory: /boinc_data/boinc/slots/14/ICMSHh7zg+001656
Moving to projects directory: /boinc_data/boinc/slots/14/ICMUAh7zg+001656
  15:29:21 STEP 1657 H= 414:15 +CPU= 25.319
  15:29:45 STEP 1658 H= 414:30 +CPU= 23.612
  15:30:11 STEP 1659 H= 414:45 +CPU= 25.556
  15:30:37 STEP 1660 H= 415:00 +CPU= 26.080
  15:31:03 STEP 1661 H= 415:15 +CPU= 25.797
  15:31:29 STEP 1662 H= 415:30 +CPU= 26.357
  15:31:55 STEP 1663 H= 415:45 +CPU= 25.234
  15:32:20 STEP 1664 H= 416:00 +CPU= 24.806
  15:32:45 STEP 1665 H= 416:15 +CPU= 25.386
  15:33:09 STEP 1666 H= 416:30 +CPU= 24.055
  15:33:36 STEP 1667 H= 416:45 +CPU= 26.081
  15:34:08 STEP 1668 H= 417:00 +CPU= 31.902
Moving to projects directory: /boinc_data/boinc/slots/14/ICMGGh7zg+001668
Moving to projects directory: /boinc_data/boinc/slots/14/ICMSHh7zg+001668
Moving to projects directory: /boinc_data/boinc/slots/14/ICMUAh7zg+001668
  15:34:34 STEP 1669 H= 417:15 +CPU= 25.857
  15:34:59 STEP 1670 H= 417:30 +CPU= 25.612
  15:35:25 STEP 1671 H= 417:45 +CPU= 25.537
  15:35:51 STEP 1672 H= 418:00 +CPU= 25.434
  15:36:17 STEP 1673 H= 418:15 +CPU= 26.225
  15:36:42 STEP 1674 H= 418:30 +CPU= 25.201
  15:37:09 STEP 1675 H= 418:45 +CPU= 26.594
  15:37:36 STEP 1676 H= 419:00 +CPU= 26.599
  15:38:01 STEP 1677 H= 419:15 +CPU= 25.491
  15:38:27 STEP 1678 H= 419:30 +CPU= 25.730
  15:38:53 STEP 1679 H= 419:45 +CPU= 25.489
  15:39:27 STEP 1680 H= 420:00 +CPU= 34.354
Moving to projects directory: /boinc_data/boinc/slots/14/ICMGGh7zg+001680
Moving to projects directory: /boinc_data/boinc/slots/14/ICMSHh7zg+001680
Moving to projects directory: /boinc_data/boinc/slots/14/ICMUAh7zg+001680
  15:39:52 STEP 1681 H= 420:15 +CPU= 24.289
  15:40:18 STEP 1682 H= 420:30 +CPU= 25.749
  15:40:44 STEP 1683 H= 420:45 +CPU= 26.638
  15:41:11 STEP 1684 H= 421:00 +CPU= 26.623
  15:41:37 STEP 1685 H= 421:15 +CPU= 26.301
  15:42:03 STEP 1686 H= 421:30 +CPU= 25.346
  15:42:29 STEP 1687 H= 421:45 +CPU= 26.003
  15:42:55 STEP 1688 H= 422:00 +CPU= 25.660
  15:43:21 STEP 1689 H= 422:15 +CPU= 25.811
  15:43:46 STEP 1690 H= 422:30 +CPU= 25.380
  15:44:13 STEP 1691 H= 422:45 +CPU= 26.161
  15:44:45 STEP 1692 H= 423:00 +CPU= 32.006
Moving to projects directory: /boinc_data/boinc/slots/14/ICMGGh7zg+001692
Moving to projects directory: /boinc_data/boinc/slots/14/ICMSHh7zg+001692
Moving to projects directory: /boinc_data/boinc/slots/14/ICMUAh7zg+001692
  15:45:11 STEP 1693 H= 423:15 +CPU= 25.829
  15:45:36 STEP 1694 H= 423:30 +CPU= 24.809
  15:46:01 STEP 1695 H= 423:45 +CPU= 25.526
  15:46:27 STEP 1696 H= 424:00 +CPU= 25.023
  15:46:53 STEP 1697 H= 424:15 +CPU= 25.950
  15:47:19 STEP 1698 H= 424:30 +CPU= 26.644
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_bl_1.13_x86_64-pc-linux-gnu
(argv1) start_date: 2016092300
(argv2) exptid: h7zg
(argv3) unique_member_id: a3ht
(argv4) batchid: 1018
(argv5) wuid: 12292856
(argv6) fclen: 20
(argv7) app_name: oifs_43r3_bl
(argv8) nthreads: 1
Working directory is: /boinc_data/boinc/slots/14
Project directory is: /boinc_data/boinc/projects/climateprediction.net/
app name: oifs_43r3_bl
version: 1.13
Location of temp folder: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856
..mkdir for temp folder for results failed
Copying: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip to: /boinc_data/boinc/slots/14/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /boinc_data/boinc/slots/14/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_49eb20c099bc794c7cc6cf8519678482 to: /boinc_data/boinc/slots/14/oifs_43r3_bl_a3ht_2016092300_20_1018_12292856.zip
Unzipping the namelist zip file: /boinc_data/boinc/slots/14/oifs_43r3_bl_a3ht_2016092300_20_1018_12292856.zip
ic_ancil_file: ic_ancil_12292856
ifsdata_file: ifsdata_12292856
climate_data_file: clim_data_12292856
horiz_resolution: 159
vert_resolution: 91
grid_type: l_2
upload_interval: 96
utstep: 900
nfrres: restart dump frequency (steps) 48
Copying IC ancils from: ../../projects/climateprediction.net/jf_f7ecfd27aa1e68bbe7e8d19800e25c73 to: /boinc_data/boinc/slots/14/ic_ancil_12292856.zip
Unzipping the IC ancils zip file: /boinc_data/boinc/slots/14/ic_ancil_12292856.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_15fe15918f7f4acb5e36042a8b0bb4d7 to: /boinc_data/boinc/slots/14/ifsdata/ifsdata_12292856.zip
Unzipping the ifsdata_zip file: /boinc_data/boinc/slots/14/ifsdata/ifsdata_12292856.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /boinc_data/boinc/slots/14/159l_2/clim_data_12292856.zip
Unzipping the climate data zip file: /boinc_data/boinc/slots/14/159l_2/clim_data_12292856.zip
Checking for progress XML file: /boinc_data/boinc/slots/14/progress_file_12292856.xml
Opened progress file ok : /boinc_data/boinc/slots/14/progress_file_12292856.xml
-- Model is restarting --
Adjusting last_iter, 1699, to previous model restart step.
Creating progress file: /boinc_data/boinc/slots/14/progress_file_12292856.xml
last_cpu_time: 44692
upload_file_number: 17
last_iter: 1681
last_upload: 1468800
model_completed: 0
total_length_of_simulation: 1728000
result_base_name: oifs_43r3_bl_a3ht_2016092300_20_1018_12292856_1_r1411055572
The child process has been launched with process id: 146150
Executing the command: /boinc_data/boinc/slots/14/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:530FCTest:1:1:146150:146150] [20240618:155131:1718743891:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2091] fp = 0x314f440
[EC_DRHOOK:530FCTest:1:1:146150:146150] [20240618:155131:1718743891:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:530FCTest:1:1:146150:146150] [20240618:155131:1718743891:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:530FCTest:1:1:146150:146150] [20240618:155131:1718743891:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:530FCTest:1:1:146150:146150] [20240618:155131:1718743891:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:530FCTest:1:1:146150:146150] [20240618:155131:1718743891:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:530FCTest:1:1:146150:146150] [20240618:155131:1718743891:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:530FCTest:1:1:146150:146150] [20240618:155131:1718743891:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:530FCTest:1:1:146150:146150] [20240618:155131:1718743891:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:530FCTest:1:1:146150:146150] [20240618:155131:1718743891:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:530FCTest:1:1:146150:146150] [20240618:155131:1718743891:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:530FCTest:1:1:146150:146150] [20240618:155131:1718743891:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:530FCTest:1:1:146150:146150] [20240618:155131:1718743891:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:530FCTest:1:1:146150:146150] [20240618:155131:1718743891:0.000] [ignore_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:530FCTest:1:1:146150:146150] [20240618:155131:1718743891:0.000] [restore_default_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:530FCTest:1:1:146150:146150] [20240618:155131:1718743891:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:530FCTest:1:1:146150:146150] [20240618:155131:1718743891:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:530FCTest:1:1:146150:146150] [20240618:155131:1718743891:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:530FCTest:1:1:146150:146150] [20240618:155131:1718743891:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x820cf0 (old at (nil))
[EC_DRHOOK:530FCTest:1:1:146150:146150] [20240618:155131:1718743891:0.000] [trapfpe_treatment@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:530FCTest:1:1:146150:146150] [20240618:155131:1718743891:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:530FCTest:1:1:146150:146150] [20240618:155131:1718743891:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:530FCTest:1:1:146150:146150] [20240618:155131:1718743891:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x820cf0 (old at (nil))
[EC_DRHOOK:530FCTest:1:1:146150:146150] [20240618:155131:1718743891:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:530FCTest:1:1:146150:146150] [20240618:155131:1718743891:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:530FCTest:1:1:146150:146150] [20240618:155131:1718743891:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:530FCTest:1:1:146150:146150] [20240618:155131:1718743891:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x820cf0 (old at (nil))
[EC_DRHOOK:530FCTest:1:1:146150:146150] [20240618:155131:1718743891:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x820cf0 (old at (nil))
[EC_DRHOOK:530FCTest:1:1:146150:146150] [20240618:155131:1718743891:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:530FCTest:1:1:146150:146150] [20240618:155131:1718743891:0.001] [catch_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  15:52:03 STEP 1680 H= 420:00 +CPU= 27.368
  15:52:29 STEP 1681 H= 420:15 +CPU= 26.431
  15:52:56 STEP 1682 H= 420:30 +CPU= 26.214
  15:53:22 STEP 1683 H= 420:45 +CPU= 26.531
  15:53:49 STEP 1684 H= 421:00 +CPU= 26.773
  15:54:16 STEP 1685 H= 421:15 +CPU= 26.701
  15:54:42 STEP 1686 H= 421:30 +CPU= 26.297
  15:55:09 STEP 1687 H= 421:45 +CPU= 26.297
  15:55:35 STEP 1688 H= 422:00 +CPU= 26.622
  15:56:02 STEP 1689 H= 422:15 +CPU= 26.192
  15:56:29 STEP 1690 H= 422:30 +CPU= 26.850
  15:56:56 STEP 1691 H= 422:45 +CPU= 26.822
  15:57:28 STEP 1692 H= 423:00 +CPU= 32.630
Moving to projects directory: /boinc_data/boinc/slots/14/ICMGGh7zg+001692
Moving to projects directory: /boinc_data/boinc/slots/14/ICMSHh7zg+001692
Moving to projects directory: /boinc_data/boinc/slots/14/ICMUAh7zg+001692
  15:57:55 STEP 1693 H= 423:15 +CPU= 26.558
  15:58:22 STEP 1694 H= 423:30 +CPU= 26.672
  15:58:49 STEP 1695 H= 423:45 +CPU= 26.821
  15:59:16 STEP 1696 H= 424:00 +CPU= 27.011
  15:59:42 STEP 1697 H= 424:15 +CPU= 26.092
  16:00:07 STEP 1698 H= 424:30 +CPU= 25.195
  16:00:33 STEP 1699 H= 424:45 +CPU= 25.788
  16:01:01 STEP 1700 H= 425:00 +CPU= 27.287
  16:01:27 STEP 1701 H= 425:15 +CPU= 26.567
  16:01:54 STEP 1702 H= 425:30 +CPU= 26.770
  16:02:21 STEP 1703 H= 425:45 +CPU= 26.552
  16:02:54 STEP 1704 H= 426:00 +CPU= 33.149
Moving to projects directory: /boinc_data/boinc/slots/14/ICMGGh7zg+001704
Moving to projects directory: /boinc_data/boinc/slots/14/ICMSHh7zg+001704
Moving to projects directory: /boinc_data/boinc/slots/14/ICMUAh7zg+001704
  16:03:21 STEP 1705 H= 426:15 +CPU= 26.201
  16:03:47 STEP 1706 H= 426:30 +CPU= 26.524
  16:04:13 STEP 1707 H= 426:45 +CPU= 25.345
  16:04:40 STEP 1708 H= 427:00 +CPU= 27.478
  16:05:05 STEP 1709 H= 427:15 +CPU= 24.092
  16:05:30 STEP 1710 H= 427:30 +CPU= 24.813
  16:05:57 STEP 1711 H= 427:45 +CPU= 26.905
  16:06:24 STEP 1712 H= 428:00 +CPU= 27.564
  16:06:51 STEP 1713 H= 428:15 +CPU= 26.574
  16:07:18 STEP 1714 H= 428:30 +CPU= 26.558
  16:07:43 STEP 1715 H= 428:45 +CPU= 25.209
  16:08:15 STEP 1716 H= 429:00 +CPU= 31.921
Moving to projects directory: /boinc_data/boinc/slots/14/ICMGGh7zg+001716
Moving to projects directory: /boinc_data/boinc/slots/14/ICMSHh7zg+001716
Moving to projects directory: /boinc_data/boinc/slots/14/ICMUAh7zg+001716
  16:08:41 STEP 1717 H= 429:15 +CPU= 26.185
  16:09:08 STEP 1718 H= 429:30 +CPU= 26.477
  16:09:34 STEP 1719 H= 429:45 +CPU= 26.098
  16:09:59 STEP 1720 H= 430:00 +CPU= 24.774
  16:10:25 STEP 1721 H= 430:15 +CPU= 26.027
  16:10:52 STEP 1722 H= 430:30 +CPU= 26.393
  16:11:17 STEP 1723 H= 430:45 +CPU= 24.937
  16:11:43 STEP 1724 H= 431:00 +CPU= 26.092
  16:12:10 STEP 1725 H= 431:15 +CPU= 26.276
  16:12:36 STEP 1726 H= 431:30 +CPU= 26.168
  16:13:02 STEP 1727 H= 431:45 +CPU= 26.298
  16:13:37 STEP 1728 H= 432:00 +CPU= 34.990
Moving to projects directory: /boinc_data/boinc/slots/14/ICMGGh7zg+001728
Moving to projects directory: /boinc_data/boinc/slots/14/ICMSHh7zg+001728
Moving to projects directory: /boinc_data/boinc/slots/14/ICMUAh7zg+001728
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMGGh7zg+001632
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMSHh7zg+001632
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMUAh7zg+001632
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMGGh7zg+001644
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMSHh7zg+001644
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMUAh7zg+001644
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMGGh7zg+001656
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMSHh7zg+001656
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMUAh7zg+001656
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMGGh7zg+001668
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMSHh7zg+001668
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMUAh7zg+001668
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMGGh7zg+001680
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMSHh7zg+001680
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMUAh7zg+001680
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMGGh7zg+001692
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMSHh7zg+001692
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMUAh7zg+001692
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMGGh7zg+001704
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMSHh7zg+001704
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMUAh7zg+001704
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMGGh7zg+001716
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMSHh7zg+001716
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMUAh7zg+001716
Zipping up the intermediate file: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_a3ht_2016092300_20_1018_12292856_1_r1411055572_17.zip
Uploading the intermediate file: upload_file_17.zip
  16:14:02 STEP 1729 H= 432:15 +CPU= 24.841
  16:14:29 STEP 1730 H= 432:30 +CPU= 26.323
  16:14:55 STEP 1731 H= 432:45 +CPU= 25.811
  16:15:21 STEP 1732 H= 433:00 +CPU= 26.236
  16:15:48 STEP 1733 H= 433:15 +CPU= 26.711
  16:16:14 STEP 1734 H= 433:30 +CPU= 26.291
  16:16:40 STEP 1735 H= 433:45 +CPU= 25.946
  16:17:08 STEP 1736 H= 434:00 +CPU= 27.279
  16:17:34 STEP 1737 H= 434:15 +CPU= 26.611
  16:18:01 STEP 1738 H= 434:30 +CPU= 26.703
  16:18:28 STEP 1739 H= 434:45 +CPU= 26.804
  16:19:01 STEP 1740 H= 435:00 +CPU= 33.268
Moving to projects directory: /boinc_data/boinc/slots/14/ICMGGh7zg+001740
Moving to projects directory: /boinc_data/boinc/slots/14/ICMSHh7zg+001740
Moving to projects directory: /boinc_data/boinc/slots/14/ICMUAh7zg+001740
  16:19:21 STEP 1741 H= 435:15 +CPU= 19.611
  16:19:41 STEP 1742 H= 435:30 +CPU= 20.066
  16:20:05 STEP 1743 H= 435:45 +CPU= 24.125
  16:20:32 STEP 1744 H= 436:00 +CPU= 26.906
  16:20:58 STEP 1745 H= 436:15 +CPU= 25.397
  16:21:24 STEP 1746 H= 436:30 +CPU= 26.206
  16:21:50 STEP 1747 H= 436:45 +CPU= 25.275
  16:22:15 STEP 1748 H= 437:00 +CPU= 25.806
  16:22:40 STEP 1749 H= 437:15 +CPU= 24.709
  16:23:06 STEP 1750 H= 437:30 +CPU= 26.096
  16:23:32 STEP 1751 H= 437:45 +CPU= 25.674
  16:24:04 STEP 1752 H= 438:00 +CPU= 31.477
Moving to projects directory: /boinc_data/boinc/slots/14/ICMGGh7zg+001752
Moving to projects directory: /boinc_data/boinc/slots/14/ICMSHh7zg+001752
Moving to projects directory: /boinc_data/boinc/slots/14/ICMUAh7zg+001752
  16:24:30 STEP 1753 H= 438:15 +CPU= 26.367
  16:24:55 STEP 1754 H= 438:30 +CPU= 24.345
  16:25:20 STEP 1755 H= 438:45 +CPU= 25.570
  16:25:47 STEP 1756 H= 439:00 +CPU= 26.398
  16:26:13 STEP 1757 H= 439:15 +CPU= 25.765
  16:26:38 STEP 1758 H= 439:30 +CPU= 25.482
  16:27:04 STEP 1759 H= 439:45 +CPU= 26.071
  16:27:32 STEP 1760 H= 440:00 +CPU= 27.141
  16:27:58 STEP 1761 H= 440:15 +CPU= 25.929
  16:28:23 STEP 1762 H= 440:30 +CPU= 25.015
  16:28:48 STEP 1763 H= 440:45 +CPU= 25.183
  16:29:20 STEP 1764 H= 441:00 +CPU= 32.023
Moving to projects directory: /boinc_data/boinc/slots/14/ICMGGh7zg+001764
Moving to projects directory: /boinc_data/boinc/slots/14/ICMSHh7zg+001764
Moving to projects directory: /boinc_data/boinc/slots/14/ICMUAh7zg+001764
  16:29:46 STEP 1765 H= 441:15 +CPU= 25.718
  16:30:12 STEP 1766 H= 441:30 +CPU= 25.780
  16:30:38 STEP 1767 H= 441:45 +CPU= 26.261
  16:31:05 STEP 1768 H= 442:00 +CPU= 26.629
  16:31:31 STEP 1769 H= 442:15 +CPU= 25.581
  16:31:57 STEP 1770 H= 442:30 +CPU= 26.602
  16:32:24 STEP 1771 H= 442:45 +CPU= 26.301
  16:32:50 STEP 1772 H= 443:00 +CPU= 26.793
  16:33:16 STEP 1773 H= 443:15 +CPU= 24.992
  16:33:41 STEP 1774 H= 443:30 +CPU= 25.584
  16:34:06 STEP 1775 H= 443:45 +CPU= 24.795
  16:34:41 STEP 1776 H= 444:00 +CPU= 35.196
Moving to projects directory: /boinc_data/boinc/slots/14/ICMGGh7zg+001776
Moving to projects directory: /boinc_data/boinc/slots/14/ICMSHh7zg+001776
Moving to projects directory: /boinc_data/boinc/slots/14/ICMUAh7zg+001776
  16:35:08 STEP 1777 H= 444:15 +CPU= 26.116
  16:35:34 STEP 1778 H= 444:30 +CPU= 26.273
  16:35:59 STEP 1779 H= 444:45 +CPU= 25.180
  16:36:25 STEP 1780 H= 445:00 +CPU= 25.779
  16:36:51 STEP 1781 H= 445:15 +CPU= 25.381
  16:37:16 STEP 1782 H= 445:30 +CPU= 25.078
  16:37:41 STEP 1783 H= 445:45 +CPU= 25.586
  16:38:08 STEP 1784 H= 446:00 +CPU= 26.811
  16:38:33 STEP 1785 H= 446:15 +CPU= 24.762
  16:38:59 STEP 1786 H= 446:30 +CPU= 25.528
  16:39:24 STEP 1787 H= 446:45 +CPU= 25.139
  16:39:57 STEP 1788 H= 447:00 +CPU= 32.452
Moving to projects directory: /boinc_data/boinc/slots/14/ICMGGh7zg+001788
Moving to projects directory: /boinc_data/boinc/slots/14/ICMSHh7zg+001788
Moving to projects directory: /boinc_data/boinc/slots/14/ICMUAh7zg+001788
  16:40:22 STEP 1789 H= 447:15 +CPU= 24.801
  16:40:47 STEP 1790 H= 447:30 +CPU= 25.365
  16:41:13 STEP 1791 H= 447:45 +CPU= 26.199
  16:41:39 STEP 1792 H= 448:00 +CPU= 25.877
  16:42:06 STEP 1793 H= 448:15 +CPU= 26.090
  16:42:32 STEP 1794 H= 448:30 +CPU= 26.210
  16:42:58 STEP 1795 H= 448:45 +CPU= 26.445
  16:43:26 STEP 1796 H= 449:00 +CPU= 27.058
  16:43:53 STEP 1797 H= 449:15 +CPU= 27.124
  16:44:21 STEP 1798 H= 449:30 +CPU= 27.548
  16:44:49 STEP 1799 H= 449:45 +CPU= 27.400
  16:45:23 STEP 1800 H= 450:00 +CPU= 33.550
Moving to projects directory: /boinc_data/boinc/slots/14/ICMGGh7zg+001800
Moving to projects directory: /boinc_data/boinc/slots/14/ICMSHh7zg+001800
Moving to projects directory: /boinc_data/boinc/slots/14/ICMUAh7zg+001800
  16:45:50 STEP 1801 H= 450:15 +CPU= 26.961
  16:46:17 STEP 1802 H= 450:30 +CPU= 26.725
  16:46:43 STEP 1803 H= 450:45 +CPU= 25.435
  16:47:08 STEP 1804 H= 451:00 +CPU= 25.004
  16:47:34 STEP 1805 H= 451:15 +CPU= 25.251
  16:47:59 STEP 1806 H= 451:30 +CPU= 25.767
  16:48:25 STEP 1807 H= 451:45 +CPU= 25.813
  16:48:52 STEP 1808 H= 452:00 +CPU= 26.558
  16:49:18 STEP 1809 H= 452:15 +CPU= 25.969
  16:49:43 STEP 1810 H= 452:30 +CPU= 24.825
  16:50:09 STEP 1811 H= 452:45 +CPU= 25.628
  16:50:41 STEP 1812 H= 453:00 +CPU= 32.245
Moving to projects directory: /boinc_data/boinc/slots/14/ICMGGh7zg+001812
Moving to projects directory: /boinc_data/boinc/slots/14/ICMSHh7zg+001812
Moving to projects directory: /boinc_data/boinc/slots/14/ICMUAh7zg+001812
  16:51:08 STEP 1813 H= 453:15 +CPU= 26.232
  16:51:34 STEP 1814 H= 453:30 +CPU= 25.745
  16:51:59 STEP 1815 H= 453:45 +CPU= 25.407
  16:52:25 STEP 1816 H= 454:00 +CPU= 26.031
  16:52:52 STEP 1817 H= 454:15 +CPU= 26.283
  16:53:18 STEP 1818 H= 454:30 +CPU= 26.026
  16:53:43 STEP 1819 H= 454:45 +CPU= 24.896
  16:54:09 STEP 1820 H= 455:00 +CPU= 25.776
  16:54:34 STEP 1821 H= 455:15 +CPU= 25.200
  16:55:00 STEP 1822 H= 455:30 +CPU= 25.921
  16:55:26 STEP 1823 H= 455:45 +CPU= 25.447
  16:56:02 STEP 1824 H= 456:00 +CPU= 36.234
Moving to projects directory: /boinc_data/boinc/slots/14/ICMGGh7zg+001824
Moving to projects directory: /boinc_data/boinc/slots/14/ICMSHh7zg+001824
Moving to projects directory: /boinc_data/boinc/slots/14/ICMUAh7zg+001824
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMGGh7zg+001728
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMSHh7zg+001728
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMUAh7zg+001728
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMGGh7zg+001740
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMSHh7zg+001740
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMUAh7zg+001740
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMGGh7zg+001752
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMSHh7zg+001752
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMUAh7zg+001752
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMGGh7zg+001764
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMSHh7zg+001764
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMUAh7zg+001764
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMGGh7zg+001776
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMSHh7zg+001776
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMUAh7zg+001776
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMGGh7zg+001788
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMSHh7zg+001788
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMUAh7zg+001788
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMGGh7zg+001800
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMSHh7zg+001800
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMUAh7zg+001800
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMGGh7zg+001812
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMSHh7zg+001812
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMUAh7zg+001812
Zipping up the intermediate file: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_a3ht_2016092300_20_1018_12292856_1_r1411055572_18.zip
Uploading the intermediate file: upload_file_18.zip
  16:56:29 STEP 1825 H= 456:15 +CPU= 26.205
  16:56:56 STEP 1826 H= 456:30 +CPU= 26.841
  16:57:20 STEP 1827 H= 456:45 +CPU= 24.522
  16:57:47 STEP 1828 H= 457:00 +CPU= 26.438
  16:58:13 STEP 1829 H= 457:15 +CPU= 26.182
  16:58:38 STEP 1830 H= 457:30 +CPU= 24.804
  16:59:04 STEP 1831 H= 457:45 +CPU= 25.254
  16:59:29 STEP 1832 H= 458:00 +CPU= 25.505
  16:59:54 STEP 1833 H= 458:15 +CPU= 24.237
  17:00:19 STEP 1834 H= 458:30 +CPU= 25.018
  17:00:44 STEP 1835 H= 458:45 +CPU= 25.143
  17:01:16 STEP 1836 H= 459:00 +CPU= 32.329
Moving to projects directory: /boinc_data/boinc/slots/14/ICMGGh7zg+001836
Moving to projects directory: /boinc_data/boinc/slots/14/ICMSHh7zg+001836
Moving to projects directory: /boinc_data/boinc/slots/14/ICMUAh7zg+001836
  17:01:41 STEP 1837 H= 459:15 +CPU= 24.931
  17:02:06 STEP 1838 H= 459:30 +CPU= 24.461
  17:02:32 STEP 1839 H= 459:45 +CPU= 25.963
  17:03:00 STEP 1840 H= 460:00 +CPU= 27.725
  17:03:27 STEP 1841 H= 460:15 +CPU= 26.389
  17:03:52 STEP 1842 H= 460:30 +CPU= 25.208
  17:04:17 STEP 1843 H= 460:45 +CPU= 25.341
  17:04:43 STEP 1844 H= 461:00 +CPU= 25.443
  17:05:08 STEP 1845 H= 461:15 +CPU= 25.340
  17:05:33 STEP 1846 H= 461:30 +CPU= 24.794
  17:05:58 STEP 1847 H= 461:45 +CPU= 24.549
  17:06:30 STEP 1848 H= 462:00 +CPU= 31.716
Moving to projects directory: /boinc_data/boinc/slots/14/ICMGGh7zg+001848
Moving to projects directory: /boinc_data/boinc/slots/14/ICMSHh7zg+001848
Moving to projects directory: /boinc_data/boinc/slots/14/ICMUAh7zg+001848
  17:06:55 STEP 1849 H= 462:15 +CPU= 25.049
  17:07:20 STEP 1850 H= 462:30 +CPU= 24.611
  17:07:45 STEP 1851 H= 462:45 +CPU= 25.147
  17:08:11 STEP 1852 H= 463:00 +CPU= 26.124
  17:08:37 STEP 1853 H= 463:15 +CPU= 25.891
  17:09:02 STEP 1854 H= 463:30 +CPU= 24.697
  17:09:29 STEP 1855 H= 463:45 +CPU= 26.553
  17:09:54 STEP 1856 H= 464:00 +CPU= 25.709
  17:10:20 STEP 1857 H= 464:15 +CPU= 25.123
  17:10:45 STEP 1858 H= 464:30 +CPU= 25.177
  17:11:11 STEP 1859 H= 464:45 +CPU= 25.947
  17:11:43 STEP 1860 H= 465:00 +CPU= 32.212
Moving to projects directory: /boinc_data/boinc/slots/14/ICMGGh7zg+001860
Moving to projects directory: /boinc_data/boinc/slots/14/ICMSHh7zg+001860
Moving to projects directory: /boinc_data/boinc/slots/14/ICMUAh7zg+001860
  17:12:05 STEP 1861 H= 465:15 +CPU= 21.506
  17:12:23 STEP 1862 H= 465:30 +CPU= 18.542
  17:12:44 STEP 1863 H= 465:45 +CPU= 20.382
  17:13:10 STEP 1864 H= 466:00 +CPU= 26.162
  17:13:36 STEP 1865 H= 466:15 +CPU= 25.464
  17:14:02 STEP 1866 H= 466:30 +CPU= 25.770
  17:14:27 STEP 1867 H= 466:45 +CPU= 25.558
  17:14:53 STEP 1868 H= 467:00 +CPU= 25.840
  17:15:20 STEP 1869 H= 467:15 +CPU= 26.435
  17:15:45 STEP 1870 H= 467:30 +CPU= 25.414
  17:16:11 STEP 1871 H= 467:45 +CPU= 25.376
  17:16:47 STEP 1872 H= 468:00 +CPU= 35.720
Moving to projects directory: /boinc_data/boinc/slots/14/ICMGGh7zg+001872
Moving to projects directory: /boinc_data/boinc/slots/14/ICMSHh7zg+001872
Moving to projects directory: /boinc_data/boinc/slots/14/ICMUAh7zg+001872
  17:17:13 STEP 1873 H= 468:15 +CPU= 25.686
  17:17:38 STEP 1874 H= 468:30 +CPU= 25.117
  17:18:04 STEP 1875 H= 468:45 +CPU= 26.312
  17:18:31 STEP 1876 H= 469:00 +CPU= 26.150
  17:18:56 STEP 1877 H= 469:15 +CPU= 25.855
  17:19:23 STEP 1878 H= 469:30 +CPU= 25.957
  17:19:49 STEP 1879 H= 469:45 +CPU= 26.375
  17:20:16 STEP 1880 H= 470:00 +CPU= 26.405
  17:20:42 STEP 1881 H= 470:15 +CPU= 26.047
  17:21:08 STEP 1882 H= 470:30 +CPU= 25.929
  17:21:36 STEP 1883 H= 470:45 +CPU= 27.474
  17:22:09 STEP 1884 H= 471:00 +CPU= 33.206
Moving to projects directory: /boinc_data/boinc/slots/14/ICMGGh7zg+001884
Moving to projects directory: /boinc_data/boinc/slots/14/ICMSHh7zg+001884
Moving to projects directory: /boinc_data/boinc/slots/14/ICMUAh7zg+001884
  17:22:36 STEP 1885 H= 471:15 +CPU= 26.479
  17:23:02 STEP 1886 H= 471:30 +CPU= 25.921
  17:23:29 STEP 1887 H= 471:45 +CPU= 26.592
  17:23:56 STEP 1888 H= 472:00 +CPU= 27.015
  17:24:23 STEP 1889 H= 472:15 +CPU= 26.079
  17:24:49 STEP 1890 H= 472:30 +CPU= 26.081
  17:25:16 STEP 1891 H= 472:45 +CPU= 27.105
  17:25:44 STEP 1892 H= 473:00 +CPU= 27.157
  17:26:11 STEP 1893 H= 473:15 +CPU= 26.676
  17:26:37 STEP 1894 H= 473:30 +CPU= 26.654
  17:27:02 STEP 1895 H= 473:45 +CPU= 24.781
  17:27:37 STEP 1896 H= 474:00 +CPU= 34.063
Moving to projects directory: /boinc_data/boinc/slots/14/ICMGGh7zg+001896
Moving to projects directory: /boinc_data/boinc/slots/14/ICMSHh7zg+001896
Moving to projects directory: /boinc_data/boinc/slots/14/ICMUAh7zg+001896
  17:28:05 STEP 1897 H= 474:15 +CPU= 27.814
  17:28:33 STEP 1898 H= 474:30 +CPU= 27.588
  17:29:00 STEP 1899 H= 474:45 +CPU= 26.687
  17:29:27 STEP 1900 H= 475:00 +CPU= 26.718
  17:29:53 STEP 1901 H= 475:15 +CPU= 26.368
  17:30:19 STEP 1902 H= 475:30 +CPU= 25.953
  17:30:41 STEP 1903 H= 475:45 +CPU= 21.579
  17:31:00 STEP 1904 H= 476:00 +CPU= 19.381
  17:31:21 STEP 1905 H= 476:15 +CPU= 20.651
  17:31:46 STEP 1906 H= 476:30 +CPU= 25.408
  17:32:11 STEP 1907 H= 476:45 +CPU= 25.104
  17:32:43 STEP 1908 H= 477:00 +CPU= 31.802
Moving to projects directory: /boinc_data/boinc/slots/14/ICMGGh7zg+001908
Moving to projects directory: /boinc_data/boinc/slots/14/ICMSHh7zg+001908
Moving to projects directory: /boinc_data/boinc/slots/14/ICMUAh7zg+001908
  17:33:09 STEP 1909 H= 477:15 +CPU= 25.109
  17:33:34 STEP 1910 H= 477:30 +CPU= 25.160
  17:33:59 STEP 1911 H= 477:45 +CPU= 25.292
  17:34:25 STEP 1912 H= 478:00 +CPU= 25.707
  17:34:51 STEP 1913 H= 478:15 +CPU= 26.224
  17:35:17 STEP 1914 H= 478:30 +CPU= 25.432
  17:35:42 STEP 1915 H= 478:45 +CPU= 25.143
  17:36:09 STEP 1916 H= 479:00 +CPU= 26.308
  17:36:33 STEP 1917 H= 479:15 +CPU= 24.930
  17:36:59 STEP 1918 H= 479:30 +CPU= 25.580
  17:37:24 STEP 1919 H= 479:45 +CPU= 24.942
  17:37:57 STEP 1920 H= 480:00 +CPU= 32.525
..The child process terminated with status: 0
>>> Printing last 70 lines from file: NODE.001_01

 
DDH------------------------------------------------------------------------
I                                                                         I
I TIME =  480.000H DDH * Diagnostic Var.,     AREA                GLOBAL  I
I                                                                         I
I-------------------------------------------------------------------------I
I   VARIABLE   I      Initial value         I        Current value        I
I-------------------------------------------------------------------------I
I  OMEGA (P/S) I      0.8296250612E-21      I       0.2127349214E-05      I
ICloud fractionI      0.0000000000E+00      I       0.4903883944E-01      I
I  Relat. Hum. I      0.2638767698E+00      I       0.3825982734E+00      I
I  PASS 01     I      0.0000000000E+00      I       0.6667190234E+03      I
DDH------------------------------------------------------------------------
  
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
 NSTEP =  1920 SCAN2M_HPOS  P
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ICMGGh7zg+001920 MODE=a
 IO-STREAM CLOSED - ICMGGh7zg+001920
 MPI-TASK:   1 -    394228308 BYTES IN      7 RECORDS TRANSFERRED IN    0.0006 SECONDS657047.2 Mbytes/s, TOTAL TIME=    0.0114( 5.3%)
 17:37:57 STEP 1920 H= 480:00 +CPU= 32.525
 END CNT3
 NSTEP =  1920 CNT0   000000000  23.314  23.314  23.379   0.997               0               0               0               0               0
IO-STREAM STATISTICS, TOTAL NO OF REC/BYTES READ           46/  1069125708 WRITTEN         7150/************
 -TASK-OPENED-OPEN-RECS IN -KBYTE IN    -RECS OUT-KBYTE OUT-WALL   -WALL IN-WALL
  OU-TOT IN -TOT OUT
   1    128    0       46      1044068     7150 ************    13.6     1.6    12.0     2.4    25.5
===-=== START OF TIMING STATISTICS ===-===
 
STATS FOR ALL TASKS
 NUM ROUTINE                                     CALLS  MEAN(ms)   MAX(ms)   FRAC(%)  UNBAL(%)
   0 CNT0     - COMPLETE EXECUTION                   1 6385464.9 6385464.9    100.00      0.00
   1 CNT4     - FORWARD INTEGRATION                  1 6381567.1 6381567.1     99.94      0.00
   8 SCAN2M - GRID-POINT DYNAMICS                  261    4499.1    4499.1     18.39      0.00
   9 SPCM     - SPECTRAL COMP.                     240     605.9     605.9      2.28      0.00
  10 SCAN2M - PHYSICS                              241   12199.7   12199.7     46.04      0.00
  11 IOPACK   - OUTPUT P.P. RESULTS                 20     862.7     862.7      0.27      0.00
  12 SPNORM   - SPECTRAL NORM COMP.                 11      39.6      39.6      0.01      0.00
  17 GRIDFPOS IN CNT4                               20      24.1      24.1      0.01      0.00
  27 RESTART FILES - WRITING                         5    3450.6    3450.6      0.27      0.00
  28 RESTART FILES - READING                         1    3147.0    3147.0      0.05      0.00
  29 SU4FPOS IN CNT4                                20       0.1       0.1      0.00      0.00
  30 DYNFPOS IN CNT4                                20    5767.5    5767.5      1.81      0.00
  31 POSDDH IN STEPO                                 2       0.8       0.8      0.00      0.00
  37 CPGLAG   - SL COMPUTATIONS                    241    5827.1    5827.1     21.99      0.00
  39 SU0YOMB                                         1     465.6     465.6      0.01      0.00
  51 SCAN2M   - SL COMM. PART 1                    241      19.9      19.9      0.08      0.00
  54 SPCM     - M TO S/S TO M TRANSP.              240     289.1     289.1      1.09      0.00
  55 SPCIMPF  - S TO M/M TO S TRANSP.              240      53.1      53.1      0.20      0.00
  56 SPNORM   - SPECTRAL NORM COMM.                 11       0.2       0.2      0.00      0.00
 102 LTINV_CTL   - INVERSE LEGENDRE TRANSFORM      524     388.1     388.1      3.18      0.00
 103 LTDIR_CTL   - DIRECT LEGENDRE TRANSFORM       500     232.0     232.0      1.82      0.00
 106 FTDIR_CTL   - DIRECT FOURIER TRANSFORM        500      88.2      88.2      0.69      0.00
 107 FTINV_CTL   - INVERSE FOURIER TRANSFORM       524     113.3     113.3      0.93      0.00
 140 SULEG       - COMP. OF LEGENDRE POL.            1      52.3      52.3      0.00      0.00
 152 LTINV_CTL   - M TO L TRANSPOSITION            524      60.7      60.7      0.50      0.00
 153 LTDIR_CTL   - L TO M TRANSPOSITION            500      33.3      33.3      0.26      0.00
 157 FTINV_CTL   - L TO G TRANSPOSITION            524      46.2      46.2      0.38      0.00
 158 FTDIR_CTL   - G TO L TRANSPOSITION            500      27.0      27.0      0.21      0.00
 400 GSTATS                                      28864       0.0       0.0      0.00      0.00
 401 GSTATS HOOK                                 26802       0.0       0.0      0.00      0.00
TOTAL MEASURED IMBALANCE =       0.0 SECONDS,  0.0 PERCENT
TOTAL WALLCLOCK TIME     6385.5 CPU TIME    6354.1 VECTOR TIME     6354.1


===-=== END   OF TIMING STATISTICS ===-===
FORECAST DAYS PER DAY      270.8
  *** END CNT0 *** 
------------------------------------------------
Moving to projects directory: /boinc_data/boinc/slots/14/ICMGGh7zg+001920
Moving to projects directory: /boinc_data/boinc/slots/14/ICMSHh7zg+001920
Moving to projects directory: /boinc_data/boinc/slots/14/ICMUAh7zg+001920
Adding to the zip: /boinc_data/boinc/slots/14/NODE.001_01
Adding to the zip: /boinc_data/boinc/slots/14/ifs.stat
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMSHh7zg+001896
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMGGh7zg+001836
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMUAh7zg+001896
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMSHh7zg+001908
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMGGh7zg+001872
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMGGh7zg+001908
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMUAh7zg+001884
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMGGh7zg+001848
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMGGh7zg+001860
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMSHh7zg+001824
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMUAh7zg+001920
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMUAh7zg+001824
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMSHh7zg+001920
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMSHh7zg+001884
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMUAh7zg+001872
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMSHh7zg+001860
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMGGh7zg+001884
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMUAh7zg+001836
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMUAh7zg+001848
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMGGh7zg+001824
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMUAh7zg+001908
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMSHh7zg+001836
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMSHh7zg+001872
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMGGh7zg+001896
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMGGh7zg+001920
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMSHh7zg+001848
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12292856/ICMUAh7zg+001860
Zipping up the final file: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_a3ht_2016092300_20_1018_12292856_1_r1411055572_19.zip
Uploading the final file: upload_file_19.zip
Uploading trickle at timestep: 1727100
17:41:28 (146147): called boinc_finish(0)

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
18 Jun 2024 23:15:43 1496046 22449669 oifs_43r3_bl_a3ht_2016092300_20_1018_12292856_1 1,727,100 51,046 0.0296


©2024 climateprediction.net