climateprediction.net home page
Task 22447728

Task 22447728

Name oifs_43r3_bl_a4go_2016092300_20_1018_12294111_0
Workunit 12294111
Created 13 Jun 2024, 4:23:22 UTC
Sent 13 Jun 2024, 19:58:15 UTC
Report deadline 12 Aug 2024, 19:58:15 UTC
Received 15 Jun 2024, 19:06:43 UTC
Server state Over
Outcome Success
Client state Done
Exit status 0 (0x00000000)
Computer ID 1496046
Run time 14 hours 12 min 57 sec
CPU time 14 hours 10 min 28 sec
Validate state Valid
Credit 2,635.55
Device peak FLOPS 5.56 GFLOPS
Application version OpenIFS 43r3 Baroclinic Lifecycle v1.13
x86_64-pc-linux-gnu
Peak working set size 5,022.75 MB
Peak swap size 5,981.07 MB
Peak disk usage 1,345.60 MB
Stderr
<core_client_version>7.20.2</core_client_version>
<![CDATA[
<stderr_txt>
:45 +CPU= 27.725
  08:57:37 STEP 1420 H= 355:00 +CPU= 26.430
  08:58:04 STEP 1421 H= 355:15 +CPU= 26.760
  08:58:31 STEP 1422 H= 355:30 +CPU= 27.276
  08:58:56 STEP 1423 H= 355:45 +CPU= 25.130
  08:59:23 STEP 1424 H= 356:00 +CPU= 26.541
  08:59:50 STEP 1425 H= 356:15 +CPU= 27.109
  09:00:17 STEP 1426 H= 356:30 +CPU= 26.908
  09:00:46 STEP 1427 H= 356:45 +CPU= 28.675
  09:01:19 STEP 1428 H= 357:00 +CPU= 33.280
Moving to projects directory: /boinc_data/boinc/slots/7/ICMGGh7zg+001428
Moving to projects directory: /boinc_data/boinc/slots/7/ICMSHh7zg+001428
Moving to projects directory: /boinc_data/boinc/slots/7/ICMUAh7zg+001428
  09:01:43 STEP 1429 H= 357:15 +CPU= 23.575
  09:02:04 STEP 1430 H= 357:30 +CPU= 21.619
  09:02:29 STEP 1431 H= 357:45 +CPU= 24.846
  09:03:00 STEP 1432 H= 358:00 +CPU= 30.462
  09:03:24 STEP 1433 H= 358:15 +CPU= 24.033
  09:03:48 STEP 1434 H= 358:30 +CPU= 23.972
  09:04:11 STEP 1435 H= 358:45 +CPU= 22.851
  09:04:35 STEP 1436 H= 359:00 +CPU= 24.513
  09:04:59 STEP 1437 H= 359:15 +CPU= 23.626
  09:05:24 STEP 1438 H= 359:30 +CPU= 24.842
  09:05:48 STEP 1439 H= 359:45 +CPU= 24.247
  09:06:15 STEP 1440 H= 360:00 +CPU= 26.742
Moving to projects directory: /boinc_data/boinc/slots/7/ICMGGh7zg+001440
Moving to projects directory: /boinc_data/boinc/slots/7/ICMSHh7zg+001440
Moving to projects directory: /boinc_data/boinc/slots/7/ICMUAh7zg+001440
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMGGh7zg+001344
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMSHh7zg+001344
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMUAh7zg+001344
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMGGh7zg+001356
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMSHh7zg+001356
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMUAh7zg+001356
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMGGh7zg+001368
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMSHh7zg+001368
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMUAh7zg+001368
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMGGh7zg+001380
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMSHh7zg+001380
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMUAh7zg+001380
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMGGh7zg+001392
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMSHh7zg+001392
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMUAh7zg+001392
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMGGh7zg+001404
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMSHh7zg+001404
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMUAh7zg+001404
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMGGh7zg+001416
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMSHh7zg+001416
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMUAh7zg+001416
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMGGh7zg+001428
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMSHh7zg+001428
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMUAh7zg+001428
Zipping up the intermediate file: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_a4go_2016092300_20_1018_12294111_0_r1581845035_14.zip
Uploading the intermediate file: upload_file_14.zip
  09:06:39 STEP 1441 H= 360:15 +CPU= 23.532
  09:07:04 STEP 1442 H= 360:30 +CPU= 25.037
  09:07:30 STEP 1443 H= 360:45 +CPU= 25.968
  09:07:57 STEP 1444 H= 361:00 +CPU= 27.010
  09:08:25 STEP 1445 H= 361:15 +CPU= 28.460
  09:08:51 STEP 1446 H= 361:30 +CPU= 25.029
  09:09:16 STEP 1447 H= 361:45 +CPU= 25.698
  09:09:43 STEP 1448 H= 362:00 +CPU= 26.339
  09:10:10 STEP 1449 H= 362:15 +CPU= 27.335
  09:10:36 STEP 1450 H= 362:30 +CPU= 26.033
  09:11:02 STEP 1451 H= 362:45 +CPU= 26.014
  09:11:33 STEP 1452 H= 363:00 +CPU= 30.888
Moving to projects directory: /boinc_data/boinc/slots/7/ICMGGh7zg+001452
Moving to projects directory: /boinc_data/boinc/slots/7/ICMSHh7zg+001452
Moving to projects directory: /boinc_data/boinc/slots/7/ICMUAh7zg+001452
  09:11:59 STEP 1453 H= 363:15 +CPU= 25.856
  09:12:27 STEP 1454 H= 363:30 +CPU= 28.007
  09:12:52 STEP 1455 H= 363:45 +CPU= 24.507
  09:13:18 STEP 1456 H= 364:00 +CPU= 26.327
  09:13:44 STEP 1457 H= 364:15 +CPU= 25.801
  09:14:11 STEP 1458 H= 364:30 +CPU= 27.424
  09:14:38 STEP 1459 H= 364:45 +CPU= 26.365
  09:15:04 STEP 1460 H= 365:00 +CPU= 26.455
  09:15:30 STEP 1461 H= 365:15 +CPU= 25.971
  09:15:58 STEP 1462 H= 365:30 +CPU= 27.993
  09:16:26 STEP 1463 H= 365:45 +CPU= 27.355
  09:16:57 STEP 1464 H= 366:00 +CPU= 31.194
Moving to projects directory: /boinc_data/boinc/slots/7/ICMGGh7zg+001464
Moving to projects directory: /boinc_data/boinc/slots/7/ICMSHh7zg+001464
Moving to projects directory: /boinc_data/boinc/slots/7/ICMUAh7zg+001464
  09:17:23 STEP 1465 H= 366:15 +CPU= 26.179
  09:17:48 STEP 1466 H= 366:30 +CPU= 25.055
  09:18:16 STEP 1467 H= 366:45 +CPU= 26.997
  09:18:42 STEP 1468 H= 367:00 +CPU= 26.579
  09:19:06 STEP 1469 H= 367:15 +CPU= 23.627
  09:19:33 STEP 1470 H= 367:30 +CPU= 27.437
  09:20:01 STEP 1471 H= 367:45 +CPU= 27.498
  09:20:30 STEP 1472 H= 368:00 +CPU= 28.675
  09:20:56 STEP 1473 H= 368:15 +CPU= 26.656
  09:21:23 STEP 1474 H= 368:30 +CPU= 26.119
  09:21:48 STEP 1475 H= 368:45 +CPU= 25.202
  09:22:21 STEP 1476 H= 369:00 +CPU= 32.850
Moving to projects directory: /boinc_data/boinc/slots/7/ICMGGh7zg+001476
Moving to projects directory: /boinc_data/boinc/slots/7/ICMSHh7zg+001476
Moving to projects directory: /boinc_data/boinc/slots/7/ICMUAh7zg+001476
  09:22:47 STEP 1477 H= 369:15 +CPU= 26.592
  09:23:13 STEP 1478 H= 369:30 +CPU= 25.504
  09:23:38 STEP 1479 H= 369:45 +CPU= 25.101
  09:24:05 STEP 1480 H= 370:00 +CPU= 27.226
  09:24:34 STEP 1481 H= 370:15 +CPU= 28.077
  09:24:59 STEP 1482 H= 370:30 +CPU= 25.256
  09:25:25 STEP 1483 H= 370:45 +CPU= 26.311
  09:25:52 STEP 1484 H= 371:00 +CPU= 26.850
  09:26:17 STEP 1485 H= 371:15 +CPU= 25.003
  09:26:43 STEP 1486 H= 371:30 +CPU= 25.562
  09:27:09 STEP 1487 H= 371:45 +CPU= 26.062
  09:27:48 STEP 1488 H= 372:00 +CPU= 38.858
Moving to projects directory: /boinc_data/boinc/slots/7/ICMGGh7zg+001488
Moving to projects directory: /boinc_data/boinc/slots/7/ICMSHh7zg+001488
Moving to projects directory: /boinc_data/boinc/slots/7/ICMUAh7zg+001488
  09:28:15 STEP 1489 H= 372:15 +CPU= 27.130
  09:28:41 STEP 1490 H= 372:30 +CPU= 25.442
  09:29:08 STEP 1491 H= 372:45 +CPU= 27.273
  09:29:35 STEP 1492 H= 373:00 +CPU= 27.407
  09:30:01 STEP 1493 H= 373:15 +CPU= 25.805
  09:30:26 STEP 1494 H= 373:30 +CPU= 24.490
  09:30:52 STEP 1495 H= 373:45 +CPU= 26.331
  09:31:19 STEP 1496 H= 374:00 +CPU= 26.809
  09:31:43 STEP 1497 H= 374:15 +CPU= 24.210
  09:32:11 STEP 1498 H= 374:30 +CPU= 27.422
  09:32:37 STEP 1499 H= 374:45 +CPU= 25.831
  09:33:08 STEP 1500 H= 375:00 +CPU= 30.797
Moving to projects directory: /boinc_data/boinc/slots/7/ICMGGh7zg+001500
Moving to projects directory: /boinc_data/boinc/slots/7/ICMSHh7zg+001500
Moving to projects directory: /boinc_data/boinc/slots/7/ICMUAh7zg+001500
  09:33:36 STEP 1501 H= 375:15 +CPU= 27.939
  09:34:02 STEP 1502 H= 375:30 +CPU= 26.468
  09:34:31 STEP 1503 H= 375:45 +CPU= 28.272
  09:34:57 STEP 1504 H= 376:00 +CPU= 26.149
  09:35:24 STEP 1505 H= 376:15 +CPU= 26.754
  09:35:50 STEP 1506 H= 376:30 +CPU= 26.599
  09:36:16 STEP 1507 H= 376:45 +CPU= 25.784
  09:36:42 STEP 1508 H= 377:00 +CPU= 26.119
  09:37:10 STEP 1509 H= 377:15 +CPU= 27.441
  09:37:32 STEP 1510 H= 377:30 +CPU= 22.755
  09:37:58 STEP 1511 H= 377:45 +CPU= 25.569
  09:38:33 STEP 1512 H= 378:00 +CPU= 34.736
Moving to projects directory: /boinc_data/boinc/slots/7/ICMGGh7zg+001512
Moving to projects directory: /boinc_data/boinc/slots/7/ICMSHh7zg+001512
Moving to projects directory: /boinc_data/boinc/slots/7/ICMUAh7zg+001512
  09:38:59 STEP 1513 H= 378:15 +CPU= 26.076
  09:39:22 STEP 1514 H= 378:30 +CPU= 22.769
  09:39:47 STEP 1515 H= 378:45 +CPU= 25.057
  09:40:14 STEP 1516 H= 379:00 +CPU= 27.049
  09:40:42 STEP 1517 H= 379:15 +CPU= 27.577
  09:41:10 STEP 1518 H= 379:30 +CPU= 28.238
  09:41:34 STEP 1519 H= 379:45 +CPU= 23.948
  09:42:02 STEP 1520 H= 380:00 +CPU= 27.459
  09:42:29 STEP 1521 H= 380:15 +CPU= 27.174
  09:42:56 STEP 1522 H= 380:30 +CPU= 27.492
  09:43:24 STEP 1523 H= 380:45 +CPU= 27.624
  09:43:59 STEP 1524 H= 381:00 +CPU= 34.961
Moving to projects directory: /boinc_data/boinc/slots/7/ICMGGh7zg+001524
Moving to projects directory: /boinc_data/boinc/slots/7/ICMSHh7zg+001524
Moving to projects directory: /boinc_data/boinc/slots/7/ICMUAh7zg+001524
  09:44:30 STEP 1525 H= 381:15 +CPU= 30.753
  09:44:55 STEP 1526 H= 381:30 +CPU= 24.767
  09:45:21 STEP 1527 H= 381:45 +CPU= 25.794
  09:45:48 STEP 1528 H= 382:00 +CPU= 26.828
  09:46:15 STEP 1529 H= 382:15 +CPU= 27.160
  09:46:42 STEP 1530 H= 382:30 +CPU= 26.840
  09:47:08 STEP 1531 H= 382:45 +CPU= 25.751
  09:47:38 STEP 1532 H= 383:00 +CPU= 29.968
  09:48:04 STEP 1533 H= 383:15 +CPU= 26.643
  09:48:29 STEP 1534 H= 383:30 +CPU= 25.022
  09:48:56 STEP 1535 H= 383:45 +CPU= 26.566
  09:49:32 STEP 1536 H= 384:00 +CPU= 36.376
Moving to projects directory: /boinc_data/boinc/slots/7/ICMGGh7zg+001536
Moving to projects directory: /boinc_data/boinc/slots/7/ICMSHh7zg+001536
Moving to projects directory: /boinc_data/boinc/slots/7/ICMUAh7zg+001536
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMGGh7zg+001440
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMSHh7zg+001440
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMUAh7zg+001440
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMGGh7zg+001452
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMSHh7zg+001452
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMUAh7zg+001452
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMGGh7zg+001464
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMSHh7zg+001464
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMUAh7zg+001464
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMGGh7zg+001476
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMSHh7zg+001476
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMUAh7zg+001476
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMGGh7zg+001488
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMSHh7zg+001488
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMUAh7zg+001488
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMGGh7zg+001500
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMSHh7zg+001500
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMUAh7zg+001500
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMGGh7zg+001512
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMSHh7zg+001512
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMUAh7zg+001512
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMGGh7zg+001524
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMSHh7zg+001524
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMUAh7zg+001524
Zipping up the intermediate file: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_a4go_2016092300_20_1018_12294111_0_r1581845035_15.zip
Uploading the intermediate file: upload_file_15.zip
  09:49:58 STEP 1537 H= 384:15 +CPU= 25.401
  09:50:27 STEP 1538 H= 384:30 +CPU= 29.419
  09:50:53 STEP 1539 H= 384:45 +CPU= 25.497
  09:51:21 STEP 1540 H= 385:00 +CPU= 27.839
  09:51:48 STEP 1541 H= 385:15 +CPU= 27.212
  09:52:15 STEP 1542 H= 385:30 +CPU= 26.531
  09:52:40 STEP 1543 H= 385:45 +CPU= 25.117
  09:53:07 STEP 1544 H= 386:00 +CPU= 27.019
  09:53:32 STEP 1545 H= 386:15 +CPU= 24.820
  09:54:01 STEP 1546 H= 386:30 +CPU= 29.080
  09:54:28 STEP 1547 H= 386:45 +CPU= 26.524
  09:55:02 STEP 1548 H= 387:00 +CPU= 34.728
Moving to projects directory: /boinc_data/boinc/slots/7/ICMGGh7zg+001548
Moving to projects directory: /boinc_data/boinc/slots/7/ICMSHh7zg+001548
Moving to projects directory: /boinc_data/boinc/slots/7/ICMUAh7zg+001548
  09:55:30 STEP 1549 H= 387:15 +CPU= 27.204
  09:55:55 STEP 1550 H= 387:30 +CPU= 25.094
  09:56:23 STEP 1551 H= 387:45 +CPU= 27.643
  09:56:47 STEP 1552 H= 388:00 +CPU= 24.344
  09:57:12 STEP 1553 H= 388:15 +CPU= 24.973
  09:57:37 STEP 1554 H= 388:30 +CPU= 24.815
  09:58:05 STEP 1555 H= 388:45 +CPU= 27.851
  09:58:31 STEP 1556 H= 389:00 +CPU= 26.722
  09:58:58 STEP 1557 H= 389:15 +CPU= 26.074
  09:59:22 STEP 1558 H= 389:30 +CPU= 24.670
  09:59:49 STEP 1559 H= 389:45 +CPU= 26.157
  10:00:21 STEP 1560 H= 390:00 +CPU= 32.442
Moving to projects directory: /boinc_data/boinc/slots/7/ICMGGh7zg+001560
Moving to projects directory: /boinc_data/boinc/slots/7/ICMSHh7zg+001560
Moving to projects directory: /boinc_data/boinc/slots/7/ICMUAh7zg+001560
  10:00:46 STEP 1561 H= 390:15 +CPU= 24.611
  10:01:10 STEP 1562 H= 390:30 +CPU= 24.536
  10:01:37 STEP 1563 H= 390:45 +CPU= 26.192
  10:02:04 STEP 1564 H= 391:00 +CPU= 27.131
  10:02:30 STEP 1565 H= 391:15 +CPU= 26.730
  10:02:59 STEP 1566 H= 391:30 +CPU= 28.745
  10:03:26 STEP 1567 H= 391:45 +CPU= 26.225
  10:03:53 STEP 1568 H= 392:00 +CPU= 26.966
  10:04:20 STEP 1569 H= 392:15 +CPU= 27.064
  10:04:46 STEP 1570 H= 392:30 +CPU= 26.083
  10:05:13 STEP 1571 H= 392:45 +CPU= 26.662
  10:05:45 STEP 1572 H= 393:00 +CPU= 32.701
Moving to projects directory: /boinc_data/boinc/slots/7/ICMGGh7zg+001572
Moving to projects directory: /boinc_data/boinc/slots/7/ICMSHh7zg+001572
Moving to projects directory: /boinc_data/boinc/slots/7/ICMUAh7zg+001572
  10:06:12 STEP 1573 H= 393:15 +CPU= 26.463
  10:06:36 STEP 1574 H= 393:30 +CPU= 23.908
  10:07:03 STEP 1575 H= 393:45 +CPU= 27.131
  10:07:29 STEP 1576 H= 394:00 +CPU= 25.736
  10:07:55 STEP 1577 H= 394:15 +CPU= 25.618
  10:08:22 STEP 1578 H= 394:30 +CPU= 26.924
  10:08:48 STEP 1579 H= 394:45 +CPU= 26.049
  10:09:14 STEP 1580 H= 395:00 +CPU= 26.334
  10:09:40 STEP 1581 H= 395:15 +CPU= 25.488
  10:10:06 STEP 1582 H= 395:30 +CPU= 26.256
  10:10:31 STEP 1583 H= 395:45 +CPU= 25.198
  10:11:06 STEP 1584 H= 396:00 +CPU= 34.680
Moving to projects directory: /boinc_data/boinc/slots/7/ICMGGh7zg+001584
Moving to projects directory: /boinc_data/boinc/slots/7/ICMSHh7zg+001584
Moving to projects directory: /boinc_data/boinc/slots/7/ICMUAh7zg+001584
  10:11:32 STEP 1585 H= 396:15 +CPU= 26.193
  10:11:59 STEP 1586 H= 396:30 +CPU= 26.599
  10:12:26 STEP 1587 H= 396:45 +CPU= 27.371
  10:12:53 STEP 1588 H= 397:00 +CPU= 26.438
  10:13:18 STEP 1589 H= 397:15 +CPU= 25.454
  10:13:45 STEP 1590 H= 397:30 +CPU= 27.044
  10:14:11 STEP 1591 H= 397:45 +CPU= 25.547
  10:14:37 STEP 1592 H= 398:00 +CPU= 26.196
  10:15:01 STEP 1593 H= 398:15 +CPU= 23.762
  10:15:29 STEP 1594 H= 398:30 +CPU= 28.251
  10:15:55 STEP 1595 H= 398:45 +CPU= 26.020
  10:16:28 STEP 1596 H= 399:00 +CPU= 32.498
Moving to projects directory: /boinc_data/boinc/slots/7/ICMGGh7zg+001596
Moving to projects directory: /boinc_data/boinc/slots/7/ICMSHh7zg+001596
Moving to projects directory: /boinc_data/boinc/slots/7/ICMUAh7zg+001596
  10:16:54 STEP 1597 H= 399:15 +CPU= 25.711
  10:17:20 STEP 1598 H= 399:30 +CPU= 25.984
  10:17:44 STEP 1599 H= 399:45 +CPU= 24.111
  10:18:11 STEP 1600 H= 400:00 +CPU= 27.261
  10:18:37 STEP 1601 H= 400:15 +CPU= 25.183
  10:19:03 STEP 1602 H= 400:30 +CPU= 25.917
  10:19:28 STEP 1603 H= 400:45 +CPU= 25.754
  10:19:56 STEP 1604 H= 401:00 +CPU= 27.351
  10:20:21 STEP 1605 H= 401:15 +CPU= 25.230
  10:20:47 STEP 1606 H= 401:30 +CPU= 25.911
  10:21:13 STEP 1607 H= 401:45 +CPU= 25.522
  10:21:45 STEP 1608 H= 402:00 +CPU= 32.146
Moving to projects directory: /boinc_data/boinc/slots/7/ICMGGh7zg+001608
Moving to projects directory: /boinc_data/boinc/slots/7/ICMSHh7zg+001608
Moving to projects directory: /boinc_data/boinc/slots/7/ICMUAh7zg+001608
  10:22:11 STEP 1609 H= 402:15 +CPU= 26.146
  10:22:38 STEP 1610 H= 402:30 +CPU= 26.561
  10:23:05 STEP 1611 H= 402:45 +CPU= 26.984
  10:23:31 STEP 1612 H= 403:00 +CPU= 26.483
  10:23:58 STEP 1613 H= 403:15 +CPU= 26.191
  10:24:24 STEP 1614 H= 403:30 +CPU= 26.114
  10:24:51 STEP 1615 H= 403:45 +CPU= 26.854
  10:25:17 STEP 1616 H= 404:00 +CPU= 26.471
  10:25:44 STEP 1617 H= 404:15 +CPU= 26.260
  10:26:10 STEP 1618 H= 404:30 +CPU= 26.822
  10:26:34 STEP 1619 H= 404:45 +CPU= 23.909
  10:27:08 STEP 1620 H= 405:00 +CPU= 33.755
Moving to projects directory: /boinc_data/boinc/slots/7/ICMGGh7zg+001620
Moving to projects directory: /boinc_data/boinc/slots/7/ICMSHh7zg+001620
Moving to projects directory: /boinc_data/boinc/slots/7/ICMUAh7zg+001620
  10:27:33 STEP 1621 H= 405:15 +CPU= 24.479
  10:27:59 STEP 1622 H= 405:30 +CPU= 26.367
  10:28:25 STEP 1623 H= 405:45 +CPU= 25.317
  10:28:52 STEP 1624 H= 406:00 +CPU= 26.980
  10:29:20 STEP 1625 H= 406:15 +CPU= 27.834
  10:29:45 STEP 1626 H= 406:30 +CPU= 25.087
  10:30:13 STEP 1627 H= 406:45 +CPU= 27.973
  10:30:41 STEP 1628 H= 407:00 +CPU= 28.179
  10:31:09 STEP 1629 H= 407:15 +CPU= 27.150
  10:31:33 STEP 1630 H= 407:30 +CPU= 24.683
  10:31:58 STEP 1631 H= 407:45 +CPU= 25.156
  10:32:35 STEP 1632 H= 408:00 +CPU= 36.607
Moving to projects directory: /boinc_data/boinc/slots/7/ICMGGh7zg+001632
Moving to projects directory: /boinc_data/boinc/slots/7/ICMSHh7zg+001632
Moving to projects directory: /boinc_data/boinc/slots/7/ICMUAh7zg+001632
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMGGh7zg+001536
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMSHh7zg+001536
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMUAh7zg+001536
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMGGh7zg+001548
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMSHh7zg+001548
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMUAh7zg+001548
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMGGh7zg+001560
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMSHh7zg+001560
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMUAh7zg+001560
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMGGh7zg+001572
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMSHh7zg+001572
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMUAh7zg+001572
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMGGh7zg+001584
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMSHh7zg+001584
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMUAh7zg+001584
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMGGh7zg+001596
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMSHh7zg+001596
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMUAh7zg+001596
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMGGh7zg+001608
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMSHh7zg+001608
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMUAh7zg+001608
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMGGh7zg+001620
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMSHh7zg+001620
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMUAh7zg+001620
Zipping up the intermediate file: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_a4go_2016092300_20_1018_12294111_0_r1581845035_16.zip
Uploading the intermediate file: upload_file_16.zip
  10:33:02 STEP 1633 H= 408:15 +CPU= 26.542
  10:33:27 STEP 1634 H= 408:30 +CPU= 25.358
  10:33:53 STEP 1635 H= 408:45 +CPU= 26.153
  10:34:22 STEP 1636 H= 409:00 +CPU= 28.289
  10:34:49 STEP 1637 H= 409:15 +CPU= 27.057
  10:35:15 STEP 1638 H= 409:30 +CPU= 25.637
  10:35:43 STEP 1639 H= 409:45 +CPU= 28.495
  10:36:11 STEP 1640 H= 410:00 +CPU= 27.285
  10:36:38 STEP 1641 H= 410:15 +CPU= 27.366
  10:37:03 STEP 1642 H= 410:30 +CPU= 24.295
  10:37:28 STEP 1643 H= 410:45 +CPU= 25.676
  10:38:01 STEP 1644 H= 411:00 +CPU= 32.787
Moving to projects directory: /boinc_data/boinc/slots/7/ICMGGh7zg+001644
Moving to projects directory: /boinc_data/boinc/slots/7/ICMSHh7zg+001644
Moving to projects directory: /boinc_data/boinc/slots/7/ICMUAh7zg+001644
  10:38:25 STEP 1645 H= 411:15 +CPU= 23.413
  10:38:51 STEP 1646 H= 411:30 +CPU= 26.467
  10:39:18 STEP 1647 H= 411:45 +CPU= 26.533
  10:39:45 STEP 1648 H= 412:00 +CPU= 26.937
  10:40:12 STEP 1649 H= 412:15 +CPU= 26.577
  10:40:40 STEP 1650 H= 412:30 +CPU= 27.608
  10:41:07 STEP 1651 H= 412:45 +CPU= 27.160
  10:41:33 STEP 1652 H= 413:00 +CPU= 25.847
  10:42:00 STEP 1653 H= 413:15 +CPU= 26.909
  10:42:29 STEP 1654 H= 413:30 +CPU= 28.063
  10:42:57 STEP 1655 H= 413:45 +CPU= 28.163
  10:43:29 STEP 1656 H= 414:00 +CPU= 32.323
Moving to projects directory: /boinc_data/boinc/slots/7/ICMGGh7zg+001656
Moving to projects directory: /boinc_data/boinc/slots/7/ICMSHh7zg+001656
Moving to projects directory: /boinc_data/boinc/slots/7/ICMUAh7zg+001656
  10:43:55 STEP 1657 H= 414:15 +CPU= 25.505
  10:44:19 STEP 1658 H= 414:30 +CPU= 24.763
  10:44:44 STEP 1659 H= 414:45 +CPU= 24.352
  10:45:09 STEP 1660 H= 415:00 +CPU= 25.479
  10:45:37 STEP 1661 H= 415:15 +CPU= 27.319
  10:46:03 STEP 1662 H= 415:30 +CPU= 25.948
  10:46:29 STEP 1663 H= 415:45 +CPU= 26.350
  10:46:55 STEP 1664 H= 416:00 +CPU= 25.350
  10:47:19 STEP 1665 H= 416:15 +CPU= 24.781
  10:47:45 STEP 1666 H= 416:30 +CPU= 25.564
  10:48:07 STEP 1667 H= 416:45 +CPU= 22.278
  10:48:39 STEP 1668 H= 417:00 +CPU= 31.440
Moving to projects directory: /boinc_data/boinc/slots/7/ICMGGh7zg+001668
Moving to projects directory: /boinc_data/boinc/slots/7/ICMSHh7zg+001668
Moving to projects directory: /boinc_data/boinc/slots/7/ICMUAh7zg+001668
  10:49:03 STEP 1669 H= 417:15 +CPU= 24.541
  10:49:31 STEP 1670 H= 417:30 +CPU= 27.082
  10:49:59 STEP 1671 H= 417:45 +CPU= 28.366
  10:50:27 STEP 1672 H= 418:00 +CPU= 27.613
  10:50:55 STEP 1673 H= 418:15 +CPU= 27.166
  10:51:24 STEP 1674 H= 418:30 +CPU= 28.925
  10:51:53 STEP 1675 H= 418:45 +CPU= 29.164
  10:52:23 STEP 1676 H= 419:00 +CPU= 29.081
  10:52:51 STEP 1677 H= 419:15 +CPU= 27.832
  10:53:20 STEP 1678 H= 419:30 +CPU= 28.332
  10:53:49 STEP 1679 H= 419:45 +CPU= 28.100
  10:54:29 STEP 1680 H= 420:00 +CPU= 39.895
Moving to projects directory: /boinc_data/boinc/slots/7/ICMGGh7zg+001680
Moving to projects directory: /boinc_data/boinc/slots/7/ICMSHh7zg+001680
Moving to projects directory: /boinc_data/boinc/slots/7/ICMUAh7zg+001680
  10:54:58 STEP 1681 H= 420:15 +CPU= 27.761
  10:55:25 STEP 1682 H= 420:30 +CPU= 26.478
  10:55:52 STEP 1683 H= 420:45 +CPU= 26.996
  10:56:18 STEP 1684 H= 421:00 +CPU= 26.074
  10:56:46 STEP 1685 H= 421:15 +CPU= 27.243
  10:57:12 STEP 1686 H= 421:30 +CPU= 26.777
  10:57:39 STEP 1687 H= 421:45 +CPU= 26.100
  10:58:07 STEP 1688 H= 422:00 +CPU= 28.719
  10:58:35 STEP 1689 H= 422:15 +CPU= 27.144
  10:59:00 STEP 1690 H= 422:30 +CPU= 25.400
  10:59:26 STEP 1691 H= 422:45 +CPU= 25.541
  10:59:51 STEP 1692 H= 423:00 +CPU= 25.281
Moving to projects directory: /boinc_data/boinc/slots/7/ICMGGh7zg+001692
Moving to projects directory: /boinc_data/boinc/slots/7/ICMSHh7zg+001692
Moving to projects directory: /boinc_data/boinc/slots/7/ICMUAh7zg+001692
  11:00:12 STEP 1693 H= 423:15 +CPU= 20.216
  11:00:41 STEP 1694 H= 423:30 +CPU= 29.468
  11:01:12 STEP 1695 H= 423:45 +CPU= 29.967
  11:01:41 STEP 1696 H= 424:00 +CPU= 28.787
  11:02:09 STEP 1697 H= 424:15 +CPU= 27.693
  11:02:37 STEP 1698 H= 424:30 +CPU= 27.446
  11:03:05 STEP 1699 H= 424:45 +CPU= 27.672
  11:03:35 STEP 1700 H= 425:00 +CPU= 29.369
  11:04:07 STEP 1701 H= 425:15 +CPU= 31.673
  11:04:36 STEP 1702 H= 425:30 +CPU= 29.021
  11:05:03 STEP 1703 H= 425:45 +CPU= 27.049
  11:05:40 STEP 1704 H= 426:00 +CPU= 35.838
Moving to projects directory: /boinc_data/boinc/slots/7/ICMGGh7zg+001704
Moving to projects directory: /boinc_data/boinc/slots/7/ICMSHh7zg+001704
Moving to projects directory: /boinc_data/boinc/slots/7/ICMUAh7zg+001704
  11:06:04 STEP 1705 H= 426:15 +CPU= 24.480
  11:06:31 STEP 1706 H= 426:30 +CPU= 26.586
  11:06:57 STEP 1707 H= 426:45 +CPU= 26.271
  11:07:25 STEP 1708 H= 427:00 +CPU= 27.557
  11:07:55 STEP 1709 H= 427:15 +CPU= 28.862
  11:08:23 STEP 1710 H= 427:30 +CPU= 28.095
  11:08:50 STEP 1711 H= 427:45 +CPU= 27.440
  11:09:19 STEP 1712 H= 428:00 +CPU= 28.189
  11:09:44 STEP 1713 H= 428:15 +CPU= 25.540
  11:10:11 STEP 1714 H= 428:30 +CPU= 26.227
  11:10:36 STEP 1715 H= 428:45 +CPU= 24.941
  11:11:10 STEP 1716 H= 429:00 +CPU= 33.882
Moving to projects directory: /boinc_data/boinc/slots/7/ICMGGh7zg+001716
Moving to projects directory: /boinc_data/boinc/slots/7/ICMSHh7zg+001716
Moving to projects directory: /boinc_data/boinc/slots/7/ICMUAh7zg+001716
  11:11:37 STEP 1717 H= 429:15 +CPU= 26.905
  11:12:04 STEP 1718 H= 429:30 +CPU= 27.254
  11:12:30 STEP 1719 H= 429:45 +CPU= 25.828
  11:12:56 STEP 1720 H= 430:00 +CPU= 25.618
  11:13:23 STEP 1721 H= 430:15 +CPU= 27.785
  11:13:49 STEP 1722 H= 430:30 +CPU= 25.077
  11:14:15 STEP 1723 H= 430:45 +CPU= 26.004
  11:14:41 STEP 1724 H= 431:00 +CPU= 26.274
  11:15:09 STEP 1725 H= 431:15 +CPU= 28.041
  11:15:36 STEP 1726 H= 431:30 +CPU= 26.589
  11:16:01 STEP 1727 H= 431:45 +CPU= 25.386
  11:16:40 STEP 1728 H= 432:00 +CPU= 38.832
Moving to projects directory: /boinc_data/boinc/slots/7/ICMGGh7zg+001728
Moving to projects directory: /boinc_data/boinc/slots/7/ICMSHh7zg+001728
Moving to projects directory: /boinc_data/boinc/slots/7/ICMUAh7zg+001728
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMGGh7zg+001632
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMSHh7zg+001632
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMUAh7zg+001632
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMGGh7zg+001644
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMSHh7zg+001644
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMUAh7zg+001644
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMGGh7zg+001656
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMSHh7zg+001656
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMUAh7zg+001656
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMGGh7zg+001668
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMSHh7zg+001668
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMUAh7zg+001668
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMGGh7zg+001680
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMSHh7zg+001680
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMUAh7zg+001680
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMGGh7zg+001692
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMSHh7zg+001692
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMUAh7zg+001692
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMGGh7zg+001704
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMSHh7zg+001704
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMUAh7zg+001704
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMGGh7zg+001716
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMSHh7zg+001716
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMUAh7zg+001716
Zipping up the intermediate file: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_a4go_2016092300_20_1018_12294111_0_r1581845035_17.zip
Uploading the intermediate file: upload_file_17.zip
  11:17:07 STEP 1729 H= 432:15 +CPU= 27.085
  11:17:36 STEP 1730 H= 432:30 +CPU= 28.139
  11:18:05 STEP 1731 H= 432:45 +CPU= 28.726
  11:18:34 STEP 1732 H= 433:00 +CPU= 28.464
  11:19:02 STEP 1733 H= 433:15 +CPU= 27.765
  11:19:30 STEP 1734 H= 433:30 +CPU= 27.794
  11:19:58 STEP 1735 H= 433:45 +CPU= 27.806
  11:20:19 STEP 1736 H= 434:00 +CPU= 20.626
  11:20:37 STEP 1737 H= 434:15 +CPU= 18.263
  11:20:57 STEP 1738 H= 434:30 +CPU= 20.057
  11:21:24 STEP 1739 H= 434:45 +CPU= 26.357
  11:21:58 STEP 1740 H= 435:00 +CPU= 34.050
Moving to projects directory: /boinc_data/boinc/slots/7/ICMGGh7zg+001740
Moving to projects directory: /boinc_data/boinc/slots/7/ICMSHh7zg+001740
Moving to projects directory: /boinc_data/boinc/slots/7/ICMUAh7zg+001740
  11:22:26 STEP 1741 H= 435:15 +CPU= 27.875
  11:22:52 STEP 1742 H= 435:30 +CPU= 26.442
  11:23:20 STEP 1743 H= 435:45 +CPU= 26.937
  11:23:45 STEP 1744 H= 436:00 +CPU= 25.527
  11:24:12 STEP 1745 H= 436:15 +CPU= 26.682
  11:24:40 STEP 1746 H= 436:30 +CPU= 27.670
  11:25:07 STEP 1747 H= 436:45 +CPU= 27.307
  11:25:26 STEP 1748 H= 437:00 +CPU= 18.913
  11:25:46 STEP 1749 H= 437:15 +CPU= 19.890
  11:26:13 STEP 1750 H= 437:30 +CPU= 26.039
  11:26:39 STEP 1751 H= 437:45 +CPU= 25.928
  11:27:12 STEP 1752 H= 438:00 +CPU= 33.046
Moving to projects directory: /boinc_data/boinc/slots/7/ICMGGh7zg+001752
Moving to projects directory: /boinc_data/boinc/slots/7/ICMSHh7zg+001752
Moving to projects directory: /boinc_data/boinc/slots/7/ICMUAh7zg+001752
  11:27:38 STEP 1753 H= 438:15 +CPU= 26.041
  11:28:06 STEP 1754 H= 438:30 +CPU= 27.704
  11:28:33 STEP 1755 H= 438:45 +CPU= 27.467
  11:29:01 STEP 1756 H= 439:00 +CPU= 27.621
  11:29:29 STEP 1757 H= 439:15 +CPU= 27.410
  11:29:56 STEP 1758 H= 439:30 +CPU= 26.925
  11:30:23 STEP 1759 H= 439:45 +CPU= 27.056
  11:30:52 STEP 1760 H= 440:00 +CPU= 28.945
  11:31:19 STEP 1761 H= 440:15 +CPU= 26.578
  11:31:47 STEP 1762 H= 440:30 +CPU= 27.777
  11:32:14 STEP 1763 H= 440:45 +CPU= 27.020
  11:32:48 STEP 1764 H= 441:00 +CPU= 33.879
Moving to projects directory: /boinc_data/boinc/slots/7/ICMGGh7zg+001764
Moving to projects directory: /boinc_data/boinc/slots/7/ICMSHh7zg+001764
Moving to projects directory: /boinc_data/boinc/slots/7/ICMUAh7zg+001764
  11:33:13 STEP 1765 H= 441:15 +CPU= 25.731
  11:33:39 STEP 1766 H= 441:30 +CPU= 25.898
  11:34:07 STEP 1767 H= 441:45 +CPU= 27.768
  11:34:34 STEP 1768 H= 442:00 +CPU= 26.712
  11:35:00 STEP 1769 H= 442:15 +CPU= 26.077
  11:35:25 STEP 1770 H= 442:30 +CPU= 24.572
  11:35:50 STEP 1771 H= 442:45 +CPU= 24.836
  11:36:16 STEP 1772 H= 443:00 +CPU= 26.264
  11:36:39 STEP 1773 H= 443:15 +CPU= 23.150
  11:36:56 STEP 1774 H= 443:30 +CPU= 17.319
  11:37:14 STEP 1775 H= 443:45 +CPU= 17.935
  11:37:45 STEP 1776 H= 444:00 +CPU= 30.731
Moving to projects directory: /boinc_data/boinc/slots/7/ICMGGh7zg+001776
Moving to projects directory: /boinc_data/boinc/slots/7/ICMSHh7zg+001776
Moving to projects directory: /boinc_data/boinc/slots/7/ICMUAh7zg+001776
  11:38:09 STEP 1777 H= 444:15 +CPU= 24.092
  11:38:37 STEP 1778 H= 444:30 +CPU= 27.674
  11:39:02 STEP 1779 H= 444:45 +CPU= 24.567
  11:39:28 STEP 1780 H= 445:00 +CPU= 26.193
  11:39:53 STEP 1781 H= 445:15 +CPU= 25.286
  11:40:19 STEP 1782 H= 445:30 +CPU= 25.651
  11:40:46 STEP 1783 H= 445:45 +CPU= 26.814
  11:41:13 STEP 1784 H= 446:00 +CPU= 26.752
  11:41:38 STEP 1785 H= 446:15 +CPU= 25.219
  11:42:03 STEP 1786 H= 446:30 +CPU= 24.999
  11:42:29 STEP 1787 H= 446:45 +CPU= 26.333
  11:43:04 STEP 1788 H= 447:00 +CPU= 34.953
Moving to projects directory: /boinc_data/boinc/slots/7/ICMGGh7zg+001788
Moving to projects directory: /boinc_data/boinc/slots/7/ICMSHh7zg+001788
Moving to projects directory: /boinc_data/boinc/slots/7/ICMUAh7zg+001788
  11:43:30 STEP 1789 H= 447:15 +CPU= 25.888
  11:43:57 STEP 1790 H= 447:30 +CPU= 26.393
  11:44:24 STEP 1791 H= 447:45 +CPU= 26.935
  11:44:50 STEP 1792 H= 448:00 +CPU= 26.085
  11:45:15 STEP 1793 H= 448:15 +CPU= 25.392
  11:45:41 STEP 1794 H= 448:30 +CPU= 25.648
  11:46:07 STEP 1795 H= 448:45 +CPU= 26.155
  11:46:33 STEP 1796 H= 449:00 +CPU= 25.427
  11:47:00 STEP 1797 H= 449:15 +CPU= 27.164
  11:47:26 STEP 1798 H= 449:30 +CPU= 25.957
  11:47:52 STEP 1799 H= 449:45 +CPU= 25.443
  11:48:25 STEP 1800 H= 450:00 +CPU= 33.083
Moving to projects directory: /boinc_data/boinc/slots/7/ICMGGh7zg+001800
Moving to projects directory: /boinc_data/boinc/slots/7/ICMSHh7zg+001800
Moving to projects directory: /boinc_data/boinc/slots/7/ICMUAh7zg+001800
  11:48:49 STEP 1801 H= 450:15 +CPU= 24.604
  11:49:15 STEP 1802 H= 450:30 +CPU= 25.856
  11:49:41 STEP 1803 H= 450:45 +CPU= 25.819
  11:50:09 STEP 1804 H= 451:00 +CPU= 27.449
  11:50:35 STEP 1805 H= 451:15 +CPU= 26.383
  11:51:05 STEP 1806 H= 451:30 +CPU= 29.657
  11:51:35 STEP 1807 H= 451:45 +CPU= 29.859
  11:52:01 STEP 1808 H= 452:00 +CPU= 26.198
  11:52:28 STEP 1809 H= 452:15 +CPU= 26.926
  11:52:54 STEP 1810 H= 452:30 +CPU= 25.434
  11:53:20 STEP 1811 H= 452:45 +CPU= 26.301
  11:53:52 STEP 1812 H= 453:00 +CPU= 31.938
Moving to projects directory: /boinc_data/boinc/slots/7/ICMGGh7zg+001812
Moving to projects directory: /boinc_data/boinc/slots/7/ICMSHh7zg+001812
Moving to projects directory: /boinc_data/boinc/slots/7/ICMUAh7zg+001812
  11:54:20 STEP 1813 H= 453:15 +CPU= 27.556
  11:54:46 STEP 1814 H= 453:30 +CPU= 25.841
  11:55:12 STEP 1815 H= 453:45 +CPU= 26.834
  11:55:39 STEP 1816 H= 454:00 +CPU= 26.130
  11:56:05 STEP 1817 H= 454:15 +CPU= 26.220
  11:56:31 STEP 1818 H= 454:30 +CPU= 25.846
  11:56:59 STEP 1819 H= 454:45 +CPU= 27.792
  11:57:23 STEP 1820 H= 455:00 +CPU= 24.761
  11:57:48 STEP 1821 H= 455:15 +CPU= 24.752
  11:58:14 STEP 1822 H= 455:30 +CPU= 25.889
  11:58:40 STEP 1823 H= 455:45 +CPU= 25.907
  11:59:18 STEP 1824 H= 456:00 +CPU= 37.661
Moving to projects directory: /boinc_data/boinc/slots/7/ICMGGh7zg+001824
Moving to projects directory: /boinc_data/boinc/slots/7/ICMSHh7zg+001824
Moving to projects directory: /boinc_data/boinc/slots/7/ICMUAh7zg+001824
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMGGh7zg+001728
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMSHh7zg+001728
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMUAh7zg+001728
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMGGh7zg+001740
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMSHh7zg+001740
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMUAh7zg+001740
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMGGh7zg+001752
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMSHh7zg+001752
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMUAh7zg+001752
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMGGh7zg+001764
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMSHh7zg+001764
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMUAh7zg+001764
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMGGh7zg+001776
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMSHh7zg+001776
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMUAh7zg+001776
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMGGh7zg+001788
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMSHh7zg+001788
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMUAh7zg+001788
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMGGh7zg+001800
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMSHh7zg+001800
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMUAh7zg+001800
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMGGh7zg+001812
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMSHh7zg+001812
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMUAh7zg+001812
Zipping up the intermediate file: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_a4go_2016092300_20_1018_12294111_0_r1581845035_18.zip
Uploading the intermediate file: upload_file_18.zip
  11:59:44 STEP 1825 H= 456:15 +CPU= 26.263
  12:00:10 STEP 1826 H= 456:30 +CPU= 25.936
  12:00:36 STEP 1827 H= 456:45 +CPU= 25.867
  12:01:03 STEP 1828 H= 457:00 +CPU= 26.933
  12:01:29 STEP 1829 H= 457:15 +CPU= 25.913
  12:01:56 STEP 1830 H= 457:30 +CPU= 26.661
  12:02:22 STEP 1831 H= 457:45 +CPU= 26.164
  12:02:49 STEP 1832 H= 458:00 +CPU= 26.812
  12:03:14 STEP 1833 H= 458:15 +CPU= 25.348
  12:03:38 STEP 1834 H= 458:30 +CPU= 23.961
  12:04:05 STEP 1835 H= 458:45 +CPU= 26.403
  12:04:37 STEP 1836 H= 459:00 +CPU= 32.227
Moving to projects directory: /boinc_data/boinc/slots/7/ICMGGh7zg+001836
Moving to projects directory: /boinc_data/boinc/slots/7/ICMSHh7zg+001836
Moving to projects directory: /boinc_data/boinc/slots/7/ICMUAh7zg+001836
  12:05:03 STEP 1837 H= 459:15 +CPU= 26.099
  12:05:28 STEP 1838 H= 459:30 +CPU= 24.548
  12:05:53 STEP 1839 H= 459:45 +CPU= 25.518
  12:06:19 STEP 1840 H= 460:00 +CPU= 25.448
  12:06:45 STEP 1841 H= 460:15 +CPU= 26.054
  12:07:12 STEP 1842 H= 460:30 +CPU= 26.841
  12:07:37 STEP 1843 H= 460:45 +CPU= 25.192
  12:08:04 STEP 1844 H= 461:00 +CPU= 26.366
  12:08:29 STEP 1845 H= 461:15 +CPU= 25.018
  12:08:54 STEP 1846 H= 461:30 +CPU= 24.958
  12:09:20 STEP 1847 H= 461:45 +CPU= 26.517
  12:09:52 STEP 1848 H= 462:00 +CPU= 31.627
Moving to projects directory: /boinc_data/boinc/slots/7/ICMGGh7zg+001848
Moving to projects directory: /boinc_data/boinc/slots/7/ICMSHh7zg+001848
Moving to projects directory: /boinc_data/boinc/slots/7/ICMUAh7zg+001848
  12:10:17 STEP 1849 H= 462:15 +CPU= 24.889
  12:10:47 STEP 1850 H= 462:30 +CPU= 29.624
  12:11:16 STEP 1851 H= 462:45 +CPU= 29.075
  12:11:42 STEP 1852 H= 463:00 +CPU= 25.926
  12:12:07 STEP 1853 H= 463:15 +CPU= 25.308
  12:12:40 STEP 1854 H= 463:30 +CPU= 32.217
  12:13:07 STEP 1855 H= 463:45 +CPU= 27.102
  12:13:35 STEP 1856 H= 464:00 +CPU= 27.772
  12:14:01 STEP 1857 H= 464:15 +CPU= 26.257
  12:14:27 STEP 1858 H= 464:30 +CPU= 26.027
  12:14:55 STEP 1859 H= 464:45 +CPU= 27.672
  12:15:31 STEP 1860 H= 465:00 +CPU= 35.721
Moving to projects directory: /boinc_data/boinc/slots/7/ICMGGh7zg+001860
Moving to projects directory: /boinc_data/boinc/slots/7/ICMSHh7zg+001860
Moving to projects directory: /boinc_data/boinc/slots/7/ICMUAh7zg+001860
  12:16:01 STEP 1861 H= 465:15 +CPU= 30.728
  12:16:27 STEP 1862 H= 465:30 +CPU= 26.002
  12:16:56 STEP 1863 H= 465:45 +CPU= 28.691
  12:17:25 STEP 1864 H= 466:00 +CPU= 28.309
  12:17:51 STEP 1865 H= 466:15 +CPU= 26.124
  12:18:21 STEP 1866 H= 466:30 +CPU= 29.982
  12:18:49 STEP 1867 H= 466:45 +CPU= 28.352
  12:19:18 STEP 1868 H= 467:00 +CPU= 28.220
  12:19:47 STEP 1869 H= 467:15 +CPU= 28.774
  12:20:14 STEP 1870 H= 467:30 +CPU= 27.175
  12:20:43 STEP 1871 H= 467:45 +CPU= 28.918
  12:21:25 STEP 1872 H= 468:00 +CPU= 40.688
Moving to projects directory: /boinc_data/boinc/slots/7/ICMGGh7zg+001872
Moving to projects directory: /boinc_data/boinc/slots/7/ICMSHh7zg+001872
Moving to projects directory: /boinc_data/boinc/slots/7/ICMUAh7zg+001872
  12:21:54 STEP 1873 H= 468:15 +CPU= 28.180
  12:22:22 STEP 1874 H= 468:30 +CPU= 27.543
  12:22:52 STEP 1875 H= 468:45 +CPU= 29.960
  12:23:23 STEP 1876 H= 469:00 +CPU= 30.293
  12:23:53 STEP 1877 H= 469:15 +CPU= 29.552
  12:24:22 STEP 1878 H= 469:30 +CPU= 27.876
  12:24:54 STEP 1879 H= 469:45 +CPU= 30.904
  12:25:20 STEP 1880 H= 470:00 +CPU= 26.116
  12:25:48 STEP 1881 H= 470:15 +CPU= 27.190
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_bl_1.13_x86_64-pc-linux-gnu
(argv1) start_date: 2016092300
(argv2) exptid: h7zg
(argv3) unique_member_id: a4go
(argv4) batchid: 1018
(argv5) wuid: 12294111
(argv6) fclen: 20
(argv7) app_name: oifs_43r3_bl
(argv8) nthreads: 1
Working directory is: /boinc_data/boinc/slots/7
Project directory is: /boinc_data/boinc/projects/climateprediction.net/
app name: oifs_43r3_bl
version: 1.13
Location of temp folder: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111
..mkdir for temp folder for results failed
Copying: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip to: /boinc_data/boinc/slots/7/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /boinc_data/boinc/slots/7/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_e0178dea3fcb99f334bda7e375c21acc to: /boinc_data/boinc/slots/7/oifs_43r3_bl_a4go_2016092300_20_1018_12294111.zip
Unzipping the namelist zip file: /boinc_data/boinc/slots/7/oifs_43r3_bl_a4go_2016092300_20_1018_12294111.zip
ic_ancil_file: ic_ancil_12294111
ifsdata_file: ifsdata_12294111
climate_data_file: clim_data_12294111
horiz_resolution: 159
vert_resolution: 91
grid_type: l_2
upload_interval: 96
utstep: 900
nfrres: restart dump frequency (steps) 48
Copying IC ancils from: ../../projects/climateprediction.net/jf_f7ecfd27aa1e68bbe7e8d19800e25c73 to: /boinc_data/boinc/slots/7/ic_ancil_12294111.zip
Unzipping the IC ancils zip file: /boinc_data/boinc/slots/7/ic_ancil_12294111.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_3b29e02eb48e7cb740cea4e77046bf9d to: /boinc_data/boinc/slots/7/ifsdata/ifsdata_12294111.zip
Unzipping the ifsdata_zip file: /boinc_data/boinc/slots/7/ifsdata/ifsdata_12294111.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /boinc_data/boinc/slots/7/159l_2/clim_data_12294111.zip
Unzipping the climate data zip file: /boinc_data/boinc/slots/7/159l_2/clim_data_12294111.zip
Checking for progress XML file: /boinc_data/boinc/slots/7/progress_file_12294111.xml
Opened progress file ok : /boinc_data/boinc/slots/7/progress_file_12294111.xml
-- Model is restarting --
Adjusting last_iter, 1882, to previous model restart step.
Creating progress file: /boinc_data/boinc/slots/7/progress_file_12294111.xml
last_cpu_time: 49761
upload_file_number: 19
last_iter: 1873
last_upload: 1641600
model_completed: 0
total_length_of_simulation: 1728000
result_base_name: oifs_43r3_bl_a4go_2016092300_20_1018_12294111_0_r1581845035
The child process has been launched with process id: 834027
Executing the command: /boinc_data/boinc/slots/7/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:530FCTest:1:1:834027:834027] [20240615:131124:1718475084:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2091] fp = 0x314f440
[EC_DRHOOK:530FCTest:1:1:834027:834027] [20240615:131124:1718475084:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:530FCTest:1:1:834027:834027] [20240615:131124:1718475084:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:530FCTest:1:1:834027:834027] [20240615:131124:1718475084:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:530FCTest:1:1:834027:834027] [20240615:131124:1718475084:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:530FCTest:1:1:834027:834027] [20240615:131124:1718475084:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:530FCTest:1:1:834027:834027] [20240615:131124:1718475084:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:530FCTest:1:1:834027:834027] [20240615:131124:1718475084:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:530FCTest:1:1:834027:834027] [20240615:131124:1718475084:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:530FCTest:1:1:834027:834027] [20240615:131124:1718475084:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:530FCTest:1:1:834027:834027] [20240615:131124:1718475084:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:530FCTest:1:1:834027:834027] [20240615:131124:1718475084:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:530FCTest:1:1:834027:834027] [20240615:131124:1718475084:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:530FCTest:1:1:834027:834027] [20240615:131124:1718475084:0.001] [ignore_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:530FCTest:1:1:834027:834027] [20240615:131124:1718475084:0.001] [restore_default_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:530FCTest:1:1:834027:834027] [20240615:131124:1718475084:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:530FCTest:1:1:834027:834027] [20240615:131124:1718475084:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:530FCTest:1:1:834027:834027] [20240615:131124:1718475084:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:530FCTest:1:1:834027:834027] [20240615:131124:1718475084:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x820cf0 (old at (nil))
[EC_DRHOOK:530FCTest:1:1:834027:834027] [20240615:131124:1718475084:0.001] [trapfpe_treatment@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:530FCTest:1:1:834027:834027] [20240615:131124:1718475084:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:530FCTest:1:1:834027:834027] [20240615:131124:1718475084:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:530FCTest:1:1:834027:834027] [20240615:131124:1718475084:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x820cf0 (old at (nil))
[EC_DRHOOK:530FCTest:1:1:834027:834027] [20240615:131124:1718475084:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:530FCTest:1:1:834027:834027] [20240615:131124:1718475084:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:530FCTest:1:1:834027:834027] [20240615:131124:1718475084:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:530FCTest:1:1:834027:834027] [20240615:131124:1718475084:0.002] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x820cf0 (old at (nil))
[EC_DRHOOK:530FCTest:1:1:834027:834027] [20240615:131124:1718475084:0.002] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x820cf0 (old at (nil))
[EC_DRHOOK:530FCTest:1:1:834027:834027] [20240615:131124:1718475084:0.002] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:530FCTest:1:1:834027:834027] [20240615:131124:1718475084:0.002] [catch_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  13:11:54 STEP 1872 H= 468:00 +CPU= 26.218
  13:12:17 STEP 1873 H= 468:15 +CPU= 23.760
  13:12:42 STEP 1874 H= 468:30 +CPU= 24.944
  13:13:07 STEP 1875 H= 468:45 +CPU= 24.478
  13:13:31 STEP 1876 H= 469:00 +CPU= 24.511
  13:13:57 STEP 1877 H= 469:15 +CPU= 25.208
  13:14:22 STEP 1878 H= 469:30 +CPU= 25.515
  13:14:47 STEP 1879 H= 469:45 +CPU= 24.625
  13:15:12 STEP 1880 H= 470:00 +CPU= 25.253
  13:15:36 STEP 1881 H= 470:15 +CPU= 23.946
  13:16:01 STEP 1882 H= 470:30 +CPU= 24.724
  13:16:26 STEP 1883 H= 470:45 +CPU= 24.759
  13:16:58 STEP 1884 H= 471:00 +CPU= 32.205
Moving to projects directory: /boinc_data/boinc/slots/7/ICMGGh7zg+001884
Moving to projects directory: /boinc_data/boinc/slots/7/ICMSHh7zg+001884
Moving to projects directory: /boinc_data/boinc/slots/7/ICMUAh7zg+001884
  13:17:23 STEP 1885 H= 471:15 +CPU= 24.902
  13:17:48 STEP 1886 H= 471:30 +CPU= 25.155
  13:18:13 STEP 1887 H= 471:45 +CPU= 25.185
  13:18:40 STEP 1888 H= 472:00 +CPU= 26.183
  13:19:05 STEP 1889 H= 472:15 +CPU= 25.284
  13:19:31 STEP 1890 H= 472:30 +CPU= 26.150
  13:19:57 STEP 1891 H= 472:45 +CPU= 25.866
  13:20:24 STEP 1892 H= 473:00 +CPU= 26.431
  13:20:50 STEP 1893 H= 473:15 +CPU= 26.099
  13:21:16 STEP 1894 H= 473:30 +CPU= 26.061
  13:21:41 STEP 1895 H= 473:45 +CPU= 25.291
  13:22:13 STEP 1896 H= 474:00 +CPU= 31.009
Moving to projects directory: /boinc_data/boinc/slots/7/ICMGGh7zg+001896
Moving to projects directory: /boinc_data/boinc/slots/7/ICMSHh7zg+001896
Moving to projects directory: /boinc_data/boinc/slots/7/ICMUAh7zg+001896
  13:22:37 STEP 1897 H= 474:15 +CPU= 24.433
  13:23:03 STEP 1898 H= 474:30 +CPU= 25.564
  13:23:27 STEP 1899 H= 474:45 +CPU= 24.512
  13:23:54 STEP 1900 H= 475:00 +CPU= 26.473
  13:24:18 STEP 1901 H= 475:15 +CPU= 24.474
  13:24:44 STEP 1902 H= 475:30 +CPU= 25.128
  13:25:09 STEP 1903 H= 475:45 +CPU= 25.022
  13:25:35 STEP 1904 H= 476:00 +CPU= 25.938
  13:26:00 STEP 1905 H= 476:15 +CPU= 24.879
  13:26:25 STEP 1906 H= 476:30 +CPU= 24.973
  13:26:49 STEP 1907 H= 476:45 +CPU= 24.697
  13:27:21 STEP 1908 H= 477:00 +CPU= 31.748
Moving to projects directory: /boinc_data/boinc/slots/7/ICMGGh7zg+001908
Moving to projects directory: /boinc_data/boinc/slots/7/ICMSHh7zg+001908
Moving to projects directory: /boinc_data/boinc/slots/7/ICMUAh7zg+001908
  13:27:46 STEP 1909 H= 477:15 +CPU= 25.290
  13:28:10 STEP 1910 H= 477:30 +CPU= 23.953
  13:28:35 STEP 1911 H= 477:45 +CPU= 24.448
  13:29:01 STEP 1912 H= 478:00 +CPU= 25.868
  13:29:25 STEP 1913 H= 478:15 +CPU= 24.157
  13:29:48 STEP 1914 H= 478:30 +CPU= 22.463
  13:30:11 STEP 1915 H= 478:45 +CPU= 23.107
  13:30:37 STEP 1916 H= 479:00 +CPU= 26.152
  13:31:04 STEP 1917 H= 479:15 +CPU= 26.730
  13:31:31 STEP 1918 H= 479:30 +CPU= 26.990
  13:31:58 STEP 1919 H= 479:45 +CPU= 26.846
  13:32:35 STEP 1920 H= 480:00 +CPU= 36.301
..The child process terminated with status: 0
>>> Printing last 70 lines from file: NODE.001_01

 
DDH------------------------------------------------------------------------
I                                                                         I
I TIME =  480.000H DDH * Diagnostic Var.,     AREA                GLOBAL  I
I                                                                         I
I-------------------------------------------------------------------------I
I   VARIABLE   I      Initial value         I        Current value        I
I-------------------------------------------------------------------------I
I  OMEGA (P/S) I      0.2793380807E-21      I       0.2209980012E-07      I
ICloud fractionI      0.0000000000E+00      I       0.7573086415E-04      I
I  Relat. Hum. I      0.2004178900E+00      I       0.2160196274E+00      I
I  PASS 01     I      0.0000000000E+00      I       0.4732604588E+01      I
DDH------------------------------------------------------------------------
  
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
 NSTEP =  1920 SCAN2M_HPOS  P
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ICMGGh7zg+001920 MODE=a
 IO-STREAM CLOSED - ICMGGh7zg+001920
 MPI-TASK:   1 -    417098948 BYTES IN      7 RECORDS TRANSFERRED IN    0.0007 SECONDS595855.6 Mbytes/s, TOTAL TIME=    0.0122( 5.7%)
 13:32:35 STEP 1920 H= 480:00 +CPU= 36.301
 END CNT3
 NSTEP =  1920 CNT0   000000000  26.240  26.240  26.633   0.985               0               0               0               0               0
IO-STREAM STATISTICS, TOTAL NO OF REC/BYTES READ           46/  1069125708 WRITTEN         1430/************
 -TASK-OPENED-OPEN-RECS IN -KBYTE IN    -RECS OUT-KBYTE OUT-WALL   -WALL IN-WALL
  OU-TOT IN -TOT OUT
   1     28    0       46      1044068     1430 ************     4.0     1.5     2.6     2.2     5.2
===-=== START OF TIMING STATISTICS ===-===
 
STATS FOR ALL TASKS
 NUM ROUTINE                                     CALLS  MEAN(ms)   MAX(ms)   FRAC(%)  UNBAL(%)
   0 CNT0     - COMPLETE EXECUTION                   1 1271341.1 1271341.1    100.00      0.00
   1 CNT4     - FORWARD INTEGRATION                  1 1267531.7 1267531.7     99.70      0.00
   8 SCAN2M - GRID-POINT DYNAMICS                   53    4566.5    4566.5     19.04      0.00
   9 SPCM     - SPECTRAL COMP.                      48     619.1     619.1      2.34      0.00
  10 SCAN2M - PHYSICS                               49   11516.1   11516.1     44.39      0.00
  11 IOPACK   - OUTPUT P.P. RESULTS                  4     927.6     927.6      0.29      0.00
  12 SPNORM   - SPECTRAL NORM COMP.                  3      40.9      40.9      0.01      0.00
  17 GRIDFPOS IN CNT4                                4      29.0      29.0      0.01      0.00
  27 RESTART FILES - WRITING                         1    3710.5    3710.5      0.29      0.00
  28 RESTART FILES - READING                         1    2972.4    2972.4      0.23      0.00
  29 SU4FPOS IN CNT4                                 4       0.0       0.0      0.00      0.00
  30 DYNFPOS IN CNT4                                 4    5895.4    5895.4      1.85      0.00
  31 POSDDH IN STEPO                                 1       1.0       1.0      0.00      0.00
  37 CPGLAG   - SL COMPUTATIONS                     49    5854.4    5854.4     22.56      0.00
  39 SU0YOMB                                         1     517.1     517.1      0.04      0.00
  51 SCAN2M   - SL COMM. PART 1                     49      20.1      20.1      0.08      0.00
  54 SPCM     - M TO S/S TO M TRANSP.               48     289.5     289.5      1.09      0.00
  55 SPCIMPF  - S TO M/M TO S TRANSP.               48      55.6      55.6      0.21      0.00
  56 SPNORM   - SPECTRAL NORM COMM.                  3       0.2       0.2      0.00      0.00
 102 LTINV_CTL   - INVERSE LEGENDRE TRANSFORM      108     382.8     382.8      3.25      0.00
 103 LTDIR_CTL   - DIRECT LEGENDRE TRANSFORM       100     228.1     228.1      1.79      0.00
 106 FTDIR_CTL   - DIRECT FOURIER TRANSFORM        100      79.7      79.7      0.63      0.00
 107 FTINV_CTL   - INVERSE FOURIER TRANSFORM       108     115.1     115.1      0.98      0.00
 140 SULEG       - COMP. OF LEGENDRE POL.            1      59.5      59.5      0.00      0.00
 152 LTINV_CTL   - M TO L TRANSPOSITION            108      61.5      61.5      0.52      0.00
 153 LTDIR_CTL   - L TO M TRANSPOSITION            100      32.2      32.2      0.25      0.00
 157 FTINV_CTL   - L TO G TRANSPOSITION            108      48.2      48.2      0.41      0.00
 158 FTDIR_CTL   - G TO L TRANSPOSITION            100      26.5      26.5      0.21      0.00
 400 GSTATS                                       6667       0.0       0.0      0.00      0.00
 401 GSTATS HOOK                                  6149       0.0       0.0      0.00      0.00
TOTAL MEASURED IMBALANCE =       0.0 SECONDS,  0.0 PERCENT
TOTAL WALLCLOCK TIME     1271.3 CPU TIME    1267.7 VECTOR TIME     1267.7


===-=== END   OF TIMING STATISTICS ===-===
FORECAST DAYS PER DAY     1363.3
  *** END CNT0 *** 
------------------------------------------------
Moving to projects directory: /boinc_data/boinc/slots/7/ICMGGh7zg+001920
Moving to projects directory: /boinc_data/boinc/slots/7/ICMSHh7zg+001920
Moving to projects directory: /boinc_data/boinc/slots/7/ICMUAh7zg+001920
Adding to the zip: /boinc_data/boinc/slots/7/NODE.001_01
Adding to the zip: /boinc_data/boinc/slots/7/ifs.stat
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMSHh7zg+001896
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMGGh7zg+001836
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMUAh7zg+001896
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMSHh7zg+001908
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMGGh7zg+001872
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMGGh7zg+001908
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMUAh7zg+001884
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMGGh7zg+001848
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMGGh7zg+001860
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMSHh7zg+001824
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMUAh7zg+001920
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMUAh7zg+001824
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMSHh7zg+001920
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMSHh7zg+001884
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMUAh7zg+001872
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMSHh7zg+001860
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMGGh7zg+001884
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMUAh7zg+001836
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMUAh7zg+001848
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMGGh7zg+001824
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMUAh7zg+001908
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMSHh7zg+001836
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMSHh7zg+001872
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMGGh7zg+001896
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMGGh7zg+001920
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMSHh7zg+001848
Adding to the zip: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_12294111/ICMUAh7zg+001860
Zipping up the final file: /boinc_data/boinc/projects/climateprediction.net/oifs_43r3_bl_a4go_2016092300_20_1018_12294111_0_r1581845035_19.zip
Uploading the final file: upload_file_19.zip
Uploading trickle at timestep: 1727100
13:36:05 (834024): called boinc_finish(0)

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
15 Jun 2024 19:07:05 1496046 22447728 oifs_43r3_bl_a4go_2016092300_20_1018_12294111_0 1,727,100 51,028 0.0295
15 Jun 2024 10:55:19 1496046 22447728 oifs_43r3_bl_a4go_2016092300_20_1018_12294111_0 864,000 25,275 0.0293


©2024 climateprediction.net