Name | oifs_43r3_bl_a4it_2016092300_20_1018_12294188_0 |
Workunit | 12294188 |
Created | 13 Jun 2024, 4:29:59 UTC |
Sent | 14 Jun 2024, 0:00:43 UTC |
Report deadline | 13 Aug 2024, 0:00:43 UTC |
Received | 18 Jun 2024, 15:24:44 UTC |
Server state | Over |
Outcome | Success |
Client state | Done |
Exit status | 0 (0x00000000) |
Computer ID | 1542649 |
Run time | 20 hours 4 min 39 sec |
CPU time | 11 hours 29 min 45 sec |
Validate state | Valid |
Credit | 2,635.55 |
Device peak FLOPS | 4.95 GFLOPS |
Application version | OpenIFS 43r3 Baroclinic Lifecycle v1.13 x86_64-pc-linux-gnu |
Peak working set size | 5,024.82 MB |
Peak swap size | 5,981.07 MB |
Peak disk usage | 2,771.20 MB |
Stderr | <core_client_version>7.18.1</core_client_version> <![CDATA[ <stderr_txt> /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMUAh7zg+001380 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMGGh7zg+001392 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMSHh7zg+001392 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMUAh7zg+001392 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMGGh7zg+001404 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMSHh7zg+001404 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMUAh7zg+001404 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMGGh7zg+001416 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMSHh7zg+001416 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMUAh7zg+001416 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMGGh7zg+001428 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMSHh7zg+001428 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMUAh7zg+001428 Zipping up the intermediate file: /home/amar/projects/climateprediction.net/oifs_43r3_bl_a4it_2016092300_20_1018_12294188_0_r48016068_14.zip Uploading the intermediate file: upload_file_14.zip 09:16:44 STEP 1441 H= 360:15 +CPU= 20.794 09:17:19 STEP 1442 H= 360:30 +CPU= 20.127 09:17:53 STEP 1443 H= 360:45 +CPU= 19.985 09:18:28 STEP 1444 H= 361:00 +CPU= 20.510 09:19:01 STEP 1445 H= 361:15 +CPU= 20.041 09:19:35 STEP 1446 H= 361:30 +CPU= 19.986 09:20:10 STEP 1447 H= 361:45 +CPU= 20.272 09:20:43 STEP 1448 H= 362:00 +CPU= 20.294 09:21:15 STEP 1449 H= 362:15 +CPU= 20.060 09:21:48 STEP 1450 H= 362:30 +CPU= 19.917 09:22:23 STEP 1451 H= 362:45 +CPU= 20.059 09:23:07 STEP 1452 H= 363:00 +CPU= 25.306 Moving to projects directory: /home/amar/slots/0/ICMGGh7zg+001452 Moving to projects directory: /home/amar/slots/0/ICMSHh7zg+001452 Moving to projects directory: /home/amar/slots/0/ICMUAh7zg+001452 09:23:41 STEP 1453 H= 363:15 +CPU= 19.971 09:24:14 STEP 1454 H= 363:30 +CPU= 20.142 09:24:48 STEP 1455 H= 363:45 +CPU= 20.026 09:25:23 STEP 1456 H= 364:00 +CPU= 20.534 09:25:56 STEP 1457 H= 364:15 +CPU= 19.966 09:26:31 STEP 1458 H= 364:30 +CPU= 20.122 09:27:03 STEP 1459 H= 364:45 +CPU= 20.009 09:27:36 STEP 1460 H= 365:00 +CPU= 20.275 09:28:09 STEP 1461 H= 365:15 +CPU= 19.995 09:28:42 STEP 1462 H= 365:30 +CPU= 19.962 09:29:17 STEP 1463 H= 365:45 +CPU= 20.078 09:29:59 STEP 1464 H= 366:00 +CPU= 25.027 Moving to projects directory: /home/amar/slots/0/ICMGGh7zg+001464 Moving to projects directory: /home/amar/slots/0/ICMSHh7zg+001464 Moving to projects directory: /home/amar/slots/0/ICMUAh7zg+001464 09:30:33 STEP 1465 H= 366:15 +CPU= 20.170 09:31:06 STEP 1466 H= 366:30 +CPU= 20.084 09:31:40 STEP 1467 H= 366:45 +CPU= 19.935 09:32:15 STEP 1468 H= 367:00 +CPU= 20.477 09:32:48 STEP 1469 H= 367:15 +CPU= 19.951 09:33:22 STEP 1470 H= 367:30 +CPU= 20.103 09:33:55 STEP 1471 H= 367:45 +CPU= 19.919 09:34:28 STEP 1472 H= 368:00 +CPU= 20.347 09:35:01 STEP 1473 H= 368:15 +CPU= 20.059 09:35:37 STEP 1474 H= 368:30 +CPU= 20.163 09:36:09 STEP 1475 H= 368:45 +CPU= 20.031 09:36:52 STEP 1476 H= 369:00 +CPU= 24.992 Moving to projects directory: /home/amar/slots/0/ICMGGh7zg+001476 Moving to projects directory: /home/amar/slots/0/ICMSHh7zg+001476 Moving to projects directory: /home/amar/slots/0/ICMUAh7zg+001476 09:37:27 STEP 1477 H= 369:15 +CPU= 20.176 09:38:01 STEP 1478 H= 369:30 +CPU= 19.967 09:38:33 STEP 1479 H= 369:45 +CPU= 20.083 09:39:09 STEP 1480 H= 370:00 +CPU= 20.413 09:39:43 STEP 1481 H= 370:15 +CPU= 19.958 09:40:16 STEP 1482 H= 370:30 +CPU= 20.158 09:40:49 STEP 1483 H= 370:45 +CPU= 19.906 09:41:24 STEP 1484 H= 371:00 +CPU= 20.451 09:41:58 STEP 1485 H= 371:15 +CPU= 19.898 09:42:30 STEP 1486 H= 371:30 +CPU= 20.043 09:43:04 STEP 1487 H= 371:45 +CPU= 20.001 09:43:52 STEP 1488 H= 372:00 +CPU= 27.715 Moving to projects directory: /home/amar/slots/0/ICMGGh7zg+001488 Moving to projects directory: /home/amar/slots/0/ICMSHh7zg+001488 Moving to projects directory: /home/amar/slots/0/ICMUAh7zg+001488 09:44:25 STEP 1489 H= 372:15 +CPU= 19.992 09:44:57 STEP 1490 H= 372:30 +CPU= 19.998 09:45:30 STEP 1491 H= 372:45 +CPU= 20.064 09:46:05 STEP 1492 H= 373:00 +CPU= 20.395 09:46:37 STEP 1493 H= 373:15 +CPU= 19.905 09:47:10 STEP 1494 H= 373:30 +CPU= 20.030 09:47:43 STEP 1495 H= 373:45 +CPU= 19.930 09:48:16 STEP 1496 H= 374:00 +CPU= 20.396 09:48:50 STEP 1497 H= 374:15 +CPU= 20.054 09:49:24 STEP 1498 H= 374:30 +CPU= 20.108 09:49:58 STEP 1499 H= 374:45 +CPU= 19.968 09:50:41 STEP 1500 H= 375:00 +CPU= 25.183 Moving to projects directory: /home/amar/slots/0/ICMGGh7zg+001500 Moving to projects directory: /home/amar/slots/0/ICMSHh7zg+001500 Moving to projects directory: /home/amar/slots/0/ICMUAh7zg+001500 09:51:15 STEP 1501 H= 375:15 +CPU= 20.036 09:51:47 STEP 1502 H= 375:30 +CPU= 19.856 09:52:21 STEP 1503 H= 375:45 +CPU= 20.001 09:52:55 STEP 1504 H= 376:00 +CPU= 20.362 09:53:30 STEP 1505 H= 376:15 +CPU= 20.134 09:54:04 STEP 1506 H= 376:30 +CPU= 20.214 09:54:37 STEP 1507 H= 376:45 +CPU= 19.975 09:55:12 STEP 1508 H= 377:00 +CPU= 20.637 09:55:47 STEP 1509 H= 377:15 +CPU= 20.100 09:56:22 STEP 1510 H= 377:30 +CPU= 20.066 09:56:54 STEP 1511 H= 377:45 +CPU= 19.977 09:57:37 STEP 1512 H= 378:00 +CPU= 25.058 Moving to projects directory: /home/amar/slots/0/ICMGGh7zg+001512 Moving to projects directory: /home/amar/slots/0/ICMSHh7zg+001512 Moving to projects directory: /home/amar/slots/0/ICMUAh7zg+001512 09:58:11 STEP 1513 H= 378:15 +CPU= 20.050 09:58:44 STEP 1514 H= 378:30 +CPU= 19.912 09:59:19 STEP 1515 H= 378:45 +CPU= 20.163 09:59:52 STEP 1516 H= 379:00 +CPU= 20.213 10:00:26 STEP 1517 H= 379:15 +CPU= 20.206 10:01:01 STEP 1518 H= 379:30 +CPU= 20.207 10:01:34 STEP 1519 H= 379:45 +CPU= 19.961 10:02:07 STEP 1520 H= 380:00 +CPU= 20.372 10:02:41 STEP 1521 H= 380:15 +CPU= 19.923 10:03:15 STEP 1522 H= 380:30 +CPU= 20.107 10:03:48 STEP 1523 H= 380:45 +CPU= 19.954 10:04:28 STEP 1524 H= 381:00 +CPU= 24.974 Moving to projects directory: /home/amar/slots/0/ICMGGh7zg+001524 Moving to projects directory: /home/amar/slots/0/ICMSHh7zg+001524 Moving to projects directory: /home/amar/slots/0/ICMUAh7zg+001524 10:05:02 STEP 1525 H= 381:15 +CPU= 20.003 10:05:37 STEP 1526 H= 381:30 +CPU= 20.081 10:06:10 STEP 1527 H= 381:45 +CPU= 19.954 10:06:44 STEP 1528 H= 382:00 +CPU= 20.329 10:07:19 STEP 1529 H= 382:15 +CPU= 20.188 10:07:52 STEP 1530 H= 382:30 +CPU= 19.988 10:08:26 STEP 1531 H= 382:45 +CPU= 20.028 10:09:00 STEP 1532 H= 383:00 +CPU= 20.306 10:09:33 STEP 1533 H= 383:15 +CPU= 19.978 10:10:08 STEP 1534 H= 383:30 +CPU= 20.130 10:10:42 STEP 1535 H= 383:45 +CPU= 19.989 10:11:28 STEP 1536 H= 384:00 +CPU= 27.766 Moving to projects directory: /home/amar/slots/0/ICMGGh7zg+001536 Moving to projects directory: /home/amar/slots/0/ICMSHh7zg+001536 Moving to projects directory: /home/amar/slots/0/ICMUAh7zg+001536 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMGGh7zg+001440 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMSHh7zg+001440 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMUAh7zg+001440 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMGGh7zg+001452 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMSHh7zg+001452 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMUAh7zg+001452 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMGGh7zg+001464 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMSHh7zg+001464 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMUAh7zg+001464 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMGGh7zg+001476 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMSHh7zg+001476 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMUAh7zg+001476 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMGGh7zg+001488 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMSHh7zg+001488 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMUAh7zg+001488 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMGGh7zg+001500 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMSHh7zg+001500 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMUAh7zg+001500 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMGGh7zg+001512 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMSHh7zg+001512 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMUAh7zg+001512 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMGGh7zg+001524 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMSHh7zg+001524 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMUAh7zg+001524 Zipping up the intermediate file: /home/amar/projects/climateprediction.net/oifs_43r3_bl_a4it_2016092300_20_1018_12294188_0_r48016068_15.zip Uploading the intermediate file: upload_file_15.zip 10:11:59 STEP 1537 H= 384:15 +CPU= 20.885 10:12:37 STEP 1538 H= 384:30 +CPU= 20.265 10:13:11 STEP 1539 H= 384:45 +CPU= 20.069 10:13:45 STEP 1540 H= 385:00 +CPU= 20.264 10:14:19 STEP 1541 H= 385:15 +CPU= 20.001 10:14:51 STEP 1542 H= 385:30 +CPU= 19.927 10:15:25 STEP 1543 H= 385:45 +CPU= 20.150 10:16:00 STEP 1544 H= 386:00 +CPU= 20.331 10:16:33 STEP 1545 H= 386:15 +CPU= 20.152 10:17:07 STEP 1546 H= 386:30 +CPU= 20.106 10:17:41 STEP 1547 H= 386:45 +CPU= 19.972 10:18:24 STEP 1548 H= 387:00 +CPU= 25.090 Moving to projects directory: /home/amar/slots/0/ICMGGh7zg+001548 Moving to projects directory: /home/amar/slots/0/ICMSHh7zg+001548 Moving to projects directory: /home/amar/slots/0/ICMUAh7zg+001548 10:18:57 STEP 1549 H= 387:15 +CPU= 20.005 10:19:32 STEP 1550 H= 387:30 +CPU= 20.056 10:20:05 STEP 1551 H= 387:45 +CPU= 19.984 10:20:40 STEP 1552 H= 388:00 +CPU= 20.451 10:21:14 STEP 1553 H= 388:15 +CPU= 20.171 10:21:47 STEP 1554 H= 388:30 +CPU= 20.033 10:22:20 STEP 1555 H= 388:45 +CPU= 19.974 10:22:54 STEP 1556 H= 389:00 +CPU= 20.304 10:23:28 STEP 1557 H= 389:15 +CPU= 20.096 10:24:01 STEP 1558 H= 389:30 +CPU= 19.919 10:24:34 STEP 1559 H= 389:45 +CPU= 19.968 10:25:19 STEP 1560 H= 390:00 +CPU= 25.431 Moving to projects directory: /home/amar/slots/0/ICMGGh7zg+001560 Moving to projects directory: /home/amar/slots/0/ICMSHh7zg+001560 Moving to projects directory: /home/amar/slots/0/ICMUAh7zg+001560 10:25:53 STEP 1561 H= 390:15 +CPU= 19.981 10:26:27 STEP 1562 H= 390:30 +CPU= 20.014 10:27:01 STEP 1563 H= 390:45 +CPU= 20.136 10:27:37 STEP 1564 H= 391:00 +CPU= 20.285 10:28:10 STEP 1565 H= 391:15 +CPU= 19.968 10:28:43 STEP 1566 H= 391:30 +CPU= 19.948 10:29:16 STEP 1567 H= 391:45 +CPU= 20.015 10:29:50 STEP 1568 H= 392:00 +CPU= 20.220 10:30:25 STEP 1569 H= 392:15 +CPU= 20.145 10:30:58 STEP 1570 H= 392:30 +CPU= 20.020 10:31:33 STEP 1571 H= 392:45 +CPU= 20.193 10:32:15 STEP 1572 H= 393:00 +CPU= 25.160 Moving to projects directory: /home/amar/slots/0/ICMGGh7zg+001572 Moving to projects directory: /home/amar/slots/0/ICMSHh7zg+001572 Moving to projects directory: /home/amar/slots/0/ICMUAh7zg+001572 10:32:47 STEP 1573 H= 393:15 +CPU= 19.897 10:33:22 STEP 1574 H= 393:30 +CPU= 20.156 10:33:55 STEP 1575 H= 393:45 +CPU= 19.915 10:34:30 STEP 1576 H= 394:00 +CPU= 20.377 10:35:03 STEP 1577 H= 394:15 +CPU= 20.082 10:35:36 STEP 1578 H= 394:30 +CPU= 19.965 10:36:08 STEP 1579 H= 394:45 +CPU= 19.999 10:36:43 STEP 1580 H= 395:00 +CPU= 20.346 10:37:16 STEP 1581 H= 395:15 +CPU= 20.134 10:37:49 STEP 1582 H= 395:30 +CPU= 19.930 10:38:23 STEP 1583 H= 395:45 +CPU= 20.036 10:39:10 STEP 1584 H= 396:00 +CPU= 27.781 Moving to projects directory: /home/amar/slots/0/ICMGGh7zg+001584 Moving to projects directory: /home/amar/slots/0/ICMSHh7zg+001584 Moving to projects directory: /home/amar/slots/0/ICMUAh7zg+001584 10:39:43 STEP 1585 H= 396:15 +CPU= 19.922 10:40:16 STEP 1586 H= 396:30 +CPU= 20.064 10:40:50 STEP 1587 H= 396:45 +CPU= 19.964 10:41:24 STEP 1588 H= 397:00 +CPU= 20.437 10:41:58 STEP 1589 H= 397:15 +CPU= 19.998 10:42:31 STEP 1590 H= 397:30 +CPU= 20.029 10:43:04 STEP 1591 H= 397:45 +CPU= 20.131 10:43:42 STEP 1592 H= 398:00 +CPU= 20.854 10:44:16 STEP 1593 H= 398:15 +CPU= 19.960 10:44:49 STEP 1594 H= 398:30 +CPU= 20.035 10:45:23 STEP 1595 H= 398:45 +CPU= 20.120 10:46:05 STEP 1596 H= 399:00 +CPU= 24.997 Moving to projects directory: /home/amar/slots/0/ICMGGh7zg+001596 Moving to projects directory: /home/amar/slots/0/ICMSHh7zg+001596 Moving to projects directory: /home/amar/slots/0/ICMUAh7zg+001596 10:46:38 STEP 1597 H= 399:15 +CPU= 19.982 10:47:12 STEP 1598 H= 399:30 +CPU= 20.020 10:47:45 STEP 1599 H= 399:45 +CPU= 20.006 10:48:19 STEP 1600 H= 400:00 +CPU= 20.385 10:48:53 STEP 1601 H= 400:15 +CPU= 20.010 10:49:27 STEP 1602 H= 400:30 +CPU= 20.121 10:50:00 STEP 1603 H= 400:45 +CPU= 20.000 10:50:36 STEP 1604 H= 401:00 +CPU= 20.616 10:51:08 STEP 1605 H= 401:15 +CPU= 20.078 10:51:41 STEP 1606 H= 401:30 +CPU= 19.924 10:52:15 STEP 1607 H= 401:45 +CPU= 20.085 10:52:57 STEP 1608 H= 402:00 +CPU= 24.989 Moving to projects directory: /home/amar/slots/0/ICMGGh7zg+001608 Moving to projects directory: /home/amar/slots/0/ICMSHh7zg+001608 Moving to projects directory: /home/amar/slots/0/ICMUAh7zg+001608 10:53:31 STEP 1609 H= 402:15 +CPU= 20.111 10:54:04 STEP 1610 H= 402:30 +CPU= 20.086 10:54:38 STEP 1611 H= 402:45 +CPU= 19.974 10:55:07 STEP 1612 H= 403:00 +CPU= 20.258 10:55:42 STEP 1613 H= 403:15 +CPU= 20.109 10:56:16 STEP 1614 H= 403:30 +CPU= 20.108 10:56:49 STEP 1615 H= 403:45 +CPU= 20.065 10:57:23 STEP 1616 H= 404:00 +CPU= 20.346 10:57:56 STEP 1617 H= 404:15 +CPU= 19.985 10:58:30 STEP 1618 H= 404:30 +CPU= 20.197 10:59:03 STEP 1619 H= 404:45 +CPU= 20.060 10:59:45 STEP 1620 H= 405:00 +CPU= 24.940 Moving to projects directory: /home/amar/slots/0/ICMGGh7zg+001620 Moving to projects directory: /home/amar/slots/0/ICMSHh7zg+001620 Moving to projects directory: /home/amar/slots/0/ICMUAh7zg+001620 11:00:19 STEP 1621 H= 405:15 +CPU= 20.086 11:00:53 STEP 1622 H= 405:30 +CPU= 20.138 11:01:26 STEP 1623 H= 405:45 +CPU= 20.119 11:02:00 STEP 1624 H= 406:00 +CPU= 20.352 11:02:34 STEP 1625 H= 406:15 +CPU= 20.037 11:03:07 STEP 1626 H= 406:30 +CPU= 20.105 11:03:41 STEP 1627 H= 406:45 +CPU= 19.985 11:04:16 STEP 1628 H= 407:00 +CPU= 20.465 11:04:50 STEP 1629 H= 407:15 +CPU= 19.919 11:05:22 STEP 1630 H= 407:30 +CPU= 20.075 11:05:56 STEP 1631 H= 407:45 +CPU= 20.045 11:06:43 STEP 1632 H= 408:00 +CPU= 27.685 Moving to projects directory: /home/amar/slots/0/ICMGGh7zg+001632 Moving to projects directory: /home/amar/slots/0/ICMSHh7zg+001632 Moving to projects directory: /home/amar/slots/0/ICMUAh7zg+001632 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMGGh7zg+001536 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMSHh7zg+001536 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMUAh7zg+001536 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMGGh7zg+001548 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMSHh7zg+001548 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMUAh7zg+001548 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMGGh7zg+001560 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMSHh7zg+001560 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMUAh7zg+001560 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMGGh7zg+001572 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMSHh7zg+001572 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMUAh7zg+001572 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMGGh7zg+001584 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMSHh7zg+001584 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMUAh7zg+001584 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMGGh7zg+001596 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMSHh7zg+001596 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMUAh7zg+001596 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMGGh7zg+001608 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMSHh7zg+001608 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMUAh7zg+001608 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMGGh7zg+001620 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMSHh7zg+001620 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMUAh7zg+001620 Zipping up the intermediate file: /home/amar/projects/climateprediction.net/oifs_43r3_bl_a4it_2016092300_20_1018_12294188_0_r48016068_16.zip Uploading the intermediate file: upload_file_16.zip 11:07:14 STEP 1633 H= 408:15 +CPU= 21.183 11:07:50 STEP 1634 H= 408:30 +CPU= 20.066 11:08:25 STEP 1635 H= 408:45 +CPU= 20.152 11:09:00 STEP 1636 H= 409:00 +CPU= 20.434 11:09:33 STEP 1637 H= 409:15 +CPU= 20.209 11:10:08 STEP 1638 H= 409:30 +CPU= 20.080 11:10:42 STEP 1639 H= 409:45 +CPU= 20.177 11:11:16 STEP 1640 H= 410:00 +CPU= 20.398 11:11:48 STEP 1641 H= 410:15 +CPU= 19.870 11:12:22 STEP 1642 H= 410:30 +CPU= 20.064 11:12:55 STEP 1643 H= 410:45 +CPU= 19.989 11:13:40 STEP 1644 H= 411:00 +CPU= 25.221 Moving to projects directory: /home/amar/slots/0/ICMGGh7zg+001644 Moving to projects directory: /home/amar/slots/0/ICMSHh7zg+001644 Moving to projects directory: /home/amar/slots/0/ICMUAh7zg+001644 11:14:13 STEP 1645 H= 411:15 +CPU= 20.017 11:14:47 STEP 1646 H= 411:30 +CPU= 20.077 11:15:22 STEP 1647 H= 411:45 +CPU= 20.014 11:15:57 STEP 1648 H= 412:00 +CPU= 20.505 11:16:32 STEP 1649 H= 412:15 +CPU= 20.136 11:17:04 STEP 1650 H= 412:30 +CPU= 20.009 11:17:38 STEP 1651 H= 412:45 +CPU= 20.026 11:18:13 STEP 1652 H= 413:00 +CPU= 20.364 11:18:47 STEP 1653 H= 413:15 +CPU= 20.047 11:19:22 STEP 1654 H= 413:30 +CPU= 20.219 11:19:55 STEP 1655 H= 413:45 +CPU= 20.050 11:20:36 STEP 1656 H= 414:00 +CPU= 25.134 Moving to projects directory: /home/amar/slots/0/ICMGGh7zg+001656 Moving to projects directory: /home/amar/slots/0/ICMSHh7zg+001656 Moving to projects directory: /home/amar/slots/0/ICMUAh7zg+001656 11:21:10 STEP 1657 H= 414:15 +CPU= 20.088 11:21:42 STEP 1658 H= 414:30 +CPU= 19.873 11:22:15 STEP 1659 H= 414:45 +CPU= 19.991 11:22:50 STEP 1660 H= 415:00 +CPU= 20.354 11:23:23 STEP 1661 H= 415:15 +CPU= 20.001 11:23:57 STEP 1662 H= 415:30 +CPU= 20.099 11:24:31 STEP 1663 H= 415:45 +CPU= 20.022 11:25:05 STEP 1664 H= 416:00 +CPU= 20.542 11:25:38 STEP 1665 H= 416:15 +CPU= 20.031 11:26:12 STEP 1666 H= 416:30 +CPU= 20.122 11:26:46 STEP 1667 H= 416:45 +CPU= 20.032 11:27:28 STEP 1668 H= 417:00 +CPU= 25.023 Moving to projects directory: /home/amar/slots/0/ICMGGh7zg+001668 Moving to projects directory: /home/amar/slots/0/ICMSHh7zg+001668 Moving to projects directory: /home/amar/slots/0/ICMUAh7zg+001668 11:28:00 STEP 1669 H= 417:15 +CPU= 19.909 11:28:34 STEP 1670 H= 417:30 +CPU= 19.977 11:29:09 STEP 1671 H= 417:45 +CPU= 20.087 11:29:43 STEP 1672 H= 418:00 +CPU= 20.287 11:30:17 STEP 1673 H= 418:15 +CPU= 20.056 11:30:50 STEP 1674 H= 418:30 +CPU= 20.055 11:31:23 STEP 1675 H= 418:45 +CPU= 20.132 11:31:58 STEP 1676 H= 419:00 +CPU= 20.303 11:32:32 STEP 1677 H= 419:15 +CPU= 20.048 11:33:05 STEP 1678 H= 419:30 +CPU= 20.034 11:33:39 STEP 1679 H= 419:45 +CPU= 19.956 11:34:26 STEP 1680 H= 420:00 +CPU= 27.843 Moving to projects directory: /home/amar/slots/0/ICMGGh7zg+001680 Moving to projects directory: /home/amar/slots/0/ICMSHh7zg+001680 Moving to projects directory: /home/amar/slots/0/ICMUAh7zg+001680 11:35:00 STEP 1681 H= 420:15 +CPU= 19.949 11:35:36 STEP 1682 H= 420:30 +CPU= 20.205 11:36:11 STEP 1683 H= 420:45 +CPU= 20.240 11:36:43 STEP 1684 H= 421:00 +CPU= 20.344 11:37:18 STEP 1685 H= 421:15 +CPU= 20.346 11:37:53 STEP 1686 H= 421:30 +CPU= 20.021 11:38:26 STEP 1687 H= 421:45 +CPU= 20.097 11:39:01 STEP 1688 H= 422:00 +CPU= 20.384 11:39:35 STEP 1689 H= 422:15 +CPU= 20.193 11:40:11 STEP 1690 H= 422:30 +CPU= 20.409 11:40:46 STEP 1691 H= 422:45 +CPU= 20.134 11:41:28 STEP 1692 H= 423:00 +CPU= 25.082 Moving to projects directory: /home/amar/slots/0/ICMGGh7zg+001692 Moving to projects directory: /home/amar/slots/0/ICMSHh7zg+001692 Moving to projects directory: /home/amar/slots/0/ICMUAh7zg+001692 11:42:02 STEP 1693 H= 423:15 +CPU= 19.979 11:42:35 STEP 1694 H= 423:30 +CPU= 19.951 11:43:07 STEP 1695 H= 423:45 +CPU= 20.036 11:43:42 STEP 1696 H= 424:00 +CPU= 20.371 11:44:15 STEP 1697 H= 424:15 +CPU= 20.131 11:44:50 STEP 1698 H= 424:30 +CPU= 20.024 11:45:23 STEP 1699 H= 424:45 +CPU= 20.040 11:45:57 STEP 1700 H= 425:00 +CPU= 20.359 11:46:31 STEP 1701 H= 425:15 +CPU= 20.169 11:47:05 STEP 1702 H= 425:30 +CPU= 20.021 11:47:37 STEP 1703 H= 425:45 +CPU= 19.874 11:48:20 STEP 1704 H= 426:00 +CPU= 25.102 Moving to projects directory: /home/amar/slots/0/ICMGGh7zg+001704 Moving to projects directory: /home/amar/slots/0/ICMSHh7zg+001704 Moving to projects directory: /home/amar/slots/0/ICMUAh7zg+001704 11:48:53 STEP 1705 H= 426:15 +CPU= 19.839 11:49:27 STEP 1706 H= 426:30 +CPU= 20.091 11:50:02 STEP 1707 H= 426:45 +CPU= 20.075 11:50:35 STEP 1708 H= 427:00 +CPU= 20.417 11:51:08 STEP 1709 H= 427:15 +CPU= 20.067 11:51:42 STEP 1710 H= 427:30 +CPU= 19.938 11:52:16 STEP 1711 H= 427:45 +CPU= 20.063 11:52:50 STEP 1712 H= 428:00 +CPU= 20.320 11:53:23 STEP 1713 H= 428:15 +CPU= 20.114 11:53:56 STEP 1714 H= 428:30 +CPU= 19.955 11:54:30 STEP 1715 H= 428:45 +CPU= 20.042 11:55:12 STEP 1716 H= 429:00 +CPU= 25.248 Moving to projects directory: /home/amar/slots/0/ICMGGh7zg+001716 Moving to projects directory: /home/amar/slots/0/ICMSHh7zg+001716 Moving to projects directory: /home/amar/slots/0/ICMUAh7zg+001716 11:55:47 STEP 1717 H= 429:15 +CPU= 20.336 11:56:23 STEP 1718 H= 429:30 +CPU= 20.266 11:56:56 STEP 1719 H= 429:45 +CPU= 19.967 11:57:31 STEP 1720 H= 430:00 +CPU= 20.554 11:58:06 STEP 1721 H= 430:15 +CPU= 20.191 11:58:39 STEP 1722 H= 430:30 +CPU= 19.848 11:59:13 STEP 1723 H= 430:45 +CPU= 20.179 11:59:47 STEP 1724 H= 431:00 +CPU= 20.284 12:00:21 STEP 1725 H= 431:15 +CPU= 20.199 12:00:56 STEP 1726 H= 431:30 +CPU= 20.140 12:01:30 STEP 1727 H= 431:45 +CPU= 20.151 12:02:16 STEP 1728 H= 432:00 +CPU= 27.644 Moving to projects directory: /home/amar/slots/0/ICMGGh7zg+001728 Moving to projects directory: /home/amar/slots/0/ICMSHh7zg+001728 Moving to projects directory: /home/amar/slots/0/ICMUAh7zg+001728 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMGGh7zg+001632 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMSHh7zg+001632 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMUAh7zg+001632 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMGGh7zg+001644 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMSHh7zg+001644 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMUAh7zg+001644 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMGGh7zg+001656 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMSHh7zg+001656 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMUAh7zg+001656 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMGGh7zg+001668 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMSHh7zg+001668 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMUAh7zg+001668 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMGGh7zg+001680 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMSHh7zg+001680 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMUAh7zg+001680 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMGGh7zg+001692 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMSHh7zg+001692 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMUAh7zg+001692 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMGGh7zg+001704 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMSHh7zg+001704 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMUAh7zg+001704 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMGGh7zg+001716 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMSHh7zg+001716 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMUAh7zg+001716 Zipping up the intermediate file: /home/amar/projects/climateprediction.net/oifs_43r3_bl_a4it_2016092300_20_1018_12294188_0_r48016068_17.zip Uploading the intermediate file: upload_file_17.zip 12:02:49 STEP 1729 H= 432:15 +CPU= 20.997 12:03:25 STEP 1730 H= 432:30 +CPU= 20.227 12:03:59 STEP 1731 H= 432:45 +CPU= 19.978 12:04:33 STEP 1732 H= 433:00 +CPU= 20.447 12:05:07 STEP 1733 H= 433:15 +CPU= 20.087 12:05:40 STEP 1734 H= 433:30 +CPU= 20.130 12:06:16 STEP 1735 H= 433:45 +CPU= 20.235 12:06:50 STEP 1736 H= 434:00 +CPU= 20.327 12:07:24 STEP 1737 H= 434:15 +CPU= 20.254 12:07:58 STEP 1738 H= 434:30 +CPU= 20.068 12:08:34 STEP 1739 H= 434:45 +CPU= 20.277 12:09:17 STEP 1740 H= 435:00 +CPU= 25.265 Moving to projects directory: /home/amar/slots/0/ICMGGh7zg+001740 Moving to projects directory: /home/amar/slots/0/ICMSHh7zg+001740 Moving to projects directory: /home/amar/slots/0/ICMUAh7zg+001740 12:09:50 STEP 1741 H= 435:15 +CPU= 19.992 12:10:26 STEP 1742 H= 435:30 +CPU= 20.135 12:11:00 STEP 1743 H= 435:45 +CPU= 20.050 12:11:34 STEP 1744 H= 436:00 +CPU= 20.388 12:12:07 STEP 1745 H= 436:15 +CPU= 20.019 12:12:40 STEP 1746 H= 436:30 +CPU= 19.881 12:13:14 STEP 1747 H= 436:45 +CPU= 20.228 12:13:48 STEP 1748 H= 437:00 +CPU= 20.239 12:14:22 STEP 1749 H= 437:15 +CPU= 20.038 12:14:55 STEP 1750 H= 437:30 +CPU= 19.974 12:15:29 STEP 1751 H= 437:45 +CPU= 20.043 12:16:13 STEP 1752 H= 438:00 +CPU= 25.481 Moving to projects directory: /home/amar/slots/0/ICMGGh7zg+001752 Moving to projects directory: /home/amar/slots/0/ICMSHh7zg+001752 Moving to projects directory: /home/amar/slots/0/ICMUAh7zg+001752 12:16:46 STEP 1753 H= 438:15 +CPU= 20.053 12:17:20 STEP 1754 H= 438:30 +CPU= 20.121 12:17:53 STEP 1755 H= 438:45 +CPU= 19.991 12:18:28 STEP 1756 H= 439:00 +CPU= 20.355 12:19:02 STEP 1757 H= 439:15 +CPU= 20.155 12:19:36 STEP 1758 H= 439:30 +CPU= 19.997 12:20:09 STEP 1759 H= 439:45 +CPU= 20.053 12:20:44 STEP 1760 H= 440:00 +CPU= 20.444 12:21:19 STEP 1761 H= 440:15 +CPU= 20.189 12:21:52 STEP 1762 H= 440:30 +CPU= 19.956 12:22:26 STEP 1763 H= 440:45 +CPU= 20.024 12:23:09 STEP 1764 H= 441:00 +CPU= 25.074 Moving to projects directory: /home/amar/slots/0/ICMGGh7zg+001764 Moving to projects directory: /home/amar/slots/0/ICMSHh7zg+001764 Moving to projects directory: /home/amar/slots/0/ICMUAh7zg+001764 12:23:42 STEP 1765 H= 441:15 +CPU= 20.040 12:24:15 STEP 1766 H= 441:30 +CPU= 19.979 12:24:48 STEP 1767 H= 441:45 +CPU= 20.002 12:25:24 STEP 1768 H= 442:00 +CPU= 20.518 12:25:57 STEP 1769 H= 442:15 +CPU= 20.057 12:26:31 STEP 1770 H= 442:30 +CPU= 20.035 12:27:04 STEP 1771 H= 442:45 +CPU= 20.024 12:27:38 STEP 1772 H= 443:00 +CPU= 20.306 12:28:12 STEP 1773 H= 443:15 +CPU= 20.005 12:28:44 STEP 1774 H= 443:30 +CPU= 19.949 12:29:17 STEP 1775 H= 443:45 +CPU= 20.061 12:30:04 STEP 1776 H= 444:00 +CPU= 27.765 Moving to projects directory: /home/amar/slots/0/ICMGGh7zg+001776 Moving to projects directory: /home/amar/slots/0/ICMSHh7zg+001776 Moving to projects directory: /home/amar/slots/0/ICMUAh7zg+001776 12:30:39 STEP 1777 H= 444:15 +CPU= 20.102 12:31:13 STEP 1778 H= 444:30 +CPU= 20.279 12:31:45 STEP 1779 H= 444:45 +CPU= 19.964 12:32:19 STEP 1780 H= 445:00 +CPU= 20.434 12:32:51 STEP 1781 H= 445:15 +CPU= 19.998 12:33:25 STEP 1782 H= 445:30 +CPU= 20.072 12:33:58 STEP 1783 H= 445:45 +CPU= 20.020 12:34:33 STEP 1784 H= 446:00 +CPU= 20.426 12:35:08 STEP 1785 H= 446:15 +CPU= 20.123 12:35:41 STEP 1786 H= 446:30 +CPU= 20.024 12:36:15 STEP 1787 H= 446:45 +CPU= 20.141 12:36:57 STEP 1788 H= 447:00 +CPU= 25.075 Moving to projects directory: /home/amar/slots/0/ICMGGh7zg+001788 Moving to projects directory: /home/amar/slots/0/ICMSHh7zg+001788 Moving to projects directory: /home/amar/slots/0/ICMUAh7zg+001788 12:37:32 STEP 1789 H= 447:15 +CPU= 20.311 12:38:07 STEP 1790 H= 447:30 +CPU= 20.136 12:38:38 STEP 1791 H= 447:45 +CPU= 19.946 12:39:13 STEP 1792 H= 448:00 +CPU= 20.504 12:39:46 STEP 1793 H= 448:15 +CPU= 20.035 12:40:18 STEP 1794 H= 448:30 +CPU= 20.058 12:40:53 STEP 1795 H= 448:45 +CPU= 20.326 12:41:27 STEP 1796 H= 449:00 +CPU= 20.621 12:42:03 STEP 1797 H= 449:15 +CPU= 20.360 12:42:37 STEP 1798 H= 449:30 +CPU= 20.066 12:43:12 STEP 1799 H= 449:45 +CPU= 20.350 12:43:54 STEP 1800 H= 450:00 +CPU= 25.034 Moving to projects directory: /home/amar/slots/0/ICMGGh7zg+001800 Moving to projects directory: /home/amar/slots/0/ICMSHh7zg+001800 Moving to projects directory: /home/amar/slots/0/ICMUAh7zg+001800 12:44:28 STEP 1801 H= 450:15 +CPU= 20.231 12:45:03 STEP 1802 H= 450:30 +CPU= 20.217 12:45:37 STEP 1803 H= 450:45 +CPU= 20.041 12:46:13 STEP 1804 H= 451:00 +CPU= 20.573 12:46:47 STEP 1805 H= 451:15 +CPU= 20.039 12:47:19 STEP 1806 H= 451:30 +CPU= 20.095 12:47:54 STEP 1807 H= 451:45 +CPU= 20.075 12:48:29 STEP 1808 H= 452:00 +CPU= 20.572 12:49:03 STEP 1809 H= 452:15 +CPU= 20.242 12:49:37 STEP 1810 H= 452:30 +CPU= 19.987 12:50:11 STEP 1811 H= 452:45 +CPU= 20.136 12:50:52 STEP 1812 H= 453:00 +CPU= 25.074 Moving to projects directory: /home/amar/slots/0/ICMGGh7zg+001812 Moving to projects directory: /home/amar/slots/0/ICMSHh7zg+001812 Moving to projects directory: /home/amar/slots/0/ICMUAh7zg+001812 12:51:25 STEP 1813 H= 453:15 +CPU= 20.065 12:52:01 STEP 1814 H= 453:30 +CPU= 20.205 12:52:35 STEP 1815 H= 453:45 +CPU= 20.173 12:53:10 STEP 1816 H= 454:00 +CPU= 20.568 12:53:43 STEP 1817 H= 454:15 +CPU= 20.057 12:54:17 STEP 1818 H= 454:30 +CPU= 20.218 12:54:50 STEP 1819 H= 454:45 +CPU= 20.040 12:55:25 STEP 1820 H= 455:00 +CPU= 20.538 12:55:59 STEP 1821 H= 455:15 +CPU= 20.212 12:56:33 STEP 1822 H= 455:30 +CPU= 20.182 12:57:06 STEP 1823 H= 455:45 +CPU= 20.122 12:57:54 STEP 1824 H= 456:00 +CPU= 27.754 Moving to projects directory: /home/amar/slots/0/ICMGGh7zg+001824 Moving to projects directory: /home/amar/slots/0/ICMSHh7zg+001824 Moving to projects directory: /home/amar/slots/0/ICMUAh7zg+001824 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMGGh7zg+001728 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMSHh7zg+001728 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMUAh7zg+001728 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMGGh7zg+001740 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMSHh7zg+001740 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMUAh7zg+001740 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMGGh7zg+001752 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMSHh7zg+001752 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMUAh7zg+001752 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMGGh7zg+001764 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMSHh7zg+001764 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMUAh7zg+001764 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMGGh7zg+001776 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMSHh7zg+001776 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMUAh7zg+001776 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMGGh7zg+001788 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMSHh7zg+001788 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMUAh7zg+001788 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMGGh7zg+001800 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMSHh7zg+001800 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMUAh7zg+001800 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMGGh7zg+001812 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMSHh7zg+001812 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMUAh7zg+001812 Zipping up the intermediate file: /home/amar/projects/climateprediction.net/oifs_43r3_bl_a4it_2016092300_20_1018_12294188_0_r48016068_18.zip Uploading the intermediate file: upload_file_18.zip 12:58:34 STEP 1825 H= 456:15 +CPU= 21.007 12:59:09 STEP 1826 H= 456:30 +CPU= 20.074 12:59:42 STEP 1827 H= 456:45 +CPU= 20.054 13:00:16 STEP 1828 H= 457:00 +CPU= 20.363 13:00:50 STEP 1829 H= 457:15 +CPU= 20.070 13:01:24 STEP 1830 H= 457:30 +CPU= 20.092 13:01:57 STEP 1831 H= 457:45 +CPU= 19.992 13:02:32 STEP 1832 H= 458:00 +CPU= 20.470 13:03:05 STEP 1833 H= 458:15 +CPU= 20.047 13:03:38 STEP 1834 H= 458:30 +CPU= 19.927 13:04:12 STEP 1835 H= 458:45 +CPU= 20.099 13:04:53 STEP 1836 H= 459:00 +CPU= 25.088 Moving to projects directory: /home/amar/slots/0/ICMGGh7zg+001836 Moving to projects directory: /home/amar/slots/0/ICMSHh7zg+001836 Moving to projects directory: /home/amar/slots/0/ICMUAh7zg+001836 13:05:28 STEP 1837 H= 459:15 +CPU= 20.105 13:06:03 STEP 1838 H= 459:30 +CPU= 20.340 13:06:36 STEP 1839 H= 459:45 +CPU= 19.905 13:07:10 STEP 1840 H= 460:00 +CPU= 20.599 13:07:44 STEP 1841 H= 460:15 +CPU= 20.067 13:08:17 STEP 1842 H= 460:30 +CPU= 20.109 13:08:51 STEP 1843 H= 460:45 +CPU= 19.950 13:09:26 STEP 1844 H= 461:00 +CPU= 20.552 13:09:59 STEP 1845 H= 461:15 +CPU= 20.016 13:10:33 STEP 1846 H= 461:30 +CPU= 20.104 13:11:08 STEP 1847 H= 461:45 +CPU= 20.172 13:11:49 STEP 1848 H= 462:00 +CPU= 25.021 Moving to projects directory: /home/amar/slots/0/ICMGGh7zg+001848 Moving to projects directory: /home/amar/slots/0/ICMSHh7zg+001848 Moving to projects directory: /home/amar/slots/0/ICMUAh7zg+001848 13:12:22 STEP 1849 H= 462:15 +CPU= 20.116 13:12:55 STEP 1850 H= 462:30 +CPU= 19.990 13:13:28 STEP 1851 H= 462:45 +CPU= 20.144 13:14:03 STEP 1852 H= 463:00 +CPU= 20.581 13:14:36 STEP 1853 H= 463:15 +CPU= 19.969 13:15:10 STEP 1854 H= 463:30 +CPU= 20.123 13:15:45 STEP 1855 H= 463:45 +CPU= 20.140 13:16:20 STEP 1856 H= 464:00 +CPU= 20.730 13:16:54 STEP 1857 H= 464:15 +CPU= 20.003 13:17:29 STEP 1858 H= 464:30 +CPU= 20.283 13:18:02 STEP 1859 H= 464:45 +CPU= 20.062 13:18:47 STEP 1860 H= 465:00 +CPU= 25.147 Moving to projects directory: /home/amar/slots/0/ICMGGh7zg+001860 Moving to projects directory: /home/amar/slots/0/ICMSHh7zg+001860 Moving to projects directory: /home/amar/slots/0/ICMUAh7zg+001860 13:19:21 STEP 1861 H= 465:15 +CPU= 20.177 13:19:54 STEP 1862 H= 465:30 +CPU= 19.956 13:20:29 STEP 1863 H= 465:45 +CPU= 20.146 13:21:04 STEP 1864 H= 466:00 +CPU= 20.473 13:21:38 STEP 1865 H= 466:15 +CPU= 19.946 13:22:12 STEP 1866 H= 466:30 +CPU= 20.183 13:22:45 STEP 1867 H= 466:45 +CPU= 19.967 13:23:20 STEP 1868 H= 467:00 +CPU= 20.474 13:23:54 STEP 1869 H= 467:15 +CPU= 20.058 13:24:28 STEP 1870 H= 467:30 +CPU= 20.206 13:25:01 STEP 1871 H= 467:45 +CPU= 20.115 13:25:49 STEP 1872 H= 468:00 +CPU= 27.970 Moving to projects directory: /home/amar/slots/0/ICMGGh7zg+001872 Moving to projects directory: /home/amar/slots/0/ICMSHh7zg+001872 Moving to projects directory: /home/amar/slots/0/ICMUAh7zg+001872 13:26:24 STEP 1873 H= 468:15 +CPU= 20.202 13:26:57 STEP 1874 H= 468:30 +CPU= 19.968 13:27:31 STEP 1875 H= 468:45 +CPU= 20.112 13:28:06 STEP 1876 H= 469:00 +CPU= 20.492 13:28:40 STEP 1877 H= 469:15 +CPU= 20.044 13:29:12 STEP 1878 H= 469:30 +CPU= 20.041 13:29:47 STEP 1879 H= 469:45 +CPU= 19.979 13:30:21 STEP 1880 H= 470:00 +CPU= 20.494 13:30:55 STEP 1881 H= 470:15 +CPU= 19.965 13:31:30 STEP 1882 H= 470:30 +CPU= 20.464 13:32:05 STEP 1883 H= 470:45 +CPU= 20.472 13:32:53 STEP 1884 H= 471:00 +CPU= 26.023 Moving to projects directory: /home/amar/slots/0/ICMGGh7zg+001884 Moving to projects directory: /home/amar/slots/0/ICMSHh7zg+001884 Moving to projects directory: /home/amar/slots/0/ICMUAh7zg+001884 13:33:31 STEP 1885 H= 471:15 +CPU= 21.082 Quit request received from BOINC client, ending the child process (argv0) ../../projects/climateprediction.net/oifs_43r3_bl_1.13_x86_64-pc-linux-gnu (argv1) start_date: 2016092300 (argv2) exptid: h7zg (argv3) unique_member_id: a4it (argv4) batchid: 1018 (argv5) wuid: 12294188 (argv6) fclen: 20 (argv7) app_name: oifs_43r3_bl (argv8) nthreads: 1 Working directory is: /home/amar/slots/0 Project directory is: /home/amar/projects/climateprediction.net/ app name: oifs_43r3_bl version: 1.13 Location of temp folder: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188 ..mkdir for temp folder for results failed Copying: /home/amar/projects/climateprediction.net/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip to: /home/amar/slots/0/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip Unzipping the app zip file: /home/amar/slots/0/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip Copying the namelist files from: ../../projects/climateprediction.net/jf_684e23235da31c81198f21791492403d to: /home/amar/slots/0/oifs_43r3_bl_a4it_2016092300_20_1018_12294188.zip Unzipping the namelist zip file: /home/amar/slots/0/oifs_43r3_bl_a4it_2016092300_20_1018_12294188.zip ic_ancil_file: ic_ancil_12294188 ifsdata_file: ifsdata_12294188 climate_data_file: clim_data_12294188 horiz_resolution: 159 vert_resolution: 91 grid_type: l_2 upload_interval: 96 utstep: 900 nfrres: restart dump frequency (steps) 48 Copying IC ancils from: ../../projects/climateprediction.net/jf_f7ecfd27aa1e68bbe7e8d19800e25c73 to: /home/amar/slots/0/ic_ancil_12294188.zip Unzipping the IC ancils zip file: /home/amar/slots/0/ic_ancil_12294188.zip ..mkdir for ifsdata folder failed Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_14fb2bf2b3b7d65d1fa89aef6dfd0a7e to: /home/amar/slots/0/ifsdata/ifsdata_12294188.zip Unzipping the ifsdata_zip file: /home/amar/slots/0/ifsdata/ifsdata_12294188.zip ..mkdir for the climate data folder failed Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /home/amar/slots/0/159l_2/clim_data_12294188.zip Unzipping the climate data zip file: /home/amar/slots/0/159l_2/clim_data_12294188.zip Checking for progress XML file: /home/amar/slots/0/progress_file_12294188.xml Opened progress file ok : /home/amar/slots/0/progress_file_12294188.xml -- Model is restarting -- Adjusting last_iter, 1886, to previous model restart step. Creating progress file: /home/amar/slots/0/progress_file_12294188.xml last_cpu_time: 39921 upload_file_number: 19 last_iter: 1873 last_upload: 1641600 model_completed: 0 total_length_of_simulation: 1728000 result_base_name: oifs_43r3_bl_a4it_2016092300_20_1018_12294188_0_r48016068 The child process has been launched with process id: 94224 Executing the command: /home/amar/slots/0/oifs_43r3_model.exe [EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1 [EC_DRHOOK:amar-HP-ENVY:1:1:94224:94224] [20240618:052540:1718706340:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2091] fp = 0x314f440 [EC_DRHOOK:amar-HP-ENVY:1:1:94224:94224] [20240618:052540:1718706340:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1 [EC_DRHOOK:amar-HP-ENVY:1:1:94224:94224] [20240618:052540:1718706340:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0) [EC_DRHOOK:amar-HP-ENVY:1:1:94224:94224] [20240618:052540:1718706340:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1 [EC_DRHOOK:amar-HP-ENVY:1:1:94224:94224] [20240618:052540:1718706340:0.001] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000 [EC_DRHOOK:amar-HP-ENVY:1:1:94224:94224] [20240618:052540:1718706340:0.001] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647) [EC_DRHOOK:amar-HP-ENVY:1:1:94224:94224] [20240618:052540:1718706340:0.001] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16 [EC_DRHOOK:amar-HP-ENVY:1:1:94224:94224] [20240618:052540:1718706340:0.001] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0 [EC_DRHOOK:amar-HP-ENVY:1:1:94224:94224] [20240618:052540:1718706340:0.001] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500 [EC_DRHOOK:amar-HP-ENVY:1:1:94224:94224] [20240618:052540:1718706340:0.001] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1 [EC_DRHOOK:amar-HP-ENVY:1:1:94224:94224] [20240618:052540:1718706340:0.001] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1 [EC_DRHOOK:amar-HP-ENVY:1:1:94224:94224] [20240618:052540:1718706340:0.001] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1 [EC_DRHOOK:amar-HP-ENVY:1:1:94224:94224] [20240618:052540:1718706340:0.001] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1 [EC_DRHOOK:amar-HP-ENVY:1:1:94224:94224] [20240618:052540:1718706340:0.001] [ignore_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef> [EC_DRHOOK:amar-HP-ENVY:1:1:94224:94224] [20240618:052540:1718706340:0.001] [restore_default_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef> [EC_DRHOOK:amar-HP-ENVY:1:1:94224:94224] [20240618:052540:1718706340:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x820cf0 (old at 0x1cf8da0) [EC_DRHOOK:amar-HP-ENVY:1:1:94224:94224] [20240618:052540:1718706340:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x820cf0 (old at (nil)) [EC_DRHOOK:amar-HP-ENVY:1:1:94224:94224] [20240618:052540:1718706340:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x820cf0 (old at 0x1cf8da0) [EC_DRHOOK:amar-HP-ENVY:1:1:94224:94224] [20240618:052540:1718706340:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x820cf0 (old at (nil)) [EC_DRHOOK:amar-HP-ENVY:1:1:94224:94224] [20240618:052540:1718706340:0.001] [trapfpe_treatment@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1 [EC_DRHOOK:amar-HP-ENVY:1:1:94224:94224] [20240618:052540:1718706340:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x820cf0 (old at 0x1cf8da0) [EC_DRHOOK:amar-HP-ENVY:1:1:94224:94224] [20240618:052540:1718706340:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x820cf0 (old at 0x1cf8da0) [EC_DRHOOK:amar-HP-ENVY:1:1:94224:94224] [20240618:052540:1718706340:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x820cf0 (old at (nil)) [EC_DRHOOK:amar-HP-ENVY:1:1:94224:94224] [20240618:052540:1718706340:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x820cf0 (old at 0x1cf8da0) [EC_DRHOOK:amar-HP-ENVY:1:1:94224:94224] [20240618:052540:1718706340:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x820cf0 (old at 0x1cf8da0) [EC_DRHOOK:amar-HP-ENVY:1:1:94224:94224] [20240618:052540:1718706340:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x820cf0 (old at 0x1cf8da0) [EC_DRHOOK:amar-HP-ENVY:1:1:94224:94224] [20240618:052540:1718706340:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x820cf0 (old at (nil)) [EC_DRHOOK:amar-HP-ENVY:1:1:94224:94224] [20240618:052540:1718706340:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x820cf0 (old at (nil)) [EC_DRHOOK:amar-HP-ENVY:1:1:94224:94224] [20240618:052540:1718706340:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x820cf0 (old at (nil)) [EC_DRHOOK:amar-HP-ENVY:1:1:94224:94224] [20240618:052540:1718706340:0.001] [catch_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef> Suspend request received from the BOINC client, suspending the child process Resuming the child process 05:27:28 STEP 1872 H= 468:00 +CPU= 29.292 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 05:30:17 STEP 1873 H= 468:15 +CPU= 28.586 05:31:29 STEP 1874 H= 468:30 +CPU= 29.051 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 05:34:12 STEP 1875 H= 468:45 +CPU= 29.575 05:35:24 STEP 1876 H= 469:00 +CPU= 29.377 05:36:34 STEP 1877 H= 469:15 +CPU= 29.681 Suspend request received from the BOINC client, suspending the child process Resuming the child process 05:38:05 STEP 1878 H= 469:30 +CPU= 28.535 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 05:40:23 STEP 1879 H= 469:45 +CPU= 30.510 05:41:25 STEP 1880 H= 470:00 +CPU= 28.783 05:42:36 STEP 1881 H= 470:15 +CPU= 29.825 05:43:36 STEP 1882 H= 470:30 +CPU= 28.990 05:44:49 STEP 1883 H= 470:45 +CPU= 29.020 05:46:17 STEP 1884 H= 471:00 +CPU= 36.874 Moving to projects directory: /home/amar/slots/0/ICMGGh7zg+001884 Moving to projects directory: /home/amar/slots/0/ICMSHh7zg+001884 Moving to projects directory: /home/amar/slots/0/ICMUAh7zg+001884 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 05:48:05 STEP 1885 H= 471:15 +CPU= 29.283 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 05:50:03 STEP 1886 H= 471:30 +CPU= 28.311 05:51:12 STEP 1887 H= 471:45 +CPU= 28.196 05:52:24 STEP 1888 H= 472:00 +CPU= 29.581 05:53:33 STEP 1889 H= 472:15 +CPU= 29.581 05:54:38 STEP 1890 H= 472:30 +CPU= 28.313 05:55:43 STEP 1891 H= 472:45 +CPU= 28.267 05:56:57 STEP 1892 H= 473:00 +CPU= 29.033 Suspend request received from the BOINC client, suspending the child process Resuming the child process 05:58:31 STEP 1893 H= 473:15 +CPU= 29.549 05:59:43 STEP 1894 H= 473:30 +CPU= 28.714 06:00:48 STEP 1895 H= 473:45 +CPU= 28.711 06:02:13 STEP 1896 H= 474:00 +CPU= 37.035 Moving to projects directory: /home/amar/slots/0/ICMGGh7zg+001896 Moving to projects directory: /home/amar/slots/0/ICMSHh7zg+001896 Moving to projects directory: /home/amar/slots/0/ICMUAh7zg+001896 06:03:24 STEP 1897 H= 474:15 +CPU= 30.037 06:04:32 STEP 1898 H= 474:30 +CPU= 30.651 06:05:31 STEP 1899 H= 474:45 +CPU= 27.775 06:06:38 STEP 1900 H= 475:00 +CPU= 28.822 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 06:09:52 STEP 1901 H= 475:15 +CPU= 30.681 06:11:01 STEP 1902 H= 475:30 +CPU= 29.918 06:11:59 STEP 1903 H= 475:45 +CPU= 27.840 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 06:14:51 STEP 1904 H= 476:00 +CPU= 29.136 06:16:00 STEP 1905 H= 476:15 +CPU= 28.472 Suspend request received from the BOINC client, suspending the child process Resuming the child process 06:17:20 STEP 1906 H= 476:30 +CPU= 28.861 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 06:20:04 STEP 1907 H= 476:45 +CPU= 29.197 06:21:28 STEP 1908 H= 477:00 +CPU= 36.198 Moving to projects directory: /home/amar/slots/0/ICMGGh7zg+001908 Moving to projects directory: /home/amar/slots/0/ICMSHh7zg+001908 Moving to projects directory: /home/amar/slots/0/ICMUAh7zg+001908 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 06:24:14 STEP 1909 H= 477:15 +CPU= 30.258 Suspend request received from the BOINC client, suspending the child process Resuming the child process 06:25:35 STEP 1910 H= 477:30 +CPU= 28.539 06:26:48 STEP 1911 H= 477:45 +CPU= 30.003 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 06:29:35 STEP 1912 H= 478:00 +CPU= 29.307 06:30:44 STEP 1913 H= 478:15 +CPU= 28.759 06:31:56 STEP 1914 H= 478:30 +CPU= 29.171 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 06:33:54 STEP 1915 H= 478:45 +CPU= 29.376 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 06:35:45 STEP 1916 H= 479:00 +CPU= 28.562 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 06:38:17 STEP 1917 H= 479:15 +CPU= 31.679 Suspend request received from the BOINC client, suspending the child process Resuming the child process 06:39:45 STEP 1918 H= 479:30 +CPU= 28.169 06:40:55 STEP 1919 H= 479:45 +CPU= 28.776 06:42:20 STEP 1920 H= 480:00 +CPU= 37.389 ..The child process terminated with status: 0 >>> Printing last 70 lines from file: NODE.001_01 DDH------------------------------------------------------------------------ I I I TIME = 480.000H DDH * Diagnostic Var., AREA GLOBAL I I I I-------------------------------------------------------------------------I I VARIABLE I Initial value I Current value I I-------------------------------------------------------------------------I I OMEGA (P/S) I 0.2820318887E-21 I 0.5556376202E-07 I ICloud fractionI 0.0000000000E+00 I 0.2024427794E-02 I I Relat. Hum. I 0.3929719276E+00 I 0.4033999369E+00 I I PASS 01 I 0.0000000000E+00 I 0.3787295608E+02 I DDH------------------------------------------------------------------------ MAXGPFV : MAX. VALUE = 0.000000000000000E+000 MAXGPFV : MAX. VALUE = 0.000000000000000E+000 NSTEP = 1920 SCAN2M_HPOS P IO-STREAM SETUP - IOTYPE = 2 NUMIOPROCS = 1 CPATH = ICMGGh7zg+001920 MODE=a IO-STREAM CLOSED - ICMGGh7zg+001920 MPI-TASK: 1 - 304372724 BYTES IN 7 RECORDS TRANSFERRED IN 0.0008 SECONDS380465.9 Mbytes/s, TOTAL TIME= 0.0427( 1.9%) 06:42:20 STEP 1920 H= 480:00 +CPU= 37.389 END CNT3 NSTEP = 1920 CNT0 000000000 27.549 27.549 65.093 0.423 0 0 0 0 0 IO-STREAM STATISTICS, TOTAL NO OF REC/BYTES READ 46/ 1069125708 WRITTEN 1430/************ -TASK-OPENED-OPEN-RECS IN -KBYTE IN -RECS OUT-KBYTE OUT-WALL -WALL IN-WALL OU-TOT IN -TOT OUT 1 28 0 46 1044068 1430 ************ 30.7 23.8 6.9 25.9 13.8 ===-=== START OF TIMING STATISTICS ===-=== STATS FOR ALL TASKS NUM ROUTINE CALLS MEAN(ms) MAX(ms) FRAC(%) UNBAL(%) 0 CNT0 - COMPLETE EXECUTION 1 4599593.7 4599593.7 100.00 0.00 1 CNT4 - FORWARD INTEGRATION 1 4569171.2 4569171.2 99.34 0.00 8 SCAN2M - GRID-POINT DYNAMICS 53 17217.2 17217.2 19.84 0.00 9 SPCM - SPECTRAL COMP. 48 2790.2 2790.2 2.91 0.00 10 SCAN2M - PHYSICS 49 38003.7 38003.7 40.49 0.00 11 IOPACK - OUTPUT P.P. RESULTS 4 2466.1 2466.1 0.21 0.00 12 SPNORM - SPECTRAL NORM COMP. 3 78.4 78.4 0.01 0.00 17 GRIDFPOS IN CNT4 4 73.9 73.9 0.01 0.00 27 RESTART FILES - WRITING 1 9864.2 9864.2 0.21 0.00 28 RESTART FILES - READING 1 28203.5 28203.5 0.61 0.00 29 SU4FPOS IN CNT4 4 0.1 0.1 0.00 0.00 30 DYNFPOS IN CNT4 4 13476.3 13476.3 1.17 0.00 31 POSDDH IN STEPO 1 1.0 1.0 0.00 0.00 37 CPGLAG - SL COMPUTATIONS 49 23737.0 23737.0 25.29 0.00 39 SU0YOMB 1 1280.9 1280.9 0.03 0.00 51 SCAN2M - SL COMM. PART 1 49 68.4 68.4 0.07 0.00 54 SPCM - M TO S/S TO M TRANSP. 48 936.6 936.6 0.98 0.00 55 SPCIMPF - S TO M/M TO S TRANSP. 48 302.5 302.5 0.32 0.00 56 SPNORM - SPECTRAL NORM COMM. 3 0.4 0.4 0.00 0.00 102 LTINV_CTL - INVERSE LEGENDRE TRANSFORM 108 1418.2 1418.2 3.33 0.00 103 LTDIR_CTL - DIRECT LEGENDRE TRANSFORM 100 667.2 667.2 1.45 0.00 106 FTDIR_CTL - DIRECT FOURIER TRANSFORM 100 241.0 241.0 0.52 0.00 107 FTINV_CTL - INVERSE FOURIER TRANSFORM 108 600.3 600.3 1.41 0.00 140 SULEG - COMP. OF LEGENDRE POL. 1 203.3 203.3 0.00 0.00 152 LTINV_CTL - M TO L TRANSPOSITION 108 355.8 355.8 0.84 0.00 153 LTDIR_CTL - L TO M TRANSPOSITION 100 75.5 75.5 0.16 0.00 157 FTINV_CTL - L TO G TRANSPOSITION 108 123.3 123.3 0.29 0.00 158 FTDIR_CTL - G TO L TRANSPOSITION 100 58.4 58.4 0.13 0.00 400 GSTATS 6667 0.0 0.0 0.00 0.00 401 GSTATS HOOK 6149 0.0 0.0 0.00 0.00 TOTAL MEASURED IMBALANCE = 0.0 SECONDS, 0.0 PERCENT TOTAL WALLCLOCK TIME 4599.6 CPU TIME 1464.7 VECTOR TIME 1464.7 ===-=== END OF TIMING STATISTICS ===-=== FORECAST DAYS PER DAY 378.2 *** END CNT0 *** ------------------------------------------------ Moving to projects directory: /home/amar/slots/0/ICMGGh7zg+001920 Moving to projects directory: /home/amar/slots/0/ICMSHh7zg+001920 Moving to projects directory: /home/amar/slots/0/ICMUAh7zg+001920 Adding to the zip: /home/amar/slots/0/NODE.001_01 Adding to the zip: /home/amar/slots/0/ifs.stat Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMGGh7zg+001872 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMUAh7zg+001872 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMSHh7zg+001860 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMSHh7zg+001872 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMGGh7zg+001908 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMUAh7zg+001860 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMSHh7zg+001884 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMSHh7zg+001836 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMSHh7zg+001896 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMGGh7zg+001848 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMSHh7zg+001824 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMUAh7zg+001824 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMUAh7zg+001908 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMUAh7zg+001848 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMGGh7zg+001920 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMGGh7zg+001824 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMSHh7zg+001848 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMGGh7zg+001884 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMUAh7zg+001884 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMSHh7zg+001920 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMGGh7zg+001836 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMGGh7zg+001896 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMUAh7zg+001896 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMGGh7zg+001860 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMSHh7zg+001908 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMUAh7zg+001920 Adding to the zip: /home/amar/projects/climateprediction.net/oifs_43r3_bl_12294188/ICMUAh7zg+001836 Zipping up the final file: /home/amar/projects/climateprediction.net/oifs_43r3_bl_a4it_2016092300_20_1018_12294188_0_r48016068_19.zip Uploading the final file: upload_file_19.zip Uploading trickle at timestep: 1727100 06:46:20 (94208): called boinc_finish(0) </stderr_txt> ]]> |
Latest Trickles Received | ||||||
---|---|---|---|---|---|---|
Time Sent (UTC) | Host ID | Result ID | Result Name | Timestep | CPU Time (sec) | Average (sec/TS) |
18 Jun 2024 14:10:38 | 1542649 | 22447805 | oifs_43r3_bl_a4it_2016092300_20_1018_12294188_0 | 1,727,100 | 41,385 | 0.0240 |
15 Jun 2024 18:36:15 | 1542649 | 22447805 | oifs_43r3_bl_a4it_2016092300_20_1018_12294188_0 | 864,000 | 20,870 | 0.0242 |
©2024 cpdn.org